Cross-platform full client ideas

[Fuzzy]

So, I'm thinking that the current miniclient only restriction on linux/mac needs to be solved at some point. In that vein, i started looking at media playback systems, and came across MPV. It appears to be a carry-on from mplayer2 and MPlayer, its open source, and seems pretty easy to plug into a different app (has a message based C API).

Does anyone have any experience with this? Opinions?

[stuckless]

I'm guessing by "full" client you mean a SageTV client, like the windows client (there is no full client for linux or mac).

I've been thinking about creating a cross platform "miniclient" using the Android sources (much of which is portable for a desktop) and using vlc4j for the media engine and javafx for the UI. I'm pretty sure vlc4j runs on all 3 major platforms, and it support creating custom datasources (from java) so, it would be fairly straight forward to integrate with sagetv miniclient apis.

But again... All that is around a miniclient architecture which is very different than creating a full client architecture.

But either way, yeah, I think we need cross platform solution.

[SageWizdom]

Not that I can Java if my life depended on it, but I'm available to test, write docs, cheer on the effort. If the conversation is cross platform, a thought I would offer is ... does this include cross architecture from a planning perspective? A full client as a distro for a Raspberry Pi or other micro P would be amazing.

I know there was an earlier conversation about an RPI as a basic server. I like the idea that there is something simple enough that I could get a couple boxes, some know components and easy set up my relatives house without being 24/7 phone support (aka. it just works).

[Narflex]

One of the other things to consider in this is that ideally you have a layer of integration between the UI rendering system and the media player. The reason for this is so that you can blend graphics on top of the video. On PCs, this has always been done using the 3D rendering system...which means the UI needs to have integration with a 3D rendering system and then the media player needs to render its output to a 3D texture (or to a block of shared memory...which then uses IPC or an in-process mechanism depending upon architecture so then the graphics system can copy the data into a texture) and then composite it in the rendering stage. That's why we added a special vo_stv output plugin to MPlayer which does exactly that...renders the video to a region of shared memory, uses system level mutexes for signaling to the UI renderer, and then the UI system copies that into a texture for compositing.

This little 'feature' of compositing the UI with video has been a HUGE pile of work over the history of SageTV in order to ensure that always works right.

And you *should* be able to run a full SageTV client on Linux already...I haven't done it in a long time; but I know at one point we had that working with the OpenGLRenderer and MPlayer. That's why there's a LinuxMPlayerPlugin in the SageTV core (even though it's used on Windows as well). Although after looking at the code some more...the JOGLSageRenderer is no longer there (I'm sure I can find it though)...and the LinuxMPlayerPlugin looks like it was using overlay for video compositing w/ color key.

OK...well after looking at that further...it's really in a far off state from something that would still work. The OpenGL implementation we used at one point relied on other components which we abandoned long ago.

Something I always wish I had done was make the rendering system and media players pluggable wrt miniclient vs. full client...in theory it's not all that complicated to do it; but it would take a lot of work.

[wayner]

I prefer a miniclient to a full client since it greatly simplifies things like plugins, fanart and configuration which (usually) have to be done separately on each full client. It is much easier to have everything based off of the server as is the case with extenders and PS.

But the problem with PS has always been the shortcomings that don't allow it to play some times of video, like 1080i h.264 files if I remember correctly. And that is the bulk of recordings for me as HD-PVRs are my main tuners. If that is fixed then I would prefer to deploy PS/miniclient rather than full clients, even for full Windows PCs.

[nyplayer]

Originally Posted by wayner

I prefer a miniclient to a full client since it greatly simplifies things like plugins, fanart and configuration which (usually) have to be done separately on each full client. It is much easier to have everything based off of the server as is the case with extenders and PS.

But the problem with PS has always been the shortcomings that don't allow it to play some times of video, like 1080i h.264 files if I remember correctly. And that is the bulk of recordings for me as HD-PVRs are my main tuners. If that is fixed then I would prefer to deploy PS/miniclient rather than full clients, even for full Windows PCs.

The problem with Miniclients as opposed to full clients is the Java Heap.

[wayner]

Originally Posted by nyplayer

The problem with Miniclients as opposed to full clients is the Java Heap.

I know as I have 9 extenders but can't run more than about 3 simultaneously. I am hoping that soon someone will get the 64 bit Java working on Windows in V9 which should solve that problem.

[stanger89]

Originally Posted by wayner

I prefer a miniclient to a full client since it greatly simplifies things like plugins, fanart and configuration which (usually) have to be done separately on each full client. It is much easier to have everything based off of the server as is the case with extenders and PS.

But the problem with PS has always been the shortcomings that don't allow it to play some times of video, like 1080i h.264 files if I remember correctly. And that is the bulk of recordings for me as HD-PVRs are my main tuners. If that is fixed then I would prefer to deploy PS/miniclient rather than full clients, even for full Windows PCs.

Agreed, I think if "placeshifter" were improved to support better decoders/renders that would be a great solution, specifically for the plugins/etc.

Maybe I'm misremembering, or out of date, but I've always had the impression/experience that placeshifter video quality just wasn't there compared to Client or Extenders, especially on Windows. A Windows version of Placeshifter that uses LAV and madVR would be awesome...

Or, Kodi actually has good video quality (not so sure about deinterlacing) and it's cross platform. If Placeshifter could match that, an OpenELEC like distro with Placeshifter would be sweet.

Originally Posted by nyplayer

The problem with Miniclients as opposed to full clients is the Java Heap.

Though with a 64 bit server that's not really a problem I believe.

[Fuzzy]

Originally Posted by Narflex

One of the other things to consider in this is that ideally you have a layer of integration between the UI rendering system and the media player. The reason for this is so that you can blend graphics on top of the video. On PCs, this has always been done using the 3D rendering system...which means the UI needs to have integration with a 3D rendering system and then the media player needs to render its output to a 3D texture (or to a block of shared memory...which then uses IPC or an in-process mechanism depending upon architecture so then the graphics system can copy the data into a texture) and then composite it in the rendering stage. That's why we added a special vo_stv output plugin to MPlayer which does exactly that...renders the video to a region of shared memory, uses system level mutexes for signaling to the UI renderer, and then the UI system copies that into a texture for compositing.

This little 'feature' of compositing the UI with video has been a HUGE pile of work over the history of SageTV in order to ensure that always works right.

And you *should* be able to run a full SageTV client on Linux already...I haven't done it in a long time; but I know at one point we had that working with the OpenGLRenderer and MPlayer. That's why there's a LinuxMPlayerPlugin in the SageTV core (even though it's used on Windows as well). Although after looking at the code some more...the JOGLSageRenderer is no longer there (I'm sure I can find it though)...and the LinuxMPlayerPlugin looks like it was using overlay for video compositing w/ color key.

OK...well after looking at that further...it's really in a far off state from something that would still work. The OpenGL implementation we used at one point relied on other components which we abandoned long ago.

Something I always wish I had done was make the rendering system and media players pluggable wrt miniclient vs. full client...in theory it's not all that complicated to do it; but it would take a lot of work.

That is sort of what interested me in MPV, as it seems to already have a limited UI layer system, and, it being open source, that could possibly be 'added to' in a fork for the client. Of course, I don't know a whole lot about it's full capabilities, as I'm only just looking into it - fishing for more details and opinions was the purpose of this thread.

I also realize that miniclients are less of a limitation now with the 64-bit server in the winds, and UI performance on them for local is much better than it was many years ago with better LAN performance.

That said, I do have a long term reasoning for this. I would really like to have an offline sync capability to a remote client. Already, a full client has almost everything it needs to work offline. I can envision being able to set sync rules, or manually flag files for synching - these could be extensions of the current Convert function - that would copy or convert files to storage local to the client. The client, then, would have synced flags and local url's for the files synced on that particular client - this way, when a client goes offline from the server, it would still be able to navigate the media, and hopefully filter returns down to just locally synced content, and allow playback of such.

Like I said, long-term - and unfortunately, I can software architect a hell of a lot better than I can actually code - so this ultimately ends up being pie in the sky wishes for myself - in hopes that other who can actually help make this happen agree with it's use case.

[jusjoken]

A thought....

Is the code for the standalone mode of the HD300 part of the open source and if so could it be leveraged to make a miniclient that could also work standalone...or is that one off code specific to the HD300?

[SageWizdom]

Originally Posted by wayner

I prefer a miniclient to a full client since it greatly simplifies things like plugins, fanart and configuration which (usually) have to be done separately on each full client. It is much easier to have everything based off of the server as is the case with extenders and PS.

I like the idea of simplified O&M in the form of fewer code bases / systems to support. I also like the idea of a single centralized place to build and test and ensure functionality.

Originally Posted by Fuzzy

That said, I do have a long term reasoning for this. I would really like to have an offline sync capability to a remote client. Already, a full client has almost everything it needs to work offline. I can envision being able to set sync rules, or manually flag files for synching - these could be extensions of the current Convert function - that would copy or convert files to storage local to the client. The client, then, would have synced flags and local url's for the files synced on that particular client - this way, when a client goes offline from the server, it would still be able to navigate the media, and hopefully filter returns down to just locally synced content, and allow playback of such.

I also greatly want this. Offline, sync, decoding at the local machine to provide the best quality.

My question is this... Instead of a server, miniclient and placeshifter, why not a central server, a mini local server and place shifter?

The server already...

• Runs on all the OS's and platforms we're talking about.
• Has its own database
• Works with all the plugins, fan/art, etc
• Is missing database synchronization code... which is already in the client

Given you would have to push /stream all the media anyway, does it matter if its the server code base versus a mini client code base? I think this is how the Linux servers/clients always kinda worked (only not replication).

I imagine some sort of peer to peer server arrangement where one may have the responsibility to record, and maybe access a video repository, but then anyone else could stream from/to anyone else. (v10)

[Fuzzy]

Originally Posted by SageWizdom

I like the idea of simplified O&M in the form of fewer code bases / systems to support. I also like the idea of a single centralized place to build and test and ensure functionality.




I also greatly want this. Offline, sync, decoding at the local machine to provide the best quality.

My question is this... Instead of a server, miniclient and placeshifter, why not a central server, a mini local server and place shifter?

The server already...

• Runs on all the OS's and platforms we're talking about.
• Has its own database
• Works with all the plugins, fan/art, etc
• Is missing database synchronization code... which is already in the client

Given you would have to push /stream all the media anyway, does it matter if its the server code base versus a mini client code base? I think this is how the Linux servers/clients always kinda worked (only not replication).

I imagine some sort of peer to peer server arrangement where one may have the responsibility to record, and maybe access a video repository, but then anyone else could stream from/to anyone else. (v10)

A lot of that is all in sage.jar (which is used for both server and client). Already, if you shutdown the server service, the client will continue to function in many ways without it. You can still navigate videos, etc - just won't play them (even if they are accessible). That, and the local working copy of the database isn't stored when the client shuts down, so it won't work at all starting back up again without a server. This is really only a couple small things to be tweaked to allow offline playback - and a few more things to make it more 'aware' of it, and support local storage. This is the easy part, if we're going to stick to just the windows client - that's why I was wishing for a full client to move to more platforms though.

[SageWizdom]

Maybe I said it badly, but I was technically proposing dropping the full client all together and always running the two separate pieces (server and placeshifter). Alternatively, does everything become a full client. From an evolution perspective, there are three very similar pieces that all require support, what path forward provides the best experience, but simplifies the suport plan?

[Fuzzy]

Originally Posted by SageWizdom

Maybe I said it badly, but I was technically proposing dropping the full client all together and always running the two separate pieces (server and placeshifter). Alternatively, does everything become a full client. From an evolution perspective, there are three very similar pieces that all require support, what path forward provides the best experience, but simplifies the suport plan?

There'd be a significant performance advantage to full clients being able to be used remotely, however, as the UI would be rendered and such locally, instead of back at the server. Really, the only reason miniclients exist is to run on extremely limited hardware, which really isn't even manufactured anymore (MediaMVP, I believe, was the first miniclient). Any hardware available now is ore than capable of rendering the UI and managing the database locally.

[wayner]

Don't you have to do a complete config on full clients, including figuring out which plugins are server plugins and which are client plugins, load all of those plugins on the client and then configure from scratch?

I always found that a major pain in the butt, especially prior to being able to export-import configs that came with Gemstone.

One feature that would be great would be a "clone from server" function where you could clone all of the plugins and config from the server on to the full client.

[Fuzzy]

Originally Posted by wayner

Don't you have to do a complete config on full clients, including figuring out which plugins are server plugins and which are client plugins, load all of those plugins on the client and then configure from scratch?

I always found that a major pain in the butt, especially prior to being able to export-import configs that came with Gemstone.

One feature that would be great would be a "clone from server" function where you could clone all of the plugins and config from the server on to the full client.

The ability to duplicate a configuration from one client to another (or from server to client) is something that was discussed many years ago, shortly after v7, and the plugin system, was released, and there were many who wanted to do so, but it never really came about, primarily because the rapid deployment of HD300 extenders reduced the number of full clients out there significantly, so there really wasn't as much of a market for it (those that kept to the clients, typically were the users least annoyed by configuration steps). I, more recently, started to do a bit of a proof of concept for actually using plugin dependencies as a way to duplicate the plugin download/installs - I essentially parsed through all the plugins on a client, and it created a single plugin manifest that had all installed plugins as dependencies. Then, installing that one plugin brought everything else in. This, of course, could also be done via a more programmatic process, but writing an out of sage utility is easier for me than an in-program one. I don't even have the proof of concept anymore, but it really is not something that would be difficult to create if there is interest out there.

[panteragstk]

Originally Posted by stanger89

A Windows version of Placeshifter that uses LAV and madVR would be awesome...

This is what I've been wanting for a long time. Would prevent me from having to use MPC-HC for movie playback.

I do know that the new Plex Media Player is built on MPV and from what I've seen it's very good.

[stanger89]

MPV sounds interesting.

[Narflex]

Modifying the placeshifter client to also have the same full DirectShow playback capabilities as the full client (when you're not transcoding placeshifted video) isn't all that hard to do actually. The reason we never did it was because it didn't seem like that problem needed to be solved...if you're on your LAN and want good Windows playback; then use the full client. If you're remote; then you likely need stuff transcoded due to bandwidth limitations...so use the Placeshifter (and video quality is already compromised due to transcoding, so not much point in going all out with the media player).

But if people really have an interest in making the Windows Placeshifter playback video in high quality on a LAN like the full client does...then go for it...I can provide all the advice you need to do it. The main thing missing is that when you change the decoder settings and such in the UI; those are done in the properties on the server...which the placeshifter client has no access to...but it likely wouldn't be that hard to include a mechanism that solves that problem (where the placeshifter client can query the server for config settings).

[Taddeusz]

One of the problems I've always had with Placeshifter is that at least on Windows it doesn't (always?) do colorspace conversion correctly. In other words it doesn't make NTSC or ATSC black RGB black and visa versa for white. You end up with a washed out video. At least that happens when doing remote playback. It might be better when playing locally. But I'm not usually using it locally.