Questions about the Sage Full Client Architecture

[stuckless]

This is mainly for Jeff... but anyone can chime in, if they have the answers

I'm curious about how a full client works with SageTV server. I know lots about the MiniClient, and in that case all the data and the UI is still on the server. But with a full client (which only runs on windows), does it have a copy of the database? If so, how does it go about synchronizing from the Server? For video playback, does it use the same server side playback services as the MiniClient? Does the Full client process a UI in the same way as the MiniClient, ie, via a series of drawing commands? Do you think it's possible to completely create a new full client that does not use AWT in any way? (ie, Oracle has deprecated AWT/Swing in favor of JavaFX -- moving forward their focus will be on JavaFX, although they state that AWT/Swing isn't being removed, yet).

So the motivations here is two fold.

Firstly, understanding how complex the FULL client is, might determine if I ever attempt to create a full client for Android. Creating a FULL client for android would offload the memory requirements from the Server to the Client.

Secondly, if oracle were to suddenly drop support for AWT... how would that affect sagetv? From the code, AWT appears to be woven pretty tightly throughout the codebase for the client and server.

[tmiranda]

I can't answer many of these questions, but I am pretty sure each client has it's own copy of the database. It gets it from the server on startup and then each time there is a change the client and the server sync.

[Narflex]

FYI, full client is possible on Linux as well...I just never bothered with it. It would likely just work if you tried to run it.

Originally Posted by stuckless

But with a full client (which only runs on windows), does it have a copy of the database? If so, how does it go about synchronizing from the Server?

Yes, the client has a full copy of the database in RAM. SageTVConnection is the class that handles pretty much all the RPCs between the server and full client (and it has both directions of communications in it..it was sort of written in an awkward way). On startup, the server sends the full DB from disk to the client and the client loads it. Then, whenever there is a transaction in the DB on the server; it also forwards this to all of the clients. (and there's lots of logic in Wizard to ensure everything is synced if stuff happens during this process). There's a newer parallel update mechanism in V9 that uses what's called 'quanta' in the code to ensure that clients don't get a return from an RPC before they have the proper DB update sent to them (previously the transaction had to be sent to all clients before an RPC would return, but this caused problems on Fiber when a box had connectivity issues as it would induce delays on other clients...so I fixed that problem).

Originally Posted by stuckless

For video playback, does it use the same server side playback services as the MiniClient?

No, it uses the same playback system as standalone SageTV...but it will end up passing 'stv://' URLs to the player (DirectShow or Mplayer). These are handled in the source filter/plugins in DirectShow/Mplayer/FFMPEG and are the same thing you used for pull mode playback in the miniclient. The MediaServer on the server end serves things up for this.

Originally Posted by stuckless

Does the Full client process a UI in the same way as the MiniClient, ie, via a series of drawing commands?

No, the full client works the same way as standalone SageTV. There's a SageRenderer abstract class that has various rendering subclasses (the one you've been looking at is the MiniClientSageRenderer). Ones exist for Java2D, OpenGL, Direct3D and Quartz. It loads the whole STV itself and does all the UI logic and rendering within the client process (so no server load is created when rendering a full client UI, unlike with the miniclient, where the server does all the UI logic and the client just does the rendering).

Originally Posted by stuckless

Do you think it's possible to completely create a new full client that does not use AWT in any way? (ie, Oracle has deprecated AWT/Swing in favor of JavaFX -- moving forward their focus will be on JavaFX, although they state that AWT/Swing isn't being removed, yet).

Yes, this is definitely possible. AWT really isn't used all that much in SageTV except for the top level window and then the Canvas inside of it (with a few other things for doing a custom title bar and another Panel just for containment purposes). And for things like Direct3D/OpenGL rendering in the full client, the 3D system does all the rendering directly to the native handle for the Canvas...so AWT really isn't used for any rendering in those cases. The Java2DSageRenderer of course uses AWT for all of its rendering (but it has more limitations when it comes to video rendering).

Originally Posted by stuckless

So the motivations here is two fold.

Firstly, understanding how complex the FULL client is, might determine if I ever attempt to create a full client for Android. Creating a FULL client for android would offload the memory requirements from the Server to the Client.

Secondly, if oracle were to suddenly drop support for AWT... how would that affect sagetv? From the code, AWT appears to be woven pretty tightly throughout the codebase for the client and server.

You'd need to port a HUGE amount of the SageTV codebase to run on Android to get the full client running there. It'd probably be a much better use of time to make sure 64bit support works properly on Windows to solve the memory problem that way. The beauty of the current full client is that it runs from the exact same codebase as the server...if you move to Android..you're going to have to fork it for API compatibility. I highly doubt Oracle would actually drop AWT...as that would require a massive amount of Java apps to no longer function. But even if they did...it'd likely be pretty easy to resolve that in SageTV.

[stuckless]

Thanks Jeff... that's very helpful. I agree that AWT is not going away, but I thought that SageTV core rendering was a lot more dependent on AWT than what you are saying, so this is good news.

[Fuzzy]

Originally Posted by Narflex

You'd need to port a HUGE amount of the SageTV codebase to run on Android to get the full client running there. It'd probably be a much better use of time to make sure 64bit support works properly on Windows to solve the memory problem that way. The beauty of the current full client is that it runs from the exact same codebase as the server...if you move to Android..you're going to have to fork it for API compatibility. I highly doubt Oracle would actually drop AWT...as that would require a massive amount of Java apps to no longer function. But even if they did...it'd likely be pretty easy to resolve that in SageTV.

I think it's my fault he started looking into this, and for me, the reason wasn't the server memory issue, it was the ability in the future to function offline from the server, to allow the ability to sync content to remote clients. Long term future feature. Obviously this would require a lot more changes than just the use of a full client - but SageTVClient will already function partially with the server out of communication, as long as it's had a chance to get the database first. If the client's wiz implementation was made more aware of it's off-line status, and was able to keep it's last known database in local storage, it might be able to work in an offline state from start-up, and still be able to view items that were synced up prior to it's offlining.

[stuckless]

Originally Posted by Fuzzy

I think it's my fault he started looking into this, and for me, the reason wasn't the server memory issue, it was the ability in the future to function offline from the server, to allow the ability to sync content to remote clients. Long term future feature. Obviously this would require a lot more changes than just the use of a full client - but SageTVClient will already function partially with the server out of communication, as long as it's had a chance to get the database first. If the client's wiz implementation was made more aware of it's off-line status, and was able to keep it's last known database in local storage, it might be able to work in an offline state from start-up, and still be able to view items that were synced up prior to it's offlining.

For me, when I started the MiniClient, I wrestled with should I build a Full Client or do a MiniClient... In the end, I settled on the MiniClient since I figured it would be easier...which I'm sure it is... just some some days doesn't feel quite so easy

I'm thinking that for a Full Client mode, I might just do a pure Android App, that provides access to the video library (probably have the concept of vfs/flows) and then allow syncing and playback there. ie, this would have no correlation to SageTV STVs, but rather it would be a pure native app. I'd have to sync the device's database with the server, but I'd probably not use Wiz.bin, since I don't want to load the entire database into memory on a phone/tablet. I don't think this would be a full client in the sense that you could use it to schedule recordings, view the epg, etc, but rather, it would be a stripped down media client that offered some basic features.

[Fuzzy]

Originally Posted by stuckless

For me, when I started the MiniClient, I wrestled with should I build a Full Client or do a MiniClient... In the end, I settled on the MiniClient since I figured it would be easier...which I'm sure it is... just some some days doesn't feel quite so easy

I'm thinking that for a Full Client mode, I might just do a pure Android App, that provides access to the video library (probably have the concept of vfs/flows) and then allow syncing and playback there. ie, this would have no correlation to SageTV STVs, but rather it would be a pure native app. I'd have to sync the device's database with the server, but I'd probably not use Wiz.bin, since I don't want to load the entire database into memory on a phone/tablet. I don't think this would be a full client in the sense that you could use it to schedule recordings, view the epg, etc, but rather, it would be a stripped down media client that offered some basic features.

I don't see much demand for such a thing, with Plex out there in the wind, and free. It pretty much already does that, and does it quite well.

[Fuzzy]

I'm actually more interested in what would actually need to be 'ported' in this. I'm pretty sure all the UI composition is done in pure java, with theonly platform specific stuff being the native libraries for rendering the UI and video (both of which you've tacked to some extent already with the miniclient), and bringing in Input (also dealt with to some extend in the miniclient). The UI rendering is already modular it seems (hence Jeff's mention of Direct3D and OpenGL renderers) - could probably 'port' your miniclient code into another renderer in that mix. Memory footprint of the program would certainly increase, but so would performance (especially having all the STV content stored locally, and stuff like fanart caching then storing locally as well).

[stuckless]

Originally Posted by Fuzzy

I don't see much demand for such a thing, with Plex out there in the wind, and free. It pretty much already does that, and does it quite well.

I don't use plex (mainly because plex is just a glob of videos without any organization), so if I'm the only one to use it, I'm ok with that as well Keep in mind, everything I work on, has to first benefit me... and if others get benefit from it, then that's just a bonus.

Working on SageTV, for me, is a hobby. It's not my job. So, when I spend a couple hours working on stuff, it needs to have a benefit to me, personally. I know there are lots of people that want to drive the areas get the attention, and they can't understand why someone would work on X when Y needs more attention, but the reality, is that when this isn't your job, you end up working on what brings the most value to you, personally. For example, I could have easily worked on the windows installer (glad jusjoken is doing that), or worked on getting the windows binaries compiled, or adding 64bit to windows, but none of that has any value to me.

I only bring this up, since, I'm sure many people will look at what I choose to work on, and think, "why is he doing that???", and well, this is why

The things are my radar... are finish the miniclient, get jetty 9 working, add a web UI, build a better Android client, and get ARM builds working for the server. Each of those things are things that I vested interest in for one reason or another... it'll likely take me years to get through it (the last one, ARM builds is less important, if I can find a low power, but decent performing, really small form factor x86 board -- NUCs might fit the bill)

[stuckless]

Actually the more I think about a "full" client, I realize I don't need a full client. From my phone, there are 2 things that I do daily, and 1 thing that I do occasionally.
1. Check was was recently recorded (wife asks, "is there anything for us to watch")
2. Check was is going to be recorded
3. Schedule new recordings (occasionally)

From the Kids POV, they only care about browsing and playing videos on demand. They currently use the Placeshifter on PC, but they'd much rather have a netflix like interface, and just watch it in a browser.

Syncing videos to my phone is something I've done rarely, and I can get by with just copying over USB (which is what I tend to do anyways).

So, maybe a better Web UI that adapts to smaller screens is a better focus (for me) than spending any amount of time on a native Android Client.

[Fuzzy]

But if you've already done most of the 'tough' work (the rendering and playback) for sage on the platform, and most the java parts of the client transfer over reasonable well - then why not go the client route, as it would still meet your needs, as well as that of others. Again, this is IF your current work would transfer over - which I think much of it probably would. That will also give you insight into how much of sage.jar transfers cleanly to arm, to aid in your final arm server goal.. :-)

I feel Placeshifter/Miniclient, itself, is really past it's utility at this point. Most hardware that us currently in use for miniclients (especailly going forward) likely could run the full client, without the need for the server to handle the UI. All that is needed is for client to be able to tap into a transcoded stream instead of the untouched one - which should be a relatively minor change.

[Fuzzy]

Unrelated to the actual architecture (full or thin client), I do feel there needs to be a mobile theme - ideally as an alternative/option for gemstone. Some mobile-centric flows and a tweaked menu arrangement would likely be enough. That work would aid any future platforms the UI is ported to.

[stuckless]

Originally Posted by Fuzzy

But if you've already done most of the 'tough' work (the rendering and playback) for sage on the platform, and most the java parts of the client transfer over reasonable well - then why not go the client route, as it would still meet your needs, as well as that of others. Again, this is IF your current work would transfer over - which I think much of it probably would. That will also give you insight into how much of sage.jar transfers cleanly to arm, to aid in your final arm server goal.. :-)

I feel Placeshifter/Miniclient, itself, is really past it's utility at this point. Most hardware that us currently in use for miniclients (especailly going forward) likely could run the full client, without the need for the server to handle the UI. All that is needed is for client to be able to tap into a transcoded stream instead of the untouched one - which should be a relatively minor change.

For me, the extenders were never really about hardware not being able run a full client, but rather, about the centralization of the client and keeping it updated, easily. Before going to SageTV, I ran a MythTV setup with MythClients, and keeping clients updates was painful. I then went to Xmbc, running on $50 used XBoxes, and that worked well, but, again, updating the clients was tedious. So when I stumbled on SageTV and it had the MVP entender I was intriqued, but when it released the HD200 I was sold. I never had to update a client again.

Today, updating clients is less of an issue, especially for something like Android TV (apps can auto-update). So, having a MiniClient or Full Client doesn't matter (to me) either way, since updating the client is pretty easy, or even automatic.

I don't think much of existing MiniClient work actually transfers all well to a full client. How the UI instructions are handled is completely different, although, in the end, a "renderer" is used to do the actual drawing. But I think there is much more in the MiniClient around image provisioning and caching that I don't think works the same way in a Full Client.

Originally Posted by Fuzzy

Unrelated to the actual architecture (full or thin client), I do feel there needs to be a mobile theme - ideally as an alternative/option for gemstone. Some mobile-centric flows and a tweaked menu arrangement would likely be enough. That work would aid any future platforms the UI is ported to.

I agree... i hope that someone does this... it won't be me, since working the STV language is not my thing, but, hopefully someone does this. The challenge I think, is really creating a stripped down client UI. I think you have live with the fact that you probably won't get 100% feature parity with the Full UI. And creating new UIs is tedious work But, if we can offload some operations, like, scheduling, for example to a web ui, then the "client" doesn't need to implement this on day 1. (or the client could delegate some operations to the main STV as Phoenix did for things like plugin management, etc).

[Fuzzy]

Originally Posted by stuckless

I don't think much of existing MiniClient work actually transfers all well to a full client. How the UI instructions are handled is completely different, although, in the end, a "renderer" is used to do the actual drawing. But I think there is much more in the MiniClient around image provisioning and caching that I don't think works the same way in a Full Client.

On a full client, the entire STV is there, so most the images are provisioned from the local filesystem. In any case, most of this work is done in sage.jar, so it likely will 'just work'.

[jusjoken]

Originally Posted by stuckless

I agree... i hope that someone does this... it won't be me, since working the STV language is not my thing, but, hopefully someone does this. The challenge I think, is really creating a stripped down client UI. I think you have live with the fact that you probably won't get 100% feature parity with the Full UI. And creating new UIs is tedious work But, if we can offload some operations, like, scheduling, for example to a web ui, then the "client" doesn't need to implement this on day 1. (or the client could delegate some operations to the main STV as Phoenix did for things like plugin management, etc).

I would like to understand better what people believe a stripped down version of Gemstone really means. Is the file size too large and affects performance so that's why it needs to be stripped down...OR... is it just that people feel the UI is too complex...OR...?

Over time I may take this on but I need to first understand the purpose. One key goal in the past with Gemstone was to leverage that people could still use other plugins as the base of Gemstone is SageTV7.xml STV. This does mean it is LARGE but it helps in the maintenance and flexibility of supporting "most" other plugins.

k

[stuckless]

Originally Posted by Fuzzy

On a full client, the entire STV is there, so most the images are provisioned from the local filesystem. In any case, most of this work is done in sage.jar, so it likely will 'just work'.

My point being, is that the MiniClient work, won't likely help here. While you are are right, in that it might just work, I'd have to port/recompile the native code to ARM/Android (sage uses it's own image loading library that is native), and, the full STV renderers on the server will not run, as is, since to run on Android, you need an opengl es version, which sagetv doesn't have. It would take some work to migrate the work done on Android side to make it work like a "renderer" that full client would use. Not mention that I'd have to strip out any java.awt.* reference in the codebase, since Android doesn't java java.awt. Even though the "drawing" renderers on the server, don't rely on awt directly (except the Java2d one), they all make extensive use of Rectangle and Color objects which don't exist on Android. That was one of the first things I had to do when I moved code to android, was strip out those references and replace with my own objects. This all goes back to Jeff's comment that if I were to do this, I'd likely be taking a copy of the entire SageTV codebase, and then ripping it apart to work on Android (which is exactly what I did for the MiniClient, but it was only a handful of files). In the end it would be a new project.

[Fuzzy]

Originally Posted by jusjoken

I would like to understand better what people believe a stripped down version of Gemstone really means. Is the file size too large and affects performance so that's why it needs to be stripped down...OR... is it just that people feel the UI is too complex...OR...?

Over time I may take this on but I need to first understand the purpose. One key goal in the past with Gemstone was to leverage that people could still use other plugins as the base of Gemstone is SageTV7.xml STV. This does mean it is LARGE but it helps in the maintenance and flexibility of supporting "most" other plugins.

k

I don't think I used the term stripped down. I believe I used the term mobile-centric. The main menu, for instance, with it's three-tiers spreading left to right, works great on a large landscape tv, but on a touch device, is not as useful as if higher tier menus collapsed down when a submenu was navigated into, this leaves more 'finger space' on a small touch screen.

Most of the flows also do not work as well (though some can be done as is with the adjustments that are there). The bigger issue, is I don't believe the flows respond to click events at all, making navigation on a touch screen very limited (only navigating with directional swipes).

[stuckless]

Originally Posted by jusjoken

I would like to understand better what people believe a stripped down version of Gemstone really means. Is the file size too large and affects performance so that's why it needs to be stripped down...OR... is it just that people feel the UI is too complex...OR...?

Over time I may take this on but I need to first understand the purpose. One key goal in the past with Gemstone was to leverage that people could still use other plugins as the base of Gemstone is SageTV7.xml STV. This does mean it is LARGE but it helps in the maintenance and flexibility of supporting "most" other plugins.

k

Personally, I'm not sure a "mobile" friendly STV needs to worry about compatibility with UI plugins. That would likely complicate stuff.

My feeling is that if you pull up the default STV or the Gemstone on a Phone, and interact with it, you get a sense for you don't like The default STV is built with a mouse or remote in mind, so interacting from a Phone using touch is awkward. For example, lists, with scrollbars are not very user friendly. On some places where you need to "click" an icon, the icon an be pretty small and you end up hitting something else.

I'm not 100% sure what a mobile theme should be... I just know what it should not be

To create a truly good experience it might require changes to there server (maybe new commands, actions, events, like handling screen rotation, etc), and definately things in the UI like larger buttons, etc.

[jusjoken]

Originally Posted by Fuzzy

I don't think I used the term stripped down. I believe I used the term mobile-centric. The main menu, for instance, with it's three-tiers spreading left to right, works great on a large landscape tv, but on a touch device, is not as useful as if higher tier menus collapsed down when a submenu was navigated into, this leaves more 'finger space' on a small touch screen.

Most of the flows also do not work as well (though some can be done as is with the adjustments that are there). The bigger issue, is I don't believe the flows respond to click events at all, making navigation on a touch screen very limited (only navigating with directional swipes).

Absolutely the touch part is something that needs work and as well some type of added swipe listener would be a good project. I have the guts of a in place menu in the Quick Launch Menu that could be leveraged for that functionality. Unfortunately I do not have a mobile android device so it may be difficult to convince the wife I need one so I can test Gemstone

k

[Fuzzy]

Originally Posted by jusjoken

Absolutely the touch part is something that needs work and as well some type of added swipe listener would be a good project. I have the guts of a in place menu in the Quick Launch Menu that could be leveraged for that functionality. Unfortunately I do not have a mobile android device so it may be difficult to convince the wife I need one so I can test Gemstone

k

used phone on ebay..

in any case, implementing swipes and drags is a multi-part job. Events need to be created in the server to handle them - the miniclient needs to detect and send them, and the UI needs to handle them. I don't think any of these are impossible - but it will definitely take the work of multiple devs to get it dealt with.