MiniClient running a Browser?

[stuckless]

After working on the Android MiniClient and understanding the layers, etc, I always had a feeling that it would be possible to get a native javascript/html client running in a browser.

A couple days ago, I decided to see if it was possible. I leveraged the Android MiniClient base code that manages connections, and delagates UI rendering, and I created a "WebSocketUIRenderer" that pushed rendering commands to browser.

This is just a proof of concept to see what the performance would be like, etc.

https://www.youtube.com/watch?v=8miFdeMnJt0

This is using the Html5 Canvas object, and the performance is decent. But, I'm not doing all the rendering commands here, AND I'm not doing font coloring. Font coloring when I enabled that, basically disabled animations entirely, as colorizing images on the HTML5 Canvas is expensive.

But, it gives way to possibly creating a WebGL version of the MiniClient. Given my dislike of opengl, I didn't go down that road.

Currently the JavaScript code to make this happen is about 100 lines of code, including whitespace and comments.

I'm not likely to take this much further, until later 2018, but I just wanted to see what was possible.

As it is right now, this "web" client basically connects to a desktop miniclient that manages the connections, etc, and uses websockets to push drawing commands to a browser. So the desktop miniclient is really a bridge to the web ui.

Although I haven't gotten that far, it would be possible to get video playback working as well, but, it would likely only support h264 content.

My initial feeling was this would be something cool to eventually put into SageTV so that when you start it up, especially the first time, you can easily configure it by launching the URL.

[wayner]

Wow - very cool. Would this potentially allow better integration with streaming services like Netflix and Youtube into "SageTV", or at least all within one (somewhat) integrated UI?

The reason I say this is that apparently you may be able to get around the Amazon/Google war limitations using browsers to play back videos rather than native clients.

[stuckless]

Originally Posted by wayner

Wow - very cool. Would this potentially allow better integration with streaming services like Netflix and Youtube into "SageTV", or at least all within one (somewhat) integrated UI?

The reason I say this is that apparently you may be able to get around the Amazon/Google war limitations using browsers to play back videos rather than native clients.

I don't think it would make for any better integration with netflix/youtube, but it might potentially mean that iOS devices could run a full miniclient.

[wayner]

Originally Posted by stuckless

I don't think it would make for any better integration with netflix/youtube, but it might potentially mean that iOS devices could run a full miniclient.

That would be awesome from my perspective as we are pretty much an all-iOS family.

[jusjoken]

Very impressive....

[FreshOne]

Very, very nice

[EnterNoEscape]

Very nice work. I think I hinted that this should be possible about a year ago and you said it wasn't a good idea.

[stuckless]

Originally Posted by EnterNoEscape

Very nice work. I think I hinted that this should be possible about a year ago and you said it wasn't a good idea.

I honestly don't remember that. It seems odd, that I'd say that, since one of the reasons why I choose libgdx at the time, was that it could be ported to the web. Although I didn't end up using that technology here, I've always felt that the browser could run a miniclient UI. Now, I will say, that I don't think we are going to get great video playback, the UI was always something I was planning on doing... Just never found the time, until now.

[reggie14]

This is really cool, though I have to wonder what performance will be like. And I doubt we'll see a truly mouse/touch-friendly STV, so while it might be functional, I assume it will be akin to trying to use the Android miniclient on a phone/tablet today.

The miniclient architecture, as I understand it, seems entirely unsuitable for low-latency actions. When you're using a remote from 10 feet away, you tolerate delays a lot more than touch, or even keyboard/mouse.

Though, it's incredible that you were able to get this much working with just about 100 lines of javascript. I guess that's way easier than building an entirely new webGUI.

[wayner]

But there are uses on your LAN so latency won't be an issue if you just want to use an iPhone or iPad to access SageTV.

[reggie14]

Originally Posted by wayner

But there are uses on your LAN so latency won't be an issue if you just want to use an iPhone or iPad to access SageTV.

Have you tried the Android miniclient? There's definite lag. Again, it's not as disruptive when you're using a remote, but I find it painful to use the miniclient on a phone.

In particular, think of things like scrolling through lists.

[stuckless]

Originally Posted by reggie14

Have you tried the Android miniclient? There's definite lag. Again, it's not as disruptive when you're using a remote, but I find it painful to use the miniclient on a phone.

In particular, think of things like scrolling through lists.

yeah, I don't really see this as replacing a finely tuned web application, but rather could it be used to quickly allow someone to configure a sagetv system from the web. ie, mayber after first install, etc.

The UI would be less feasible on a mobile device, since, touch is tricky, but on a desktop, the sagetv ui already supported mouse, so, pretty much the default stv is very usable with using a mouse.

And to be honest, I needed to learn something new, and I've been wanting to play with websockets for a some time, and this allowed me to do that in a semi productive way

[stuckless]

I decided to take a stab at creating a WebGL version of the renderer in JavaScript... wow, what a difference. It's fast, and I was able to get all the colour blending working correctly.

As is, right now, someone could use this to fully configure a sagetv system using just a browser.



The good news here, to be honest, is that this forced me to learn webgl which is close to opengles which is what is used on Android. So, I might use this knowledge and create a pure opengl renderer for Android and see if that gives slightly better UI performance there.

[wayner]

Does this do video playback or just render the UI? If the latter, do you have to call another application to do the video playback, like you use Exoplayer and IJKplayer in Android? Is that feasible on iOS? How would you do that on iOS? AirVideo HD has a nice playback engine that works well and has a 30 sec skip back and forward function.

[stuckless]

Originally Posted by wayner

Does this do video playback or just render the UI? If the latter, do you have to call another application to do the video playback, like you use Exoplayer and IJKplayer in Android? Is that feasible on iOS? How would you do that on iOS? AirVideo HD has a nice playback engine that works well and has a 30 sec skip back and forward function.

Right now, it's just rendering the UI and handling some basic keystrokes and mouse events. Not even all the rendering commands are there, yet.

For video, if I get there... it'll use the html <video> element and a custom Media Source Extension that knows how to work with push and pull buffers from the server. So, no, it will not use an external player, if the player happens it'll use the browser's native player, which also means that only h264 content will be playable.

[wayner]

Originally Posted by stuckless

So, no, it will not use an external player, if the player happens it'll use the browser's native player, which also means that only h264 content will be playable.

Don't mobile devices now support H.265 content as well? And does that limitation still exist on other OSes like Win or Linux or would they be able to play MPEG-2, etc?

[stuckless]

Originally Posted by wayner

Don't mobile devices now support H.265 content as well? And does that limitation still exist on other OSes like Win or Linux or would they be able to play MPEG-2, etc?

I guess to be correct, the video player might play lots of different kinds of video, but h264 is the only thing that is in the spec, from what I can tell. But sure, depending on the browser and OS, etc, it might do more.