Android/AndroidTV Placeshifter client (Discussion)

[stuckless]

My goal is to build a fully functioning MiniClient (ie, HD300 like) app for Android for use on Phones, Tablets, and especially Android TV.

NEW: Indiegogo Campaign


This project will likely unfold in a couple of milestones.

Milestone 1 (Progressing)

• Placeshifter can render a SageTV ui, and performance is good.
• We can now see and interact with the SageTV UI on Android devices (tablets, phones, tv)
• There are some rendering glitches, but they'll be solved over time.
• Screen flickers from time to time

Milestone 2 (Progressing)

• Video playback (Working using ExoPlayer and IJKPlayer -- see notes below about each player)
• FF, REW, pause and play are working.
• brewston created a simple video showing the performance on a Nexus Player.

As for Jan 2016 the major functional parts of the MiniClient are complete. The Beta Release will be used to clean up the user experience, and stabilize the client.

Milestone 3 (Not Started)
- Build a "simple" desktop placeshifter using the same base code, but pulling in vlc4j. This would get rid of the mplayer dependency and enable it to work on 32/64bit clients, including Mac.

Milestone 4 (Not Started)
- Depending how (or if) the iOS version is progressing, I might take a stab at using Java on iOS to build an iOS version of the placeshifter. This is possible, since I've already done this with another app. You read my comments about Java on ios here.

Internal Players'
ExoPlayer
- AC3 passthrough (only)
- MPEG2 hardware decoding if the device supports it
- H264 hardware decoding
- H265 hardware decoding if the device supports it
- NO support for MPEG2-PS
- NO support for AVI
- Will play MKV, MP4, FLV, TS (mpegts), MPG containers
- If it has to transcode a file, it will not play (sagetv sends MPEG2-PS stream)

IJKPlayer
- Plays any file type
- NO MPEG2 hardware decoding support, even if the device supports it
- Hardware decoding for h264
- Hardware decoding for h265 (if the device supports it)
- NO AC3 passthrough (it will be decoded and played)

If this all works, then I think that AndroidTV units could be a viable placeshifter/extender supporting all the same features as our HD100-300 extenders.

From an Android Phone perspective, if this works, then I see no need for me to update/enhance the Android apps that I've already done., since this would offer more functionality.

One thing that we would need to consider though, is a "Mobile Friendly' SageTV theme. ie, the existing Themes are likely good for TV, but on a phone they might not be so great, so it would be awesome if someone worked on a Mobile theme (that won't be me )

Wired vs Wireless
This client works over both, but in my experience when wired the UI is as fluid as it would be on an HD300, except that in the client, the UI is rendered at native resolution, whereas the HD300 renders at 1280x720 (I think). Because the UI resolution is likely higher for this client, then it takes more memory, and network bandwidth to create a smooth UI experience. On WiFi the UI works well, but it works much better wired. Likewise, for video playback, wired will most likely provide a better experience. When comparing with the HD300, you should be comparing wired connections in both cases.

Navigation
Tablet/Phone

• Swipe Left/Right/Up/Down on the screen is a LEFT/RIGHT/UP/DOWN keypress
• Touch anywhere on the screen is Mouse Focus
• Double Tap is SELECT/ENTER
• 3 Finger Swipe UP - OPTIONS
• 3 Finger Swipe Left/Right - REW/FF
• Swipe in from LEFT edge will show a menu of buttons for on screen navigation (includes up, down, left, right, home, options, and media controls)
• Swipe in from the RIGHT edge should show overlay of Android Buttons. The Back button will send "BACK" command.
• Back Button sends "BACK" command (also used to exit on screen navigation)

AndroidTV

• UP/DOWN/LEFT/RIGHT - standard navigation
• Middle Button - SELECT/ENTER
• Middle Button LONG PRESS - Bring up on screen navigation (or OPTIONS - determined by preferences)
• MEDIA Pause/Play - Pause/Play
• MEDIA FF/REW - FF/REW
• MEDIA STOP - STOP
• Controller A - SELECT/ENTER
• Controller B - ESCAPE
• Controller X - STOP
• Controller Y - PAUSE/PLAY
• Back Button sends "BACK" command (also exits on screen navigation, or if video is playing, will stop the video)
• Long Press Center Button - OPTIONS or On Screen Navigation (depending on preferences)

If you have Smart Remote enabled in the preferences, then when a video is playing/paused, then the following remote buttons will be in effect.

• LEFT/RIGHT - Skip Back / Skip Forward
• Middle Button - Pause/Play
• MEDIA Pause/Play - Pause/Play
• MEDIA FF/REW - FF/REW
• MEDIA STOP - STOP
• BACK will stop the video playback

Hardware that Works - I've Tested
If it's in this list, then I've verified that the app does run

• Nexus 5 - ARMv7 - Android M (6.0)
• Nexus Player - ARMv7 - 32bit - Developer Edition - Android 5.0
• NVidia Shield TV

Hardware that Works - User Feedback
This is a list of hardware where people have had success in getting the app to function

• Nexus Player
• Fire Stick
• NVidia Shield TV
• Nexus 4
• Samsung Tab S (10.4)
• Nexus 7
• Nexus 7 (mixed results)
• Sony TV (xbr65x850c) with Android TV built in. Wired 10/100 ethernet. (mpeg2ts choppy, mkv fine)
• LG G4
• Matricom G-Box Q
• Hisense Vision Model 40K368AW (TV Device: Works, but has audio/video sync issues)
• FireTV First Gen

Hardware that will not Work (yet)
If it's in the list then someone has said "It doesn't work"

• Samsung Galaxy Tab 8.9 (garbled UI)
• Dell Inpiron Laptop (UI is 1/4 screen size)

Submitting Issues
https://github.com/OpenSageTV/sagetv-miniclient/issues

Source Code
https://github.com/OpenSageTV/sagetv-miniclient

[dinki]

Very excited about this possibility. Do you think, and I know it's early, that this would work on an Amazon FireTV Stick?

[stuckless]

Originally Posted by dinki

Very excited about this possibility. Do you think, and I know it's early, that this would work on an Amazon FireTV Stick?

I don't see why not. The only android specific part of this will be the MediaPlayer and the initial start screen, and I suspect both of those would work on the FireTV. I'm pretty sure with the latest firmware, the FireTV is android 5 API compatible, so it should work.

[dinki]

Great news! I'm pretty thrifty/frugal/cheap. A SageTV client for under $40 is a very attractive thing to me (and perhaps others) especially for kid's rooms, kitchen, etc.

I'm available for testing whenever you get to that stage. I've got a handful of Android devices running various OS versions.

[dranockcir]

Awesome. I have two Android MX2's gathering dust. They aren't AndroidTV but run Kit Kat. I used to use them for XBMC/Kodi and Plex clients. I would donate one to you Sean if you needed for development or whatever. I have no coding skills to help but I sure would love to see an Android Placeshifter!

[stuckless]

Originally Posted by dranockcir

Awesome. I have two Android MX2's gathering dust. They aren't AndroidTV but run Kit Kat. I used to use them for XBMC/Kodi and Plex clients. I would donate one to you Sean if you needed for development or whatever. I have no coding skills to help but I sure would love to see an Android Placeshifter!

Thanks... I have a couple Android TV developer editions that Google sent me... I was using it to create a native SageTV app, but I think the placeshifter is a better way to go.

[cncb]

Originally Posted by stuckless

If this all works, then I think that AndroidTV units could be a viable placeshifter/extender supporting all the same features as our HD100-300 extenders.

I hope this works out because I just ordered a Shield TV and this could be the ultimate Sage extender since it also includes native streaming apps (Netflix, etc.) like many of us always wanted in the HD300.

[cncb]

Originally Posted by dinki

Do you think, and I know it's early, that this would work on an Amazon FireTV Stick?

Do you have 1080i MPEG2 tv signals? When I tried the Fire TV (box) it really struggled with MPEG2 playback/deinterlacing.

[Fuzzy]

Originally Posted by cncb

Do you have 1080i MPEG2 tv signals? When I tried the Fire TV (box) it really struggled with MPEG2 playback/deinterlacing.

Part of the advantage of going the placeshifter route is the transcoding. All the android/ios client would ever have to decode is (currently) MPEG4, and the server would be left to transcode whatever material to that. I'm hoping at some point we can move to a newer ffmpeg on the server to transition to h.264 video instead, which would certainly improve quality.

[dinki]

Originally Posted by cncb

Do you have 1080i MPEG2 tv signals? When I tried the Fire TV (box) it really struggled with MPEG2 playback/deinterlacing.

Yes I do have 1080i ATSC signals and did have trouble with this on my MX2 running Android. Another user on the Matricom forum said he was able to play these without problem using mplayer (I think) but the internal Kodi player had problems with it. I flashed my MX2 with a Linux distro and still had the problem in Kodi but found that reducing my output resolution to 720p fixed the problem for 1080i playback. Note that 720p and 480p did not cause any problems with the MX2 but 1080i did until changing that output resolution.

With all that said, I have not tried playing any 1080i content on the FireTV stick. What were you using to play these files?

[cncb]

Originally Posted by dinki

With all that said, I have not tried playing any 1080i content on the FireTV stick. What were you using to play these files?

I was using Kodi too.

[cncb]

Originally Posted by Fuzzy

Part of the advantage of going the placeshifter route is the transcoding. All the android/ios client would ever have to decode is (currently) MPEG4, and the server would be left to transcode whatever material to that.

I assume direct streaming is possible if the client can handle it?

[Taddeusz]

In general ARM devices don't have the necessary licenses to decode MPEG2 in hardware. In those cases it has to be done in software. Unless you have a really powerful ARM processor that's likely to be a problem.

[Fuzzy]

Originally Posted by cncb

I assume direct streaming is possible if the client can handle it?

Yes, direct streaming is possible via the placeshifter system, but if there are issues with media content playback, it seems just that much 'simpler' to transcode on the server. Also the pass through ffmpeg ensures that the less tolerant decoding systems on most mobile platforms are not botched up by a screwed up timeline, or other recording glitch.

[cncb]

Originally Posted by Taddeusz

In general ARM devices don't have the necessary licenses to decode MPEG2 in hardware. In those cases it has to be done in software. Unless you have a really powerful ARM processor that's likely to be a problem.

Which is why I ordered the Shield TV: apparently it can already do MPEG2 hardware decoding in Kodi. I also think I read that the Nexus player will be able to do this in the next version of Android TV but I'm not sure.

[Narflex]

If you do transcoding...there's a few downsides:

1) Reduces quality (although with high enough bitrate H264, this would be negligible)
2) Latency in transport controls (i.e. skipping around is not as fast because the transcoder needs to restart each time you do this)
3) No Smooth Fast Forward or Rewind (technically this could be achieved, but it's very complicated to do it and would require lots of effort)

[stuckless]

Originally Posted by Narflex

If you do transcoding...there's a few downsides:

1) Reduces quality (although with high enough bitrate H264, this would be negligible)
2) Latency in transport controls (i.e. skipping around is not as fast because the transcoder needs to restart each time you do this)
3) No Smooth Fast Forward or Rewind (technically this could be achieved, but it's very complicated to do it and would require lots of effort)

On an HD300 today... are there cases where you transcode? I've played MKV, MP4, AVI and TS file, and it's always super snappy skipping around... so I'm guessing it's not transcoding.

From a placeshifter point of view, does sagetv know when to transcode, or it is up to the placeshifter to ask for a transcoded stream?

[Taddeusz]

Originally Posted by stuckless

On an HD300 today... are there cases where you transcode? I've played MKV, MP4, AVI and TS file, and it's always super snappy skipping around... so I'm guessing it's not transcoding.

From a placeshifter point of view, does sagetv know when to transcode, or it is up to the placeshifter to ask for a transcoded stream?

Yeah, I was curious if there is some kind of device capabilities parameter for the MiniClient. If I do an Apple TV app it should be able to play progressive H.264. But MPEG2 and interlaced content are probably out.

[Narflex]

Originally Posted by stuckless

On an HD300 today... are there cases where you transcode? I've played MKV, MP4, AVI and TS file, and it's always super snappy skipping around... so I'm guessing it's not transcoding.

From a placeshifter point of view, does sagetv know when to transcode, or it is up to the placeshifter to ask for a transcoded stream?

The HD300 will not transcode if its playing back something on your LAN. When it's remotely connected; it uses bandwidth statistics to determine if it needs to go into transcode mode or not.

Originally Posted by Taddeusz

Yeah, I was curious if there is some kind of device capabilities parameter for the MiniClient. If I do an Apple TV app it should be able to play progressive H.264. But MPEG2 and interlaced content are probably out.

Yes there are. The protocol has the client send back the full list of audio/video codecs and container formats that it supports. Then the server uses this to determine compatibility and whether or not transcoding is required. NOTE however, that we haven't fully verified if this works properly in all cases. We did various special cases in there along the way..BUT the whole logic around this is definitely present so if there are issues, they should not be hard to resolve.

[MattHelm]

Originally Posted by Narflex

If you do transcoding...there's a few downsides:

1) Reduces quality (although with high enough bitrate H264, this would be negligible)
2) Latency in transport controls (i.e. skipping around is not as fast because the transcoder needs to restart each time you do this)
3) No Smooth Fast Forward or Rewind (technically this could be achieved, but it's very complicated to do it and would require lots of effort)

It's also a waste of power, as it runs most CPUs all cores at 100%., were as streaming only uses about 2% of 1 core, with the clock turned way down.