Transcoding for miniclient users

[ojones]

I am cross-posting a comment I made in the Sage Github Issues list to see if anyone has any input. In order to make the android miniclient fully functional across fast WAN links (VPNs are easy these days...) changes to a few default assumptions would be enormously useful.

(1) If sage detects bandwidth <10Mbps it forces transcoding. I attempted to contribute some code to bypass this behavior but doesn't appear to be effective.

(2) The bandwidth test doesn't appear to be fully effective (especially over VPN links on asymmetric connections - fails on a 150/25 client-side link and detects bandwidth at ~ 18Mbps).

(3) If forced into transcoding the default ffmeg assumptions result in CIF @ 352x240 and produce a stream at ~0.5 to 1Mbps (~160-200KB/s). Whether or not we can tame the bandwidth forced transcoding it would be very useful to step up the resolution to at least 720p. I know this portion of the plumbing is fairly deep in the bowels of the sage code but hoping tweaking the resolution might be more feasible than switching codecs which I understand is prohibitively complicated.

Any thoughts or votes on priority? I use plex for previously recorded content but for live TV and comskip nothing beats Sage!

https://github.com/google/sagetv/issues/340

[wayner]

Good suggestions. It would also be nice if you didn't get asked for username and password when connected via VPN over WAN. I really like the solution to step up to 720p. When you are watching over a WAN what is your client - an extender, AndroidTV or PlaceShifter? The 1080i bug when streaming to PlaceShifters also needs to be fixed.

I don't mean to highjack the thread but while some thought is being put into transcoding it might also be wise to think about the implications of 4K files streaming to miniclients that can't handle 4K files. I presume that 4K content will start to become more prevalent in the next year or two, especially if ATSC 3.0 tuners start to be adopted. It might be nice to keep source files in 4K but then transcode them when streaming to 1080 clients. That may require a bit of horsepower in the server though so it would be useful to use hardware acceleration like QS where possible.

[ojones]

Thanks wayner! I agree 4K will be the future but my gut says the nvidia shield / miniclient will be the prevalent player as HD 200/300's expire given the codec support for playback will likely stay ahead of sage.

Across the WAN link I use nvidia shield's (the tablet one) for testing but for production we have four sony screens with built in Android TV. I can stream the original files from Plex (no transcode) but can't wait until I can demonstrate comskip/sage live TV at full resolution. Best part is the TV's remote control mapping is near perfect so great WAF once I can get the trascoding killed off or resolution increased...

I've tested the full pc client across the link as well but lag seems to introduce bad audio (stutter)...

[wayner]

Do you have SageTV installed in AndroidTV on the Sony TVs? How does it work and what model TVs are these? I am considering buying a Sony with AndroidTV.

I have an AndroidTV client but I still find the HD200s and HD300s to be more reliable, especially since some of my content is still MPEG-2. These units also seem very durable - I have yet to have an HD200 fail on me although I have one HD300 that acts flakey and they seem to run hotter.

Speaking of your last point - do you also need to worry about latency over the connection as well as bitrate?

[ojones]

Re latency - on one link i have about 60ms to deal with which isn't pretty but should be acceptable once a stream starts playing. My other link is <10ms. Server side I am on fiber (gigabit / symmetric) and I'm running my sage server in a commercial grade server (dual processor; 3+ghz; 12 virtual cores allocated to sage). I agree the HD300's are almost unstoppable. The android player (in my shield) is frankly quite flakey. Lots if jumping around in the time scale of a video - notwithstanding stuckless's excellent work I get the feeling the environment is pretty rough.

I will try to look up the TV specs and forward them - I believe they were purchased Spring 2017 if that helps.

[ojones]

Originally Posted by ojones

I am cross-posting a comment I made in the Sage Github Issues list to see if anyone has any input. In order to make the android miniclient fully functional across fast WAN links (VPNs are easy these days...) changes to a few default assumptions would be enormously useful.

(1) If sage detects bandwidth <10Mbps it forces transcoding. I attempted to contribute some code to bypass this behavior but doesn't appear to be effective.

(2) The bandwidth test doesn't appear to be fully effective (especially over VPN links on asymmetric connections - fails on a 150/25 client-side link and detects bandwidth at ~ 18Mbps).

(3) If forced into transcoding the default ffmeg assumptions result in CIF @ 352x240 and produce a stream at ~0.5 to 1Mbps (~160-200KB/s). Whether or not we can tame the bandwidth forced transcoding it would be very useful to step up the resolution to at least 720p. I know this portion of the plumbing is fairly deep in the bowels of the sage code but hoping tweaking the resolution might be more feasible than switching codecs which I understand is prohibitively complicated.

Any thoughts or votes on priority? I use plex for previously recorded content but for live TV and comskip nothing beats Sage!

https://github.com/google/sagetv/issues/340

It appears that the commit to disable the 10Mbps threshold for no transcoding didn't make it into the windows installer until 9.1.8!!! I was on 9.1.7. On Narflex's advice I checked the version and suddenly this works!. Note - this doesn't help on slow connections and it would be fabulous if we could get transcoding working at a higher bandwidth/resolution than 352x240 but it does provide a bandaid if fast connections were testing incorrectly as I had run into repeatedly.

[jm9843]

Re: the new 'miniplayer/wan_prevent_push' property for client over VPN without transcoding

What kind of upload bandwidth is required for this (assuming a single client with broadcast 1080i MPEG2)? Comcast caps my upstream to 11 Mbps on the Blast tier.

[wayner]

Originally Posted by jm9843

What kind of upload bandwidth is required for this (assuming a single client with broadcast 1080i MPEG2)? Comcast caps my upstream to 11 Mbps on the Blast tier.

Placeshifter doesn't appear to handle 1080i content on your LAN - but broadcast MPEG-2 uses 19Mbps, although it can be less if split amongs sub-channels as well. So you couldn't stream over the internet without transcoding. But MPEG-2 is a very old codec.

Would transcoding fix the 1080i issue, even when doing it on a LAN rather than a WAN.

[ojones]

My fix is to transcode ALL recordings during recording via OpenDCT... before Sage gets it the stream is already 1080p h264 in a mpeg2 TS wrapper. This makes the stream digestible by the android miniclient/placeshifter and cuts bandwidth down to <16mps. 16mbps seams to be the magic number for adequate streaming without further transcoding BUT that is at 1-3ms lag and no packet loss (simulating wan by routing between subsets on my lan in a lab environment). My Fios is 940mb/s upload so gating issue is on the receiving side. Further caveat, it takes some real horsepower to make the opendct trick work... i’m Using a latest generation dual processor Xeon at 3.4ghz with 12 physical cores (24 wHT).

[wayner]

Why does the transcoded file have a bitrate of about 16Mbps? Surely you can get a 19Mbps MPEG-2 file transcoded to about 6Mbps without quality loss given the superior efficiency of H.264 codec.

[ojones]

I have no doubt I could get more compression out of my file - to be honest I haven't played with the source files yet. My primary reason for running the real time transcode on the OpenDCT side of things is to clean up the files so they play in the miniclient. Sounds like a good next step for me to play around with. And a point of clarification - a half hour program runs 3.3GB or 1.833MB/s or 15MB/s. Streaming seems to like an extra mbps of cushion to make the audio sound right

[wayner]

Files from my HD-PVR, which are H.264, appear to be about 3.6GB/hr - and that is on the Great-H.264 setting of the HD-PVR.

Keep us posted on your results, I wouldn't mind doing what you are doing as I have had trouble trying to lock my STBs to 720p. Of course it would be better if we could just get PS fixed.

[ojones]

Wayner - I think we share an interest in getting opendct to compress 1080P a little more than the opendct defaults. Any suggestions on ffmpeg settings for me to try? Ifiddled with some obvious ones to no avail.

[wayner]

I haven't really used OpenDCT - however it increasingly looks like the future for SageTV. I think I installed it to try to get some IP camera streams into SageTV but I never got that working. I just use HD-PVRs and HD-HR tuners directly.

[trk2]

I think ideally the transcoding would be done by Sage's internal ffmpeg rather then force everyone to use OpenDCT. However unless/until Sage's ffmpeg is updated OpenDCT is probably the easiest option.

[ojones]

I am on day #3 after moving to ‘very fast’ and CRF 20 for OpenDCT /ffmpeg reatime transcoding settings. Reduction in file-size is 30-50% 7-11Mbps on average down from 12-16mbps. I am running my OpenDCT instance in a Ubuntu VM on my hyperv box and so far have’t had any known bad recordings. One recording appears to load several vcpu’s to 50-60% with the balance (12 total at 1:1 cores including HT all on one numa) at 15-30%. Tonight I have four simultaneous recordings - crossing my fingers!

[EnterNoEscape]

Originally Posted by trk2

I think ideally the transcoding would be done by Sage's internal ffmpeg rather then force everyone to use OpenDCT. However unless/until Sage's ffmpeg is updated OpenDCT is probably the easiest option.

I completely agree with this. I really want SageTV to have better built in abilities in this area so that all network encoders would benefit and the placeshifter experience could be improved. One could produce a new transcoding engine for SageTV without replacing the one that's already there. This would allow for people to use the old trusted engine and more adventurous people to use the new one to find issues to fix.

[phantomii]

So this is an old thread, and I never used the PlaceShifter before this week and now have some questions. I own two houses, one of which is set up as a partial/part time AirBNB. With Cable bills climbing through the roof and I cancelled my Cable TV in my non AirBNB house. Then I attempted to use the Placeshifter to connect to my AIRBNB (where I will keep cable for guests) to watch television from that house. Got it working from my NVidia Shield quickly and works pretty well. However the Picture quality was lees than I would like to see. The AirBNB house has HDHomerun Prime devices and most channels are read freely so I am using OpenDCT.

Having read through this thread it appears others were looking for ways to improve Placeshifter Quality or maybe disable or improve the transcoding. This said the thread does not look like a solution was reached?

So is there a way to disable the transcoding or improve the picture Quality when using the Placeshifter??? Would love to get this working if it is at all possible.

John

[tvmaster2]

Originally Posted by phantomii

So this is an old thread, and I never used the PlaceShifter before this week and now have some questions. I own two houses, one of which is set up as a partial/part time AirBNB. With Cable bills climbing through the roof and I cancelled my Cable TV in my non AirBNB house. Then I attempted to use the Placeshifter to connect to my AIRBNB (where I will keep cable for guests) to watch television from that house. Got it working from my NVidia Shield quickly and works pretty well. However the Picture quality was lees than I would like to see. The AirBNB house has HDHomerun Prime devices and most channels are read freely so I am using OpenDCT.

Having read through this thread it appears others were looking for ways to improve Placeshifter Quality or maybe disable or improve the transcoding. This said the thread does not look like a solution was reached?

So is there a way to disable the transcoding or improve the picture Quality when using the Placeshifter??? Would love to get this working if it is at all possible.

John

Reduce your source output (cable box, satellite receiver) to 720p. I found Placeshifter is mostly crap trying to move 1080 material. .02 cents

[Tiki]

Originally Posted by tvmaster2

Reduce your source output (cable box, satellite receiver) to 720p. I found Placeshifter is mostly crap trying to move 1080 material. .02 cents

Doesn’t apply to the HDHR devices he is using.