Network encoder

[guho]

I just upgraded from V7 to V9. Works great so far! Thanks. I am using my own network encoder with the R5000-HD. I read on this forum that it is now possible to write network encoders that stream to SageTV directly instead of writing their own .ts file. What should I change in my network encoder to do this? Is it a variation of the START command or a new command? Can you point to updated network encoder protocol docs? Thanks.

[EnterNoEscape]

Originally Posted by guho

I just upgraded from V7 to V9. Works great so far! Thanks. I am using my own network encoder with the R5000-HD. I read on this forum that it is now possible to write network encoders that stream to SageTV directly instead of writing their own .ts file. What should I change in my network encoder to do this? Is it a variation of the START command or a new command? Can you point to updated network encoder protocol docs? Thanks.

You wrote a network encoder for the R5000-HD? That's great to hear and is really good timing. Are you also using network encoder protocol 3.0?

What happens is on START and SWITCH you are given an "upload ID" and you can use that upload ID to authorize opening a file for writing through the SageTV media server. This is the class that does all of that for OpenDCT. That file also shows some of the details on how to use the built in remuxer on the SageTV side of things if you don't want to implement SWITCH or you just want to have SageTV clean your recordings up a little.

All of the magic on the SageTV side of things is in the media server class.

A typical network encoder conversation will go like this:
(S = SageTV to Network Encoder, N = Network Encoder to SageTV, SM = SageTV Media Server to Network Encoder, NM = Network Encoder to SageTV Media Server)

S: VERSION
N: 3.0
S: NOOP
N: OK
S: START Network Encoder Name Digital TV Tuner|1295665805|502|2890245964968|/recordings/RecordingName-1968967-0.ts|Great
(tune in the channel 502 and get the stream)
(connect to media server TCP socket)
NM: WRITEOPEN /recordings/RecordingName-1968967-0.ts 1295665805
SM: OK
N: OK
NM: WRITE 0 32768
(write 32768 bytes into the NM socket; the media server will not reply here, you just keep feeding it)
NM: WRITE 32768 32768
(write 32768 bytes into the NM socket)
NM: WRITE 65536 32768
(write 32768 bytes into the NM socket)
S: SWITCH Network Encoder Name Digital TV Tuner|1496210288|502|/recordings/NextRecordingName-1968967-0.ts|Great
(find an I frame to transition on in the stream; the channel 502 is provided, but SageTV will never tell you to change the channel on SWITCH)
NM: WRITE 98304 8192
(write 8192 bytes into the NM socket)
NM: CLOSE
SM: OK
(re-connect to media server TCP socket; this might not be neccessary)
NM: WRITEOPEN /recordings/NextRecordingName-1968967-0.ts 1496210288
SM: OK
(it is important to not return OK to SageTV until you have closed the previous recording file)
N: OK
NM: WRITE 0 32768
(write 32768 bytes into the NM socket)
NM: WRITE 32768 32768
(write 32768 bytes into the NM socket)
S: STOP Network Encoder Name Digital TV Tuner
(perform any actions needed to correctly stop the streaming from the capture device)
NM: CLOSE
SM: OK
N: OK
S: NOOP
N: OK

There isn't any real documentation on this, you just need to look at the code. I will be eventually implementing a 4.0 protocol and I will be documenting that one.

[guho]

Originally Posted by EnterNoEscape

You wrote a network encoder for the R5000-HD? That's great to hear and is really good timing. Are you also using network encoder protocol 3.0?

Yes I did, as I was experiencing many crashes when using the official R5000 software, especially during the time FIOS TV had glitches in the signal (probably due to a bad ONT).

I am using protocol 2.1. My network encoder uses a text file to map channel numbers to program numbers. I have a Cygwin script that queries my HDHomeRun Prime to populate it. Example file for FIOS enclosed.

Attached is the C++ source code and compiled win32 binary, which I am using on my HTPC with three Moto DCT cable box R5000HDs. It requires the Cypress EZUSB API ("cyioctl.h") to compile.

I am not 100% sure that streaming to the SageTV server instead of into .ts file will provide overall efficiency gains, but it is worth trying.

[EnterNoEscape]

Originally Posted by guho

I am not 100% sure that streaming to the SageTV server instead of into .ts file will provide overall efficiency gains, but it is worth trying.

The efficiency gains isn't why it's great timing. I'm adding a splitter to SageTV that works most efficiently when the network encoder uses the media server. This will address back to back padding when airings are on the same channel so that you can have the padding for both recordings without using other encoders.

There is also some C++ code to use the media server in the SageTV project, but I doubt you would need it based on what I see in your code.

I didn't realize you were getting the full unfiltered stream with all possible programs when using an R5000-HD. I actually have one I bought on eBay already installed a while back that I carefully removed taking many critical pictures of so that I could re-install it on my QIP2500 cable box. I was successful, but I never used it with SageTV and then I moved, so I had to remove it, Nextcom stopped selling the software and haven't got around to doing anything with it since. OpenDCT does similar lookups on demand for ClearQAM mapping.

[guho]

Originally Posted by EnterNoEscape

I didn't realize you were getting the full unfiltered stream with all possible programs when using an R5000-HD.

Yes, I get the full multi-program transport stream so I need the mapping to know which program to isolate. R5000-HD is still the only game in town for DRM (copy once) channels & SageTV. I am able to get the raw multiprogram stream even for MPEG4 channels but somehow there is another level of encryption or coding going on, because I found the mpeg4 streams captured via R5000HD are not playable via VLC media player. If there is someone on this forum who knows about this, I would love to enable mpeg4 capture via R5000HD. For now, I use my HDHomeRun Prime for any mpeg4 recordings.

[EnterNoEscape]

Originally Posted by guho

Yes, I get the full multi-program transport stream so I need the mapping to know which program to isolate. R5000-HD is still the only game in town for DRM (copy once) channels & SageTV. I am able to get the raw multiprogram stream even for MPEG4 channels but somehow there is another level of encryption or coding going on, because I found the mpeg4 streams captured via R5000HD are not playable via VLC media player. If there is someone on this forum who knows about this, I would love to enable mpeg4 capture via R5000HD. For now, I use my HDHomeRun Prime for any mpeg4 recordings.

If you can post a sample somewhere, I can take a look at it and maybe give you an idea of what's going on. If I can figure it out, I just might mod the free SD box I still have and don't really use for anything.

[wayner]

Originally Posted by guho

R5000-HD is still the only game in town for DRM (copy once) channels & SageTV.

Not totally true as the HD-PVR works with SageTV and encrypted channels and any channel and I believe there are also some HDMI capture devices that can capture supposedly encrypted HDMI and there are also (or were) devices that you could use to strip out HDCP.

[EnterNoEscape]

Originally Posted by wayner

Not totally true as the HD-PVR works with SageTV and encrypted channels and any channel and I believe there are also some HDMI capture devices that can capture supposedly encrypted HDMI and there are also (or were) devices that you could use to strip out HDCP.

Yes, but it is the only way to get the original transport stream for DRM content.

[wayner]

Does that provide any particular benefits vs not having the original transport stream?

[EnterNoEscape]

The picture quality is as good as it can be and usually at a lower bitrate. Also for those who care about closed captioning, you get that too. You can't look at the live playback from your STB and the recording being created by the HD-PVR and not tell me the HD-PVR output is a little soft. Also with the original content you don't need to be concerned with the colors being calibrated on your capture device.

[wayner]

I won't argue that the HD-PVR picture is perfect, but it is pretty close. What about some of the other HDMI encoders - are they doing a D-A-D conversion as well? Or does the signal stay digital?

And while we are on the topic of encoders - is there any hope of every getting 4K encoders into SageTV? I believe you have played around with the BM1000 or similar devices. Do they have these types of devices that can accept 4K inputs? The one reason that I could see leaving SageTV is if 4K starts to become more widespread and I upgrade all of my TVs to 4K.

Whenever I ask this question I get responses that the hardware doesn't exist or can't handle 4K. But remember that the HD-PVR came out nine years ago and that 4K today is more prevalent in consumer devices than 1080i was in 2008. I believe the HD-PVR uses encoding chips built for cameras - couldn't a 4K version of the HD-PVR work the same way? Don't most high end phones now have 4K video built in? Not to mention Go-Pros, drones, etc.

[EnterNoEscape]

Originally Posted by wayner

I won't argue that the HD-PVR picture is perfect, but it is pretty close. What about some of the other HDMI encoders - are they doing a D-A-D conversion as well? Or does the signal stay digital?

And while we are on the topic of encoders - is there any hope of every getting 4K encoders into SageTV? I believe you have played around with the BM1000 or similar devices. Do they have these types of devices that can accept 4K inputs? The one reason that I could see leaving SageTV is if 4K starts to become more widespread and I upgrade all of my TVs to 4K.

Whenever I ask this question I get responses that the hardware doesn't exist or can't handle 4K. But remember that the HD-PVR came out nine years ago and that 4K today is more prevalent in consumer devices than 1080i was in 2008. I believe the HD-PVR uses encoding chips built for cameras - couldn't a 4K version of the HD-PVR work the same way? Don't most high end phones now have 4K video built in? Not to mention Go-Pros, drones, etc.

That's the other thing you get when the stream is just digital. I wouldn't be surprised if the R5000-HD handled 4K without any issues.

The HDMI encoders that I'm using are so good I cannot tell the difference between the original stream and the re-encoded stream, but the HD-PVR isn't as good.

4K encoders exist, but they are still in the $800+ range which is too much even if I was crazy interested. When they get down to the $200 range I might take a closer look and maybe 4K will be more predominant. To me 4K is a nice thing, but certainly not a requirement for my daily TV recordings.

[wayner]

Originally Posted by EnterNoEscape

To me 4K is a nice thing, but certainly not a requirement for my daily TV recordings.

That's what people used to say about HD. However most of us only get a handful of channels with far less than 100% of the content on those channels in 4K.

The other issue is that many/most clients can't handle 4K either.

[EnterNoEscape]

Originally Posted by wayner

That's what people used to say about HD. However most of us only get a handful of channels with far less than 100% of the content on those channels in 4K.

The other issue is that many/most clients can't handle 4K either.

I never said that about HD. I've always loved HD, there's an incredibly noticeable benefit and I've always been an advocate for 1080p since it was a standard. It's just that unless I'm watching something I'm completely immersed in like a movie, it doesn't really do much for me. Resolution is also only one part of the puzzle; it means very little if the bitrate or the encoder is inadequate.

[wayner]

Shockingly there appear to be people who still think that about HD - have you noticed that some of the posts here about folks asking questions about analog tuners. I can't believe that folks are still watching/recording SD. The driving force for me switching from MCE to Sage about 9 years ago was that SageTV supported the HD-PVR which was the best solution for HD, particularly in Canada where we couldn't get CableCARDs. The HD-PVR was not supported in MCE for a very long time. And the SageTV extenders were fabulous compared to the alternatives.

The other thing that is going along with 4K is other improvements beyond just rez - like HDR.

[guho]

Originally Posted by EnterNoEscape

If you can post a sample somewhere, I can take a look at it and maybe give you an idea of what's going on. If I can figure it out, I just might mod the free SD box I still have and don't really use for anything.

Here is a sample capture of channel BeIN Sports HDTV:
spts
https://1drv.ms/v/s!AqAlU5UQxL06hOFThMmOwbmmj12DKA
mpts
https://1drv.ms/u/s!AqAlU5UQxL06hOFV-5b9XQu_FfdBzQ

when viewed in VLC or MPC HC, I hear the sound but no video. Hopefully it is something obvious.

[EnterNoEscape]

@guho

I decided that since I have a chance to use my R5000-HD finally thanks to the software you wrote that I would go ahead and wire my box up. There's a gap in the directions I wrote down and it's been long enough that I just don't remember. Can you either post a picture (or two) or describe where specifically the leads are soldered for the power on detection from point to point and how you have IR blasting set up from point to point? I'm sure I would get there eventually, but it will likely be faster to just ask someone who has done this a few times.

I caught your post on AVS (I used to frequent that forum a lot; I actually could have helped you, but I never saw your plea so I'm glad someone else stepped up). Unfortunately the only details you posted there are details that I already had very well photographed and documented.

[EnterNoEscape]

Originally Posted by guho

Here is a sample capture of channel BeIN Sports HDTV:
spts
https://1drv.ms/v/s!AqAlU5UQxL06hOFThMmOwbmmj12DKA
mpts
https://1drv.ms/u/s!AqAlU5UQxL06hOFV-5b9XQu_FfdBzQ

when viewed in VLC or MPC HC, I hear the sound but no video. Hopefully it is something obvious.

So the problem is that stream is scrambled. Everything but one of the audio tracks are flagged back and forth between Scrambled with even key and Scrambled with odd key. I'm not sure if there's a way to get around that and it probably means that eventually you will be unable to record anything since I see that they could flag MPEG-2 content the same way if they wanted to.

Edit: I did read something about CA descriptors in the PMT packets, but I haven't really seen what that looks like so I'm not sure I'd know it if I saw it. Maybe some of our DVB experienced users can help explain how this normally works and maybe I can provide a solution based on that information.

[guho]

Originally Posted by EnterNoEscape

@guho

I decided that since I have a chance to use my R5000-HD finally thanks to the software you wrote that I would go ahead and wire my box up. There's a gap in the directions I wrote down and it's been long enough that I just don't remember. Can you either post a picture (or two) or describe where specifically the leads are soldered for the power on detection from point to point and how you have IR blasting set up from point to point? I'm sure I would get there eventually, but it will likely be faster to just ask someone who has done this a few times.

I caught your post on AVS (I used to frequent that forum a lot; I actually could have helped you, but I never saw your plea so I'm glad someone else stepped up). Unfortunately the only details you posted there are details that I already had very well photographed and documented.

I can't simply attach photos as the limits on photo size are too small on this forum. Admins please fix this if possible.

The IR blasting uses 2k2 resistor and looks like this when soldered in. This picture is of the board behind the STB front panel.
https://1drv.ms/i/s!AqAlU5UQxL06hOFbqys9C0YuLZB3Qw

the power sense goes here:
https://1drv.ms/i/s!AqAlU5UQxL06hOFcHC8mHeBNkOA78Q

I hope it all works out with your installation.

[EnterNoEscape]

Originally Posted by guho

I can't simply attach photos as the limits on photo size are too small on this forum. Admins please fix this if possible.

The IR blasting uses 2k2 resistor and looks like this when soldered in. This picture is of the board behind the STB front panel.
https://1drv.ms/i/s!AqAlU5UQxL06hOFbqys9C0YuLZB3Qw

the power sense goes here:
https://1drv.ms/i/s!AqAlU5UQxL06hOFcHC8mHeBNkOA78Q

I hope it all works out with your installation.

Those are super clear. Thank you.