OpenDCT: An open source digital cable tuner.

[ChePazzo]

I just wanted to say, "THANK YOU" for opendct! Shortly after Google bought SageTV, my cable company removed all clearQAM channels and, not having a Windows PC (linux only), I was unable to effectively use my HDHR-Prime with SageTV (there was some talk about how to get it running, but I never got it to work).

I was at a loss. I switched to MythTV as my backend, but MythTV didn't really have any front-end hardware. I tried XBMC, but that was hit or miss (broke every time the myth backend db changed version) and the comskip never really worked. So, I ended up setting up some creative collections of symlinks and crons on the myth server so that I could continue to use my SageTV Extender, but (because it was playing recordings from symlinks), I could only delete recordings from backend, etc, etc, etc.

Enter OpenDCT. I set up a brand new VM on Ubuntu 16.04 and ran the install script, and everything -- just -- worked!

I feel like today, after 5 long lost painful years, I finally got my SageTV back!

thank you!

[EnterNoEscape]

• Writes out larger chunks of data per write on FFmpeg transcoder.
• Added a timing component to checking the file length since it's an expensive operation when we are just trying to catch up.
• Fixed failure to deallocate the input context when detection fails and the buffer is reset.
• Added handling for when Cling is unable to open the requested port.

[EnterNoEscape]

Originally Posted by ChePazzo

I just wanted to say, "THANK YOU" for opendct! Shortly after Google bought SageTV, my cable company removed all clearQAM channels and, not having a Windows PC (linux only), I was unable to effectively use my HDHR-Prime with SageTV (there was some talk about how to get it running, but I never got it to work).

I was at a loss. I switched to MythTV as my backend, but MythTV didn't really have any front-end hardware. I tried XBMC, but that was hit or miss (broke every time the myth backend db changed version) and the comskip never really worked. So, I ended up setting up some creative collections of symlinks and crons on the myth server so that I could continue to use my SageTV Extender, but (because it was playing recordings from symlinks), I could only delete recordings from backend, etc, etc, etc.

Enter OpenDCT. I set up a brand new VM on Ubuntu 16.04 and ran the install script, and everything -- just -- worked!

I feel like today, after 5 long lost painful years, I finally got my SageTV back!

thank you!

You're welcome!

[waylo]

I've noticed an interesting side effect recently, maybe related to the file writing changes implemented in recent versions.

For many .ts files, I am not able to run them in VLC anymore. Other players, such as MPC, work fine. This is not for all, but most.

Not a necessity for VLC to work, but I use it sometimes to check the embedded closed captioning.

I believe this started with 0.5.1.

[EnterNoEscape]

Originally Posted by waylo

I've noticed an interesting side effect recently, maybe related to the file writing changes implemented in recent versions.

For many .ts files, I am not able to run them in VLC anymore. Other players, such as MPC, work fine. This is not for all, but most.

Not a necessity for VLC to work, but I use it sometimes to check the embedded closed captioning.

I believe this started with 0.5.1.

I just did a check on all my recordings from last night and the all played ok in VLC 2.1.5. Are you noticing any particular patterns? Is it H.264 content, 1080i content or maybe just certain channels?

0.5.1 had an optimization put in that makes sure a key video frame always gets written first. It also looks for a key frame before streaming the audio streams. In one that you can play in MPC-HC, but not VLC, what's the audio offset? I don't think getting a key frame is as important for the audio stream, so I could change things so only the video is optimized. For the video, it appears to have made a huge difference at least for nyplayer.

[HelenWeathers]

Originally Posted by waylo

I've noticed an interesting side effect recently, maybe related to the file writing changes implemented in recent versions.

For many .ts files, I am not able to run them in VLC anymore. Other players, such as MPC, work fine. This is not for all, but most.......

I have the same problem with Windows Media Player not playing the .ts files. I've gone back and played files from the last few weeks and the issue seems to have started for me the day I installed OpenDCT_0.4.46_x86. All recordings after that date don't play in WMP in a watchable way.

Variety of channels and video formats. ie last two Vikings episodes don't play but previous 8 play just fine.

They all play well in Sage.

[Daweeze]

Question:

Is there any way that you can present video on a channel via OpenDCT to express the reason it is not presentable? For instance, if I tried to get HBO to tune, instead of no Signal, present the reason like the channel is blocked for the copy flag or whatnot. Example, CMT does not work for me at home, I am not sure if they are blocking due to all of the music videos or whatnot but it would be nice to know if it was a flagging issue. Could you either force the Closed Captioning to state the issue or present a video frame and repeat it over and over to display the code? You are doing some amazing things here and was wondering if it were possible to do this as an extension. I don't even remember how to find a channel copy flag but you wouldn't have to know how to if this were possible.

Thanks!

-Travis

[waylo]

Looking through my install hx, almost all .ts files after a certain point have the VLC difficulty. I believe I installed 0.5.1 right before that point.

Looking through my records and exploring recent recordings, all but 1 of the .ts files could not play in VLC v2.2.2 (unfortunately I seem to have erased that one--that would have been nice to examine more carefully).

There doesn't seem to be much rhyme or reason. They are a myriad of stations. Both H264 and mpeg2 files are affected.

Here are the audio offsets in MPC, and the stations for the files that could not play:
AMC
-1s 140ms.
-996ms

Food network
-875ms

CNN:
-1s36ms

ABC
-874ms

The smallest clip I have is ~800Mb (could definitely get something smaller if I start the recording at the right time), but I could certainly upload it to a site if you wish to examine it closer.

[EnterNoEscape]

Originally Posted by HelenWeathers

I have the same problem with Windows Media Player not playing the .ts files. I've gone back and played files from the last few weeks and the issue seems to have started for me the day I installed OpenDCT_0.4.46_x86. All recordings after that date don't play in WMP in a watchable way.

Variety of channels and video formats. ie last two Vikings episodes don't play but previous 8 play just fine.

They all play well in Sage.

Originally Posted by waylo

Looking through my install hx, almost all .ts files after a certain point have the VLC difficulty. I believe I installed 0.5.1 right before that point.

Looking through my records and exploring recent recordings, all but 1 of the .ts files could not play in VLC v2.2.2 (unfortunately I seem to have erased that one--that would have been nice to examine more carefully).

There doesn't seem to be much rhyme or reason. They are a myriad of stations. Both H264 and mpeg2 files are affected.

Here are the audio offsets in MPC, and the stations for the files that could not play:
AMC
-1s 140ms.
-996ms

Food network
-875ms

CNN:
-1s36ms

ABC
-874ms

The smallest clip I have is ~800Mb (could definitely get something smaller if I start the recording at the right time), but I could certainly upload it to a site if you wish to examine it closer.

Thanks for the feedback. I am able to see the problem in Windows Media Player. I'll need to mess around with a few things to work out what's going on, I have a few ideas.

[nyplayer]

I tested all types of my Comcast recordings which include H.264 720p 1080i ... MPEG-TS both 720P and 1080I and they all play fine in VLC and Windows mediaplayer.... of course I am firmware 20160328atest2.

[EnterNoEscape]

Originally Posted by EnterNoEscape

Thanks for the feedback. I am able to see the problem in Windows Media Player. I'll need to mess around with a few things to work out what's going on, I have a few ideas.

So here's the deal. The old SageTV FFmpeg consumer produces recordings that will playback without issue in WMP. The key difference is the output stream timebase used. The old FFmpeg consumer was using the codec timebase for the stream timebase which is technically incorrect. The standard timebase for MPEG-TS is typically 90khz. SageTV even makes this assumption when it's streaming to a miniclient. The newer transcoding FFmpeg consumer uses 90khz explicitly which produces a file that looks from a technical perspective exactly as it should. Fixing this value made Comskip more consistent too.

Interestingly enough if you run ffmpeg.exe on the same raw content it will produce files with the same issues. The raw file with no modifications at all plays back ok in Windows Media Player smoothly too. Transcoded content doesn't have this issue. MPEG-PS content did not have this issue.

My best guess is just based on the only real significant thing that FFmpeg does with MPEG-TS streams, is it injects what I consider an abnormally large number of PAT and PMT packets into the stream. I tried to get this number to go down to be more in line with what's normal over a month ago, but for whatever reason every time I changed the ratio for it to use, it either got even crazier or far too low. I'll give this another go and see what I can do, but the issue could have more to do with the interleave pattern FFmpeg employs and if that's that the case then I probably will not be fixing it other than providing an option to use the wrong timebase since it counter-intuitively somehow ends up producing more compatible files.

Update: I just made a little progress. It looks like the issue has less to do with the timebase, and more to do with support for VBR MPEG-TS muxing. When I specify an actual CBR muxrate, the problems appear to go away.

[stanger89]

Originally Posted by EnterNoEscape

This doesn't make any sense. You're running the exact same firmware as me and yet I haven't seen this happen once on my setup without the URL being completely wrong or the virtual channel actually not existing.

There's only one more thing that I can think of for you to try. Change sagetv.device.parent.<parent_id>.http.producer to opendct.producer.HTTPProducerImpl in opendct.properties.

For some reason, not it all seems to be working

[EnterNoEscape]

Originally Posted by stanger89

For some reason, not it all seems to be working

Is that with or without the change I suggested?

[EnterNoEscape]

Originally Posted by Daweeze

Question:

Is there any way that you can present video on a channel via OpenDCT to express the reason it is not presentable? For instance, if I tried to get HBO to tune, instead of no Signal, present the reason like the channel is blocked for the copy flag or whatnot. Example, CMT does not work for me at home, I am not sure if they are blocking due to all of the music videos or whatnot but it would be nice to know if it was a flagging issue. Could you either force the Closed Captioning to state the issue or present a video frame and repeat it over and over to display the code? You are doing some amazing things here and was wondering if it were possible to do this as an extension. I don't even remember how to find a channel copy flag but you wouldn't have to know how to if this were possible.

Thanks!

-Travis

You can always get this information from the web interface of the device you're tuning. I could have some very small pre-rendered video clips that would just loop with the message, but wouldn't you rather see an error in SageTV vs. thinking your recordings are actually there? I get that your use case would likely be in regards to live TV watching, but to OpenDCT, recordings and live TV are the same thing. SageTV doesn't tell it which one it's doing.

After saying all of that, I actually think this could be possible on channel preview since OpenDCT can tell when what's happening in that case. Would that interest you?

[brbjr]

How long should it take to tune a channel using the HD Homerun Connect Extend with transcoding?

Right now it takes approx 15 seconds to tune a channel. Is this normal? Tuning on my Prime is approx 3 seconds.

I attached my Opendct programdata folder for review.

[Daweeze]

Originally Posted by EnterNoEscape

...I get that your use case would likely be in regards to live TV watching, but to OpenDCT, recordings and live TV are the same thing. SageTV doesn't tell it which one it's doing.

After saying all of that, I actually think this could be possible on channel preview since OpenDCT can tell when what's happening in that case. Would that interest you?

Yeah, I guess I considered that this was in respect to live TV. Is there a way to sense incompatible flagging with the message for only a few seconds and then dropping the signal so SageTV can sense the No Signal? I would imagine if someone did try to record something and it didn't work that it's better to know why anyway other than guessing. For me (Comcast) I think it's just entire channels that are flagged, not programs. The preview option seems nice. Would that be useable only during channel add/delete and remapping or somewhere else in SageTV? Also, is there still a limit on the number of channels that preview due to the SageTV channel number issue? The CMT channel is in the 700's.

Thanks for any options you might provide for consideration.

I know you were working on the Web Interface a while back so I am sure we will see it in there some time down the road too. You haven't mentioned the interface in a good while though.

-Travis

[stanger89]

Originally Posted by EnterNoEscape

Is that with or without the change I suggested?

Yeah, oddly enough. Guess we'll see how it works.

[EnterNoEscape]

Originally Posted by brbjr

How long should it take to tune a channel using the HD Homerun Connect Extend with transcoding?

Right now it takes approx 15 seconds to tune a channel. Is this normal? Tuning on my Prime is approx 3 seconds.

I attached my Opendct programdata folder for review.

I don't actually know since I don't have one. I was waiting for someone to provide feedback. I can see it's taking about 10 seconds for FFmpeg to positively identify the stream that we want to use. There are some improvements that need to be made when a program can't be provided to FFmpeg, so unfortunately I'll just need to say stay tuned on that one. For now at least I actually think you'd be best served using the raw consumer since the HDHomeRun is already remuxing the content for you. For the Extend devices, just change the values for sagetv.device.<unique_id>.consumer to opendct.consumer.RawSageTVConsumerImpl.

Originally Posted by Daweeze

Yeah, I guess I considered that this was in respect to live TV. Is there a way to sense incompatible flagging with the message for only a few seconds and then dropping the signal so SageTV can sense the No Signal? I would imagine if someone did try to record something and it didn't work that it's better to know why anyway other than guessing. For me (Comcast) I think it's just entire channels that are flagged, not programs. The preview option seems nice. Would that be useable only during channel add/delete and remapping or somewhere else in SageTV? Also, is there still a limit on the number of channels that preview due to the SageTV channel number issue? The CMT channel is in the 700's.

Thanks for any options you might provide for consideration.

I know you were working on the Web Interface a while back so I am sure we will see it in there some time down the road too. You haven't mentioned the interface in a good while though.

-Travis

OpenDCT has a standard method to access the copy protection status of any capture device in use. I'll see if I can do something interesting with that. When a channel doesn't come in, it should be displayed in the log after some effort why it wouldn't come in. Although, I can see the value of instant feedback.

The web interface got put on the back burner because now I'm not sure how I want to proceed. I was thinking about some kind of integration with the new SageTV web interface when that comes out instead of the program having it's own interface. I really don't want to maintain multiple interfaces and I want to keep the fragmentation in SageTV down if possible. This would also take care of the nuisance details of things like putting a username and password on the interface. Building a good web interface definitely doesn't happen over night.

[waynedunham]

Originally Posted by CptKirk

With the problems I'd been having I dug into the server some more yesterday. I found the device manager was constantly refreshing itself. Turned out it was one of the Hauppauge USB tuners appearing/disappearing/appearing/etc/etc.

Got that all fixed and went ahead and updated pretty much all the drivers on the system. Did 5 simultaneous recordings on all tuners. Everything was looking good.

But had a few more halt detected in program late last night afterwards.

I've attached the log files.

Just a question here since I had that issue with one of my HD-PVR's a while back. Have you checked the power supply to that unit? On mine, it would appear and disappear randomly and with increasing frequency.

It turned out that it was a bad power supply for that HD-PVR. The voltage was wavering and once it dropped below a certain threshold the HD-PVR would disappear from the system. When the voltage wandered back up a bit it would reappear, so apparently that power supply was hovering right on the threshold of where the HD-PVR could operate properly.
I checked the voltage coming out of the power supply (wall wart) and it indeed was significantly below the stated voltage.

Once I replaced the power supply all was well again and that HD-PVR is more stable than it had been since it was new so maybe that PS was marginal right out of the box.

[EnterNoEscape]

The MPEG-TS muxrate setting appears to be very fickle in the pre-compiled JavaCPP binary. I can set it up to 9K. If I go beyond 9K to lets say 10K, suddenly FFmpeg interprets it as 997M which will create absurdly huge files because of all the null packets. I can't really tell if it's a bug in JNI, JavaCPP or the version of FFmpeg it's currently based on. I'll be compiling the most recent binaries eventually, but for now this will continue to be an issue. Since I personally am most interested in SageTV clients and miniclients, I'll be putting an option in for the transcoding consumer so you can optionally enable the old timebase approach for compatibility. Does this sound good to everyone?


Daweeze,

I just wanted to give you an update on your request. I learned that SageTV won't even try to playback a video until a certain amount data is available. I created some pre-rendered clips that show Copy Once and Copy Never in very clear large letters. I was testing my implementation on HBO (Copy Once) and Showtime (Copy Freely) on channel preview and live tv. I got it to work fast enough that I think a lot of people who didn't have the feature before will actually be quite surprised at how convenient it is. I was a little skeptical about it working well, but I really think this will save people time.


brbjr,

After looking at how the FFmpeg consumer handled not knowing what the program was, it was actually a lot more trivial than I thought to fix the problem. Your tuning times should go way down when I release the next beta.