OpenDCT: An open source digital cable tuner.

[jgsouthard]

Originally Posted by EnterNoEscape

It's good to see you're finally heading somewhere pleasant. I was very concerned we might have a lot to work on when you were among some of the people saying that all the other software wasn't working for them.

OpenDCT is already more reliable and robust for me than the latest versions of either PrimeNetEncoder or SageDCT, and based on your recent efforts I'm confident that any remaining issues will get resolved. I don't want to take anything away from their excellent work on those tools -- If it hadn't been for SageDCT many of us would have had no choice but to abandon SageTV several years ago, and recently PNE has worked for many even where SageDCT didn't. But I'm definitely liking what I see with OpenDCT.

Originally Posted by EnterNoEscape

The only thing I can advise you on with this one is to try increasing the size of producer.rtp.nio.native_udp_receive_buffer=1328000. It looks like you only have a little bit of a problem, so I'd probably bump it up to 2-3MB, but I've seen people suggest 8MB+ receive buffer sizes as "recommended." Just keep in mind the end result does consume proportionally more RAM multiplied by each tuner. 1328000 was chosen because it's a multiple of the typical RTP packet size. That means that under normal conditions the OS will buffer up to 1000 packets per tuner for OpenDCT.

OK, I may play around with that parameter a bit. But I'm not too concerned about it -- That's the only error I saw this morning, and it occurred during the hour when I was recording on all six tuners at once (which would be unusual for me in normal use), and I didn't notice any artifacts at that point in the recordings.

[EnterNoEscape]

I completely agree on SageDCT. If SageDCT had not happened I probably wouldn't be where I am now with SageTV. My wife and I record a lot of TV. Renting 3 HD STB's is expensive and they're electricity hogs. Even with my HDHomeRun dual tuner doing ClearQAM, it was not enough. I also hate how the HD-PVR really softens the picture at any quality level.

The funny thing is that I started to write OpenDCT, not because SageDCT wasn't working out for me, but because I wanted to be able to use my old InfiniTV 4 for ClearQAM and there were no solutions for SageTV that would do it.

It was extremely fortunate that Bill posted his network encoder. I had come to the conclusion based on my own experiences with SageDCT that FFmpeg would never work out for me, so I started writing my own re-muxer in Java. Bill's implementation of an FFmpeg TS remuxer which is what OpenDCT uses is very good. If it wasn't for Bill, OpenDCT would not be where it is already. We probably wouldn't even be having this conversation.

PrimeNetEncoder was never an option for me because I only owned Ceton devices. I now own a Prime, but I really just bought it to support it better.

[jgsouthard]

Originally Posted by EnterNoEscape

It was extremely fortunate that Bill posted his network encoder. I had come to the conclusion based on my own experiences with SageDCT that FFmpeg would never work out for me, so I started writing my own re-muxer in Java. Bill's implementation of an FFmpeg TS remuxer which is what OpenDCT uses is very good. If it wasn't for Bill, OpenDCT would not be where it is already. We probably wouldn't even be having this conversation.

Thanks to both of you.

I'm not a programmer even though I have a long-dormant technical background, but I think I understand and appreciate how the creative process works.

In addition to spending too much time messing with computer stuff, I love to cook (and my wife seems to appreciate it). I almost never use a recipe as is; I instead create my own. But my best recipes are ones where I leveraged key ideas from others. So every time someone asks about one of my recipes, my answer is something like "well, it's not totally mine." So I get it.

But the truth is, OpenDCT wouldn't be where it is without you, either.

[jgsouthard]

Originally Posted by EnterNoEscape

The only thing I can advise you on with this one is to try increasing the size of producer.rtp.nio.native_udp_receive_buffer=1328000.

I changed producer.rtp.nio.native_udp_receive_buffer to 2656000 yesterday, and I have seen no more of those errors. In fact, I can't find any errors at all in wrapper.log, although I see a few warnings for "PES packet size mismatch" occasionally. But right now, everything is running great.

[troll5501]

Originally Posted by EnterNoEscape

This release addresses some communication recovery issues when the HDHomeRun drops off the network or stops talking for any other reason. It also should completely fix 1 character responses.

This release also adds an extra recovery element to the current buffering. If the circular buffer overflows, the data that would otherwise be discarded, is now placed in a dynamically allocated queue that is allowed to grow up to 4 times the buffer size. This will help out mostly in situations when computer processing resources are consumed to the point that FFmpeg temporarily can't keep up with the live data. It should buy you over 10 seconds if no FFmpeg processing is possible or there is temporary hard drive congestion.

So far, 0.4.17 is looking good! The HDHomeRunControl errors are gone. I like the concept of the dynamic overflow buffer, however this version does seem more susceptible to buffer issues and stuttering vs. 0.4.16 (or maybe it was 15) when I run my "regedit" load test. Maybe due to the additional code paths for checking/managing the overflow buffer? However, under normal use with the 3 Prime tuners I am not seeing any buffering problems.

I still need to keep rw_buffer_size at 256K or I have buffer issues. I even set up a test VM with CentOS to try out the Linux version of OpenDCT and I saw the same thing there. Just kicking off some recordings using all default settings generated many buffering issues. Increasing to 256K fixed it. Not sure why nobody else is seeing this. Maybe it's related to the model/speed of CPU I am using (probably not as fast as the newer i5/i7 series).

[troll5501]

Originally Posted by jgsouthard

I see a few warnings for "PES packet size mismatch" occasionally.

I see these several times per hour, and they often occur at the start or end of commercials or when the main program resumes after a commercial break.

[EnterNoEscape]

Originally Posted by troll5501

So far, 0.4.17 is looking good! The HDHomeRunControl errors are gone. I like the concept of the dynamic overflow buffer, however this version does seem more susceptible to buffer issues and stuttering vs. 0.4.16 (or maybe it was 15) when I run my "regedit" load test. Maybe due to the additional code paths for checking/managing the overflow buffer? However, under normal use with the 3 Prime tuners I am not seeing any buffering problems.

You could be right. There is one more step when data is written that checks to make sure the overflow buffer doesn't have any content before it writes anything. The check should be fairly inexpensive resource-wise since it just checks a number. I'll give it a once over to see if I can make it any better.

Originally Posted by troll5501

I still need to keep rw_buffer_size at 256K or I have buffer issues. I even set up a test VM with CentOS to try out the Linux version of OpenDCT and I saw the same thing there. Just kicking off some recordings using all default settings generated many buffering issues. Increasing to 256K fixed it. Not sure why nobody else is seeing this. Maybe it's related to the model/speed of CPU I am using (probably not as fast as the newer i5/i7 series).

My vote is hardware performance. Also worth noting and I should probably have this somewhere, Linux will limit Java to 128k on the UDP receive buffer. You can increase this temporarily (in this example to 2MB):

sysctl -w net.core.rmem_max=2097152

...or permanently by adding the following line to /etc/sysctl.conf:

net.core.rmem_max=2097152

Originally Posted by troll5501

I see these several times per hour, and they often occur at the start or end of commercials or when the main program resumes after a commercial break.

That sounds right. Sometimes commercials are not very cleanly inserted into the broadcast. I could see that especially happening on live broadcasts. You might also see them at the very start of a recording since sometimes parts of the data being referenced were broadcasted before they could be received. I wouldn't worry about them unless they appear to be effecting the overall watching experience. As long as every RTP packet is making it in, short of a malfunction in the program, you're getting the best you're likely going to get.

[EnterNoEscape]

This release marks the start of official support for Fedora 23. Also for people using the any Linux version of OpenDCT, you should know that the conf folder will be moved from /opt/opendct to /etc/opendct on installation. If you upgrade on the command line, you will see this happen and it tells you it happened. This change is because Debian complains about files marked for configuration not being installed to the /etc location. For more details on this check the 0.4.18-beta release notes.

The circular buffer was cleaned up a little and all debug logging was commented out because at this point it's only useful during development.

I noticed with the latest unofficial firmware when tuning using ClearQAM on the InfiniTV 6, tuning into the frequency 0 to silence the RTP stream would sometimes cause the tuner to be unresponsive until it was soft rebooted. Instead the program now sets the program to 65535 which has the same effect without locking up the tuner. I will also note that on the InfiniTV 6 with the unofficial firmware, it also addresses weak signals. It seems that a side-effect of this fix is very dodgy signal locking in ClearQAM.

[KryptoNyte]

Thank you for the installation instructions - they were very concise. Coming from SageDCT, this was just fantastic to have in front of me during the process and I will use it again to upgrade my own SageTV server to OpenDCT.

I did, however, use Java Version 1.7.80 32 bit, on Windows 7, SageTV7.

Originally Posted by Daweeze

Ok, so here's the latest after I uninstalled OpenDCT 0.3.13-beta (Also manually removed folder under C:\ProgramData).

For Reference:

Operating System: Windows 8.1
SageTV Version: 7.1.9.256
Java Version: 1.8.0_66 (32-bit)
Network Tuners: 2x HDHomeRun Prime (CableCard) = 6 Tuners

Installation:

1) Downloaded OpenDCT 0.3.14-beta.
2) Stopped SageDCT Service, Changed Startup Type to Disabled.
3) Stopped SageTV Service.
4) Made Backup Copy of Sage.properties and Wiz.bin.
5) Installed OpenDCT 0.3.14-beta.
6) Ran "OpenDCT Run as Console".
7) CTRL-C on OpenDCT Following Download of Channel Lineups.
8) Ran "OpenDCT Start Service" - Success!
9) Edited and Saved Sage.properties and changed network_encoder_discovery=false to network_encoder_discovery=true.
10) Started SageTV Service.
11) Started SageTV.
12) Setup>Setup Video Sources>Add New Source.
13) Selected New Video Capture Device (Ex. DCT-HDHomeRun Prime Tuner 13113C69-0 on 127.0.0.1:9000).
14) Selected "Digital TV Tuner" (Box Turns Red).
15) Selected "Digital TV Tuner" Again (Message: "The Broadcast Standard is being determined. Please wait...").
16) Selected "Use Tuner on Capture Card to tune Antenna/Cable TV. (To select between Antenna of Cable, use the SageTV installation that is controlling this Network Encoder)".
17) Selected "Use or Copy an Existing Channel Lineup for this Source".
18) Selected the Existing Lineup (Ex. Comcast Digital - Fairfield).
19) Selected "Use the same channel lineup".
20) Repeated Steps 12-19 For Each Remaining OpenDCT Tuner Available.
21) Exited SageTV.
22) Stopped SageTV Service.
23) Edited and Saved Sage.properties and Removed the "mmc/encoders" Entries For Each SageDCT Tuner Identified. (Ex. "mmc/encoders/123456990/video_capture_device_name=SageDCT-HDHomeRun Prime Tuner 13113C69-1" is the Identifying Entry That Ties My SageDCT Tuner to #1234560990, So All "mmc/encoders/1234560990....." Entries Were Deleted As Were 5 Other SageDCT Tuner Entries.
24) Started SageTV Service.
25) Started SageTV.
26) Setup>Setup Video Sources>Add New Source.
27) Confirmed only OpenDCT Tuner Entries Were Present.
28) Tested Recording On All 6 Tuners Simultaneously - Success!

Notes:

1) Default configuration worked fine, no modifications of any file were necessary.
2) OpenDCT Tuners show "DCT-TunerName Etc....." as a Source. I expected "OpenDCT-TunerName Etc....." as a Source. I see no reason why to not do that since SageDCT does it as well.


So far so good, thanks for the hard work by everyone on this project and helping keep SageTV alive. I have high hopes with this release!!

Also, I have attached a copy of the Channel Lineup that was bombing out in the 0.3.13-beta in case it helps clarify what a Lineup might return and why.

Thanks again!

-Travis

[Daweeze]

Originally Posted by KryptoNyte

Thank you for the installation instructions - they were very concise. Coming from SageDCT, this was just fantastic to have in front of me during the process and I will use it again to upgrade my own SageTV server to OpenDCT.

I did, however, use Java Version 1.7.80 32 bit, on Windows 7, SageTV7.

No problem, I am glad to have helped someone else move to this amazing option for SageTV!

-Travis

[EnterNoEscape]

Originally Posted by Daweeze

No problem, I am glad to have helped someone else move to this amazing option for SageTV!

-Travis

I added a link to your post on the first post to make it easier to find since it has been helping several people. It's probably best for the developer to not also write all of the documentation because when you know the program so well, it leads to unintentionally leaving details out that might be crucial for someone to know.

[KryptoNyte]

The only part that I could use a little clarification on;

6) Ran "OpenDCT Run as Console".
7) CTRL-C on OpenDCT Following Download of Channel Lineups.

I wasn't sure when that command window was finished to CTRL-C out of it. Does that window show "success" when it finishes as an indicator to close out of the window?

[EnterNoEscape]

The console thing isn't my favorite step, but if there's a Java version related problem you'll actually see it in the console window as opposed to wondering why the service won't start. You can just go straight to running the service for about 30 seconds, then stop it and have the same effect. When it's run in the console window, you're effectively running the service visibly. Everything you see in the console window also ends up in wrapper.log. Since discovery is an ongoing thing in OpenDCT, you'll never actually see anything equivalent to success. Most devices check in under 10 seconds. That's why I recommend just waiting about 30 seconds, then hitting Ctrl-C. If you wait closer to a minute, you'll see the channel lineups download too, but it is not required.

[crawfofd]

EnterNoEscape, I have a quick question for you. I looked but was unable to find the answer, so I apoligise if it has already be asked or answered.
Can I use openDCT to access a extra ceton 4 pcie card that does not have a cable card. Basically can I use it as a tuner for ota??

[crawfofd]

Also wanted to complement you on the software. It has worked great for me.
Thanks for all your work.

[crawfofd]

Update, Dont need to answer my last. I found the info I needed.

[EnterNoEscape]

Three posts in a row; it looks like you're just trying to get your post count up. As you probably found out, without a CableCARD you're only going to get ClearQAM. Depending on your provider, the number of channels you can tune like that may be very limited.

[crawfofd]

Yes, I should have put it all in one post! Thanks for the info. I realized that I would not be able to use it for OTA with an antenna.

[brandypuff]

Question:

If i integrate my Prime into SageTV using OpenDCT, will my recorded shows display program guide information for what the Prime recorded? My current setup using WMC controlled by RemotePotato to record shows using a Prime with a cable card and then converting the .WTV files to TS streams does not have any way to disply the program guide so i have to go by the recorded filename at best.

For some reason, even though i have a cable card in the Prime, some shows will just not be recorded unless i use WMC since that is the only app that will save encrypted content. I can watch any show OK using the Prime, however.

[Fuzzy]

Originally Posted by brandypuff

Question:

If i integrate my Prime into SageTV using OpenDCT, will my recorded shows display program guide information for what the Prime recorded? My current setup using WMC controlled by RemotePotato to record shows using a Prime with a cable card and then converting the .WTV files to TS streams does not have any way to disply the program guide so i have to go by the recorded filename at best.

For some reason, even though i have a cable card in the Prime, some shows will just not be recorded unless i use WMC since that is the only app that will save encrypted content. I can watch any show OK using the Prime, however.

Sage cannot record DRM'd channels from the Prime - and the DRM'd WTV files cannot be converted to TS streams either. In short, there's no way to watch DRM's recordings outside of WMC. The HDHR streaming does support playback of live DRM'd streams to a DTCP-IP enabled reciever.

Most 'robust' setup, if you need to watch DRM'd channels, is to ahve an HDHR Prime for all the non-DRM'd channels, and an HD-PVR with a STB for the DRM'd ones.