OpenDCT: An open source digital cable tuner.

[EnterNoEscape]

Changes:

• InfiniTV HTTP tuning now stops the tuner on STOP instead of just effectively silencing the stream. It turns out this optimization made zero difference in overall tuning times.
• FFmpeg direct write flushing can be disabled by setting consumer.ffmpeg.min_direct_flush_size to -1.
• Resume logic now includes waiting for all network interfaces that SageTV communicates on. Before it only waited for network interfaces that had capture devices on them which worked for most people, but some people don't have any devices accessed by the same network interface used by SageTV.
• Fixed firewall script for Ubuntu.

[jm9843]

Originally Posted by EnterNoEscape

I'm also looking into hardware acceleration like QuickSync and DXVA2, but those will not likely be available at first.

Thanks for the detailed reply outlining your plans! Regarding this part of your comment, this might be of interest to you: Accelerating Your FFmpeg-Based Applications with Intel Quick Sync Video

[EnterNoEscape]

Thanks for the link. I compiled my own copy of FFmpeg 2.8 with QSV about a week after it was announced as an officially supported encoder/decoder last year. I was real-time transcoding 6 streams simultaneously on an Ivy Bridge processor. It's very impressive.

[BobbyDing]

Originally Posted by EnterNoEscape

Java 1.7 is still ok.

Yes. Starting on 0.4.23 and up to enable "native" HDHomeRun discovery which also enables devices other than the Prime to be used, you need do the following:

1. Stop the SageTV service.
2. Stop the OpenDCT service.
3. Open opendct.properties.
4. Change the value of discovery.exp_enabled to true.
5. Start the OpenDCT service and wait a few seconds.
6. Start the SageTV service.

After making those changes, you should now see new tuners in SageTV named after all of the discovered HDHomeRun devices that responded on your network. I'm curious with so many HDHomeRun devices how well it will perform. If you don't see everything the first time, restart the SageTV service.

The Connect devices also have HTTP streaming capability that OpenDCT can use if you have done a channel scan via the web interface on the Connect. If for any reason the URL doesn't work, it has a failsafe to just use RTP. You should find HTTP to be more reliable. If it causes you any grief, you can turn it off by setting hdhr.allow_http_tuning to false. The Extend can also do transcoding via HTTP, if you want you can set the "global" profile via hdhr.extend_transcode_profile. If you're just trying to save disk space, I would just use 'heavy'. Check the release notes for 0.4.25-Beta on what profiles are available.

It's just labeled experimental in case something bad happens, so you're prepared for some potentially interesting troubleshooting. By testing this functionality out you're also testing out the way the Prime will eventually be handled too.

.

Thank You. And do I just follow Daweeze's install instructions for V3 when installing V4? No difference between the installs?
.

[EnterNoEscape]

You can follow those directions for either version. Getting the essentials working is the same between 0.3 and 0.4. There is no real difference in installation.

[gprichardson1]

First, a belated thanks for making 0.3.17 available. That seems to have fixed my halted recordings/no signal issue I had with one channel. Thanks!

I ran into another problem where it looks like I'm getting corruption in the stream and the video will pause for a second or two, pixelate then resume. This happens at various times in recordings. The timestamps of entries in the log like the ones below match up with the corruption in the recordings:

PES packet size mismatch
Continuity check failed for pid #### expected X got Y

I found this post from back in January by troll5501 where he made these two changes in opendct.properties file to fix the same problem:

opendct: consumer.ffmpeg.upload_id_enabled=false
opendct: consumer.ffmpeg.rw_buffer_size=262144

Early indications are that these changes have cured the corruption, which is great! There was some discussion on what the rw_buffer_size is and what changing it does but I haven't been able to find out anything about setting upload_id_enabled to false. Am I setting myself up for any other problems by setting this to false? What does this setting do?

Just curious, is there a page somewhere that describes what all the settings are in the properties file and what the different values (i.e. false vs. true) will accomplish?

Thanks!

[EnterNoEscape]

The very first post describes several of them including that one. For many of them there is no reason to change the default unless you're troubleshooting, so I have not described every one. There are some new ones in 0.4 that are only described in the release notes. I will eventually document them better, but I don't want to document things that could change fairly soon.

Setting the upload_id_enabled property to false just means that OpenDCT will write directly to the file system instead of using the SageTV's upload id feature. The latest 0.4 default is false because some people have been having issues with it. Each stream adds the the overall load and heap size of the SageTV instance, so I suspect the issues they are having are in that realm.

[BobbyDing]

Can V4.xx be installed over an existing V3.xx? Or should 3 be removed first?

Thanks,

Bobby

[SageWizdom]

Greetings,
I had everything running great and then broke something. I'm comfortable saying that it was me, but I need help finding it. Specifically, can you provide a list of options to turn on debugging?

I have very little output in /var/log/opendct/wrapper.log and almost nothing in opendct.log. Is there a way to enable debugging so that every command is printed/shown as it is executed, and so that data flow from the HDHR is noted.

Basically, I want to trace the command going from Sage -> OpenDCT -> HDHR and then the data flow in the reverse direction with the hope that this will allow me to figure out where I have a configuration or other issue.

Thank you.

[EnterNoEscape]

Let's start with attaching what you do have in /var/log/opendct/wrapper.log and /var/log/opendct/opendct.log. Also have you already tried disabling ufw just to be sure it's not the firewall?

If you want to turn on TRACE logging, edit /opt/opendct/lib/log4j2.xml and change <Root level="DEBUG"> to <Root level="TRACE"> and if <AppenderRef ref="RollingFile"/> also has a level attribute, change that to TRACE too. You will likely need to edit this file as root.

In opendct.properties, change sagetv.log_noop_and_size=false to true if you want to see every single NOOP and GET_FILE_SIZE request from SageTV.

[BobbyDing]

When upgrading from V3 to V4, should V3 be removed first (including the tuners in Sage), or should it be safe to installl V4 over V3 (both 32bit)? If V3 should be removed, should anything be removed in sage.properties?

Thanks!

[nyplayer]

Originally Posted by BobbyDing

When upgrading from V3 to V4, should V3 be removed first (including the tuners in Sage), or should it be safe to installl V4 over V3 (both 32bit)? If V3 should be removed, should anything be removed in sage.properties?

Thanks!

I just stopped OPenDCT ... Ran the V4 Install started the console it updated the OpenDCT property ... restarted OpenDCT and did not do anything to SageTV properties.

[BobbyDing]

Originally Posted by nyplayer

I just stopped OPenDCT ... Ran the V4 Install started the console it updated the OpenDCT property ... restarted OpenDCT and did not do anything to SageTV properties.

Thanks for the reply. Since I only had a few hours today I went ahead and removed V3 (to be on the safe side). Then I installed V4. All went perfectly for the Prime and I was once again tuning cable stations in V4.

I then enabled the ATSC tuner support and all the other tuners became available in sage. I added one of the Tampa tuners from OpenDCT (actually 2 tuners, but one HDHR Connect), used the same EPG listing from the sage tuners, and then removed original SageTV tuner references. I then did the same for my Orlando tuners. When I went to the guide, it only showed the Prime and Orlando stations. I tried re starting sage, and re installing the original Tampa tuners, but no joy. Any suggestions?
.
.
UPDATE: Fixed. Not sure why, but I had to delete all the tuners (including the Prime tuners) from the source setup and start from scratch, installing the only the OTA OpenDCT tuners first, one city at a time, saving the source setup and checking the guide between each one. This worked. Then I installed the Prime tuners. Success! All the channels are there and in proper order. The only error I made along the way was doing a channel scan on one of the HDHR OTA tuners. Things came up I had never seen before in the channel listing and I could not get rid of them. The fix for this was to select a different zip code (nearby) and then I had a clean listing again.

Sage is now using the OpenDCT encoder(s) exclusively, and everything appears to be tuning properly. Tonight Sage should start recording favorites, which will test simultaneous OTA and Prime recordings.

Thanks,

Bobby

[SageWizdom]

Originally Posted by EnterNoEscape

Let's start with attaching what you do have in /var/log/opendct/wrapper.log and /var/log/opendct/opendct.log. Also have you already tried disabling ufw just to be sure it's not the firewall?

If you want to turn on TRACE logging, edit /opt/opendct/lib/log4j2.xml and change <Root level="DEBUG"> to <Root level="TRACE"> and if <AppenderRef ref="RollingFile"/> also has a level attribute, change that to TRACE too. You will likely need to edit this file as root.

In opendct.properties, change sagetv.log_noop_and_size=false to true if you want to see every single NOOP and GET_FILE_SIZE request from SageTV.

Thanks for the help. I let the system run and can provide fantastically inconsistent results. For example, through placeshifter, channel 3 will play sporadically (several minutes playing, then several minutes of no signal). I also let the system run overnight. It generates files and even segments (ex. Show-Episode-id-1.mpg, Show-Episode-id-2.mpg) but they are all currently 0 bytes. I believe these are non channel 3 so this may not be a surprise.

I verified that UFW is inactive. I also validated iptables (iptables -L) and INPUT, OUTPUT, FORWARD are all ACCEPT.

Log files are attached. Sorry if there is something obvious in there I'm missing.

I would note that I have a opendct_cling.log, opendct_cling.log.1, opendct_cling.log.lck, opendct_cling.log.lck.1 and they are also all 0 bytes.

In my /etc/opendct/conf/lineup directory I have no files. In my opendct.properties, I swear I used to have config items for my HDHR, but looking now I dont see them, so not sure if I'm crazy or otherwise broke things.

I at one point I completely deleted my Sage and OpenDCT installs (/opt/sagetv, /opt/opendct, /etc/opendct, /var/log/opendct) and reinstalled, but that did not appear to help (trying to add as complete details as I can think of).

As this is mostly a test server, I'm happy to erase, reinstall, etc for any trouble shooting / etc that you recommend.

Thank you for all of your help.

(Side note: I was thinking of starting a specific OpenDCT troubleshooting thread where the first post is all the steps to debug through the streaming video flow. ex. check encoder, check ... etc. Please let me know if you have a preference and/or anything you would specifically want in it.)

[SageWizdom]

I should note one other thing. I note that in the /etc/init.d script you chown things to be owned by the opendct user and group, but I dont have an opendct user or group. when opendct runs, I think its running as root.

[EnterNoEscape]

You should have an opendct user and group. They were created when the package was installed. The original intention was to run OpenDCT as the user opendct, but that doesn't work because it needs to be able to set itself at a higher priority than other threads. I was taking advantage of the fact that that script has root privileges before it would have launched the OpenDCT service with only the opendct user privileges to handle some potential troubleshooting situations.

To verify the group exists:

cat /etc/group | grep opendct

To verify the user account exists:

cat /etc/passwd | grep opendct

To remove OpenDCT on Ubuntu, you should be using:

apt-get remove opendct

Any other method to remove it will likely put things in an unknown state.

I saw a few times the HDHomeRun tuners were detected, but it took two minutes. That's probably why every time I see SageTV make a discovery request, the tuners have not already loaded. You'll also want to make sure nothing is using port 8501. Cling UPnP uses that port by default and annoyingly doesn't throw a fit if it's already in use. I had to make it a specific port so the firewall rules would work. You can change the port if you want by changing the value of upnp.service.configuration.http_listen_port in opendct.properties. I should also mention that if it takes two minutes for a device to come in, either your computer is really that slow or you might have some UDP packet loss related issues.

It might help to know that Linux will limit Java to 128k on the UDP receive buffer. You can increase this temporarily (in this example to 2MB):

sysctl -w net.core.rmem_max=2097152

...or permanently by adding the following line to /etc/sysctl.conf:

net.core.rmem_max=2097152

As for what to do, I would start the OpenDCT service. Then look for entries like this one in /var/log/opendct/opendct.log to indicate that the capture devices are ready:

 INFO  SageTVManager - The capture device 'DCT-HDHomeRun Prime Tuner 131F6BC2-0' is ready.

As soon as you see that message, that means that SageTV can communicate with that capture device via OpenDCT. Then start the SageTV service. You should then see the capture devices in SageTV.


I'm not sure what to say about the 0k files. I suspect this has to do with system/network performance. In other words, OpenDCT created the file, but then never had anything to write to it. Any chance you still have logs under /var/log/opendct/archive that might have logging from when you were having tuning issues? Without those, it's hard to tell what you should be looking at.

As for a troubleshooting guide; I'll just take a wait and see approach. This project is still very young and taking shape, so things may change, but I am doing my best to make those changes clear in the RELEASE.md file on GitHub. Also if you would like to improve the bash scripts, documentation or anything else about the project that you discover is not working as expected, please create the pull request on GitHub. GitHub also has a wiki, per project, so let me know if that approach interests you and I'll make sure you have appropriate access.

[BobbyDing]

Originally Posted by EnterNoEscape

[*]Change the value of discovery.exp_enabled to true.

Does this need to be set back to false at any point?

Thanks

[nyplayer]

Originally Posted by BobbyDing

Does this need to be set back to false at any point?

Thanks

Nope no need it is better to keep it on in case you add a new tuner in the future.

[EnterNoEscape]

That needs to stay true or the tuners will no longer be discovered. It also allows OpenDCT to monitor for IP address changes which can certainly happen with DHCP.

[nyplayer]

OpenDCT 0.4.30-Beta,

I am getting no Signal see attached file. I think it is treating the Prime as a QAM tuner.