OpenDCT: An open source digital cable tuner.

[NetworkGuy]

Originally Posted by EnterNoEscape

I'm working on an exciting major overhaul within the SageTV core at the moment,

Any hints on what to expect?

[bialio]

Also, I just saw that OpenDCT could work with the HDHR Extend to use the h264 encoding they offer - is there a post somewhere buried in this thread about how to set that up? I'd love to get my recorded OTA programming out of mpeg2...

btl.

[EnterNoEscape]

Originally Posted by NetworkGuy

Any hints on what to expect?

I'm not sure you really need a hint. The related issue is open on GitHub.

Originally Posted by bialio

Also, I just saw that OpenDCT could work with the HDHR Extend to use the h264 encoding they offer - is there a post somewhere buried in this thread about how to set that up? I'd love to get my recorded OTA programming out of mpeg2...

btl.

Like many things, it's in the release notes.

[ldl72]

Good Morning....

I am trying to transition away from cable and purchased 2 HDHR CONNECTS for OTA programming. I am running the latest version of OpenDCT with Sage V9. Everything is working well, but the size of the recorded content has increased greatly compared to when I was using the HDHR Prime.

With the HDHR Primes, a 45min recording (MPEG-TS 4:3 480i@29.97fps)would be 1.5+/-gigs. With the Connects (MPEG-TS 16:9 1080i@29.97fps) I am seeing 7+gigs.

What are my options for producing smaller size recordings? I see that there is a transcoding option in OpenDCT, but I am not sure how to configure the profile for what I want. Also not sure about the PC load requirements to transcode 4+ streams at once.

[EnterNoEscape]

Originally Posted by ldl72

Good Morning....

I am trying to transition away from cable and purchased 2 HDHR CONNECTS for OTA programming. I am running the latest version of OpenDCT with Sage V9. Everything is working well, but the size of the recorded content has increased greatly compared to when I was using the HDHR Prime.

With the HDHR Primes, a 45min recording (MPEG-TS 4:3 480i@29.97fps)would be 1.5+/-gigs. With the Connects (MPEG-TS 16:9 1080i@29.97fps) I am seeing 7+gigs.

What are my options for producing smaller size recordings? I see that there is a transcoding option in OpenDCT, but I am not sure how to configure the profile for what I want. Also not sure about the PC load requirements to transcode 4+ streams at once.

If you don't have a particular reason to encode to H.264, I would recommend just having SageTV re-encode those files after being recorded because you can allow it to take more time to produce a file that's significantly smaller and higher quality than what you can get while live transcoding. Live transcoding is more for people who have devices that would not be able to play live TV if it's not H.264.

[ldl72]

Originally Posted by EnterNoEscape

If you don't have a particular reason to encode to H.264, I would recommend just having SageTV re-encode those files after being recorded because you can allow it to take more time to produce a file that's significantly smaller and higher quality than what you can get while live transcoding. Live transcoding is more for people who have devices that would not be able to play live TV if it's not H.264.

EnterNoEscape.....Thanks for the input and OpenDCT. I will look into that option.

[amg]

Originally Posted by ldl72

Good Morning....

With the HDHR Primes, a 45min recording (MPEG-TS 4:3 480i@29.97fps)would be 1.5+/-gigs. With the Connects (MPEG-TS 16:9 1080i@29.97fps) I am seeing 7+gigs.

Do realize that the difference here is recording Standard Def vs HiDef.

[sflamm]

Just installed SageTV 9 and configured OpenDCT identically to before...

Now every time the BM3000-HDMI stops I see the following error in the log:

16:53:04.185 [SageTVRequestHandler-157:STB-1] DEBUG BasicCaptureDevice - Stopping consumer thread...
16:53:04.185 [SageTVRequestHandler-157:STB-1] DEBUG BasicCaptureDevice - Waiting for consumer thread to stop...
16:53:04.186 [RawSageTVConsumerImpl-159:STB-1] DEBUG RawSageTVConsumerImpl - Raw consumer was closed by an interrupt exception => java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at opendct.consumer.buffers.SeekableCircularBufferNIO.waitForBytes(SeekableCircularBufferNIO.java:152)
at opendct.consumer.buffers.SeekableCircularBufferNIO.read(SeekableCircularBufferNIO.java:583)
at opendct.consumer.RawSageTVConsumerImpl.run(RawSageTVConsumerImpl.java:164)
at java.lang.Thread.run(Unknown Source)

16:53:04.186 [RawSageTVConsumerImpl-159:STB-1] INFO RawSageTVConsumerImpl - Raw consumer thread is now stopping.
16:53:04.187 [RawSageTVConsumerImpl-159:STB-1] DEBUG RawSageTVConsumerImpl - Bytes available to be read = 0
16:53:04.188 [RawSageTVConsumerImpl-159:STB-1] DEBUG RawSageTVConsumerImpl - Space available for writing in bytes = 2097151
16:53:04.188 [RawSageTVConsumerImpl-159:STB-1] INFO RawSageTVConsumerImpl - Raw consumer thread has stopped.
16:53:04.188 [SageTVRequestHandler-157:STB-1] DEBUG GenericHttpCaptureDevice - Capture device is now re-unlocked.
16:53:04.189 [SageTVRequestHandler-157:STB-1] DEBUG SageTVRequestHandler - Replied: 'OK'

Running OpenDCT OpenDCT_0.5.24_x86

Seems to be working otherwise...

[sflamm]

The configuration in opendct.properties is (same as before):

agetv.device.79219557.channel_padding=0
sagetv.device.79219557.consumer=opendct.consumer.RawSageTVConsumerImpl
sagetv.device.79219557.custom_channels=
sagetv.device.79219557.delay_to_wait_after_tuning=0
sagetv.device.79219557.device_name=STB-1
sagetv.device.79219557.encoder_listen_port=9000
sagetv.device.79219557.encoder_merit=0
sagetv.device.79219557.encoder_pool=generic_http
sagetv.device.79219557.exclusive_server_address=
sagetv.device.79219557.fast_network_encoder_switch=true
sagetv.device.79219557.http_password=
sagetv.device.79219557.http_username=
sagetv.device.79219557.last_channel=550
sagetv.device.79219557.pretuning_executable=
sagetv.device.79219557.stopping_executable=
sagetv.device.79219557.stopping_executable_delay_ms=15000
sagetv.device.79219557.streaming_url=http\://10.0.0.39/0.ts
sagetv.device.79219557.streaming_url2=
sagetv.device.79219557.streaming_url2_channels=
sagetv.device.79219557.transcode_profile=
sagetv.device.79219557.tuning_delay_ms=0
sagetv.device.79219557.tuning_executable=
sagetv.device.79219558.channel_padding=0
sagetv.device.79219558.consumer=opendct.consumer.RawSageTVConsumerImpl
sagetv.device.79219558.custom_channels=
sagetv.device.79219558.delay_to_wait_after_tuning=0
sagetv.device.79219558.device_name=STB-2
sagetv.device.79219558.encoder_listen_port=9000
sagetv.device.79219558.encoder_merit=0
sagetv.device.79219558.encoder_pool=generic_http
sagetv.device.79219558.exclusive_server_address=
sagetv.device.79219558.fast_network_encoder_switch=true
sagetv.device.79219558.http_password=
sagetv.device.79219558.http_username=
sagetv.device.79219558.last_channel=-1
sagetv.device.79219558.pretuning_executable=
sagetv.device.79219558.stopping_executable=
sagetv.device.79219558.stopping_executable_delay_ms=15000
sagetv.device.79219558.streaming_url=http\://10.0.0.39/0.ts
sagetv.device.79219558.streaming_url2=
sagetv.device.79219558.streaming_url2_channels=
sagetv.device.79219558.transcode_profile=
sagetv.device.79219558.tuning_delay_ms=0
sagetv.device.79219558.tuning_executable=

[sflamm]

After changing the consumer back to:

opendct.consumer.DynamicConsumerImpl

There are no exceptions....

Using the consumer:

opendct.consumer.RawSageTVConsumerImpl

causes exceptions every time the stream is stopped by SageTV:

19:03:39.642 [SageTVRequestHandler-101:Unknown] DEBUG SageTVRequestHandler - SageTV sent: 'STOP STB-1 HDMI'
19:03:39.642 [SageTVRequestHandler-101:STB-1] DEBUG GenericHttpCaptureDevice - Stopping encoding...
19:03:39.642 [SageTVRequestHandler-101:STB-1] DEBUG HTTPCaptureDeviceServices - Stopping producer thread...
19:03:39.642 [SageTVRequestHandler-101:STB-1] DEBUG HTTPCaptureDeviceServices - Waiting for producer thread to stop...
19:03:39.643 [NIOHTTPProducerImpl-102:STB-1] DEBUG NIOHTTPProducerImpl - The socket has been closed.
19:03:39.643 [NIOHTTPProducerImpl-102:STB-1] INFO NIOHTTPProducerImpl - Producer thread has stopped.
19:03:39.643 [SageTVRequestHandler-101:STB-1] DEBUG BasicCaptureDevice - Stopping consumer thread...
19:03:39.643 [SageTVRequestHandler-101:STB-1] DEBUG BasicCaptureDevice - Waiting for consumer thread to stop...
19:03:39.643 [RawSageTVConsumerImpl-103:STB-1] DEBUG RawSageTVConsumerImpl - Raw consumer was closed by an interrupt exception => java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at opendct.consumer.buffers.SeekableCircularBufferNIO.waitForBytes(SeekableCircularBufferNIO.java:152)
at opendct.consumer.buffers.SeekableCircularBufferNIO.read(SeekableCircularBufferNIO.java:583)
at opendct.consumer.RawSageTVConsumerImpl.run(RawSageTVConsumerImpl.java:164)
at java.lang.Thread.run(Unknown Source)

19:03:39.645 [RawSageTVConsumerImpl-103:STB-1] INFO RawSageTVConsumerImpl - Raw consumer thread is now stopping.
19:03:39.648 [RawSageTVConsumerImpl-103:STB-1] DEBUG RawSageTVConsumerImpl - Bytes available to be read = 0
19:03:39.649 [RawSageTVConsumerImpl-103:STB-1] DEBUG RawSageTVConsumerImpl - Space available for writing in bytes = 2097151
19:03:39.649 [RawSageTVConsumerImpl-103:STB-1] INFO RawSageTVConsumerImpl - Raw consumer thread has stopped.
19:03:39.650 [SageTVRequestHandler-101:STB-1] DEBUG GenericHttpCaptureDevice - Capture device is now re-unlocked.
19:03:39.650 [SageTVRequestHandler-101:STB-1] DEBUG SageTVRequestHandler - Replied: 'OK'
19:03:39.656 [StoppingThread-105] DEBUG GenericHttpCaptureDevice - Stopping executable will be run in 15000 milliseconds.

[sflamm]

Connected two BM3000-HDMI...

When each is recording individually it is fine. When both are recording simultaneously there are all sorts of issues in the log file and video/audio stream from the second source plays/sounds as if it is in half-rate slow-mo.

Here is the log content (full log attached):

01:22:22.095 [FFmpegTransSageTVConsumerImpl-118:STB-1] INFO FFmpegContext - Primary:
Input #0, mpegts, from 'input-mpeg-ts.ts':
Duration: 00:00:01.07, start: 42516.269433, bitrate: 5435 kb/s
Program 1
Stream #0:0[0x1e1], 24, 1/90000: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p, 1920x1080, 29.97 fps, 29.97 tbr, 90k tbn, 60 tbc
Stream #0:1[0x1e2], 19, 1/90000: Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 49 kb/s

01:22:22.095 [FFmpegTransSageTVConsumerImpl-118:STB-1] DEBUG FFmpegContext - Calling avformat_alloc_output_context2
01:22:22.096 [FFmpegTransSageTVConsumerImpl-118:STB-1] INFO FFmpegContext - Primary:
Output #0, mpegts, to '\\nas\public\RecordedTV\LateNightWithSethMeyers-JohnOliverZosiaMametBrannDailor-52331-3.ts':
Stream #0:0, 0, 1/90000: Video: h264, yuv420p, 1920x1080, q=2-31, 90k tbn, 60 tbc
Stream #0:1, 0, 1/90000: Audio: aac, 44100 Hz, stereo, fltp, 49 kb/s

01:22:22.096 [FFmpegTransSageTVConsumerImpl-118:STB-1] DEBUG FFmpegTranscoder - Writing header
01:22:22.096 [FFmpegTransSageTVConsumerImpl-118:STB-1] INFO FFmpegTranscoder - Initialized FFmpeg transcoder stream output.
01:22:22.099 [FFmpegTransSageTVConsumerImpl-118:STB-1] DEBUG NULL - [3d2de580] unknown SEI type 229
01:22:22.264 [SageTVRequestHandler-115:STB-1] DEBUG SageTVRequestHandler - Replied: 'OK'
01:22:22.940 [FFmpegTransSageTVConsumerImpl-113:STB-2] DEBUG NULL - [0127d2e0] unknown SEI type 229
01:22:22.941 [FFmpegTransSageTVConsumerImpl-118:STB-1] DEBUG NULL - [3d2de580] unknown SEI type 229
01:22:23.960 [FFmpegTransSageTVConsumerImpl-113:STB-2] DEBUG NULL - Repeated 1 time: [3d2de580] unknown SEI type 229.
01:22:23.960 [FFmpegTransSageTVConsumerImpl-113:STB-2] DEBUG NULL - [0127d2e0] unknown SEI type 229
01:22:24.927 [FFmpegTransSageTVConsumerImpl-118:STB-1] DEBUG NULL - Repeated 1 time: [0127d2e0] unknown SEI type 229.
01:22:24.927 [FFmpegTransSageTVConsumerImpl-118:STB-1] DEBUG NULL - [3d2de580] unknown SEI type 229
01:22:25.955 [FFmpegTransSageTVConsumerImpl-113:STB-2] DEBUG NULL - Repeated 1 time: [3d2de580] unknown SEI type 229.
01:22:25.955 [FFmpegTransSageTVConsumerImpl-113:STB-2] DEBUG NULL - [0127d2e0] unknown SEI type 229
01:22:26.929 [FFmpegTransSageTVConsumerImpl-118:STB-1] DEBUG NULL - [3d2de580] unknown SEI type 229
01:22:26.965 [FFmpegTransSageTVConsumerImpl-113:STB-2] DEBUG NULL - [0127d2e0] unknown SEI type 229
01:22:27.938 [FFmpegTransSageTVConsumerImpl-118:STB-1] DEBUG NULL - Repeated 1 time: [0127d2e0] unknown SEI type 229.
01:22:27.939 [FFmpegTransSageTVConsumerImpl-118:STB-1] DEBUG NULL - [3d2de580] unknown SEI type 229
01:22:28.960 [FFmpegTransSageTVConsumerImpl-113:STB-2] DEBUG NULL - Repeated 1 time: [3d2de580] unknown SEI type 229.
01:22:28.961 [FFmpegTransSageTVConsumerImpl-113:STB-2] DEBUG NULL - [0127d2e0] unknown SEI type 229
01:22:29.916 [FFmpegTransSageTVConsumerImpl-118:STB-1] DEBUG NULL - [3d2de580] unknown SEI type 229
01:22:29.950 [FFmpegTransSageTVConsumerImpl-113:STB-2] DEBUG NULL - [0127d2e0] unknown SEI type 229
01:22:30.947 [FFmpegTransSageTVConsumerImpl-118:STB-1] DEBUG NULL - [3d2de580] unknown SEI type 229
01:22:30.950 [FFmpegTransSageTVConsumerImpl-113:STB-2] DEBUG NULL - [0127d2e0] unknown SEI type 229
01:22:31.259 [SageTVRequestHandler-115:Unknown] DEBUG SageTVRequestHandler - SageTV sent: 'STOP STB-1 HDMI'
01:22:31.259 [SageTVRequestHandler-115:STB-1] DEBUG GenericHttpCaptureDevice - Stopping encoding...
01:22:31.259 [SageTVRequestHandler-115:STB-1] DEBUG HTTPCaptureDeviceServices - Stopping producer thread...
01:22:31.260 [SageTVRequestHandler-115:STB-1] DEBUG HTTPCaptureDeviceServices - Waiting for producer thread to stop

[EnterNoEscape]

Originally Posted by sflamm

Just installed SageTV 9 and configured OpenDCT identically to before...

Now every time the BM3000-HDMI stops I see the following error in the log:



Running OpenDCT OpenDCT_0.5.24_x86

Seems to be working otherwise...

That's just the socket closing like it's supposed to. Note that the exception is tagged DEBUG. It's just there in case it actually is a problem. In this context, it's definitely not.

Use RawSageTVConsumerImpl for these kinds of streams. None of the exceptions you have listed are actually problems. They are completely normal and just a result of the consumer being told to stop waiting for more data and the connection to the capture device being told to close. I see you're using FFmpegTransSageTVConsumerImpl (probably because it's the default for DynamicConsumerImpl) which isn't needed at all because these are already clean streams and you don't need FFmpeg to clean them up.

[videolog]

Well, stopped working again this morning. After the last time, a couple of weeks ago, I went back to using Sage DCT. No hangs. I really want to use Open Dct, as it is still being developed and has a future, but I just can't get it to stay working. I was determined this time so I had Comcast come and check the line. The numbers were spot on. I then updated the ceton's to the latest beta that everyone here is using. I did a fresh install of the latest Sage and Open Dct. Did all that on Saturday. Everything was fine until this morning. Not sure where to go from here. This is the latest log.

[EnterNoEscape]

Originally Posted by videolog

Well, stopped working again this morning. After the last time, a couple of weeks ago, I went back to using Sage DCT. No hangs. I really want to use Open Dct, as it is still being developed and has a future, but I just can't get it to stay working. I was determined this time so I had Comcast come and check the line. The numbers were spot on. I then updated the ceton's to the latest beta that everyone here is using. I did a fresh install of the latest Sage and Open Dct. Did all that on Saturday. Everything was fine until this morning. Not sure where to go from here. This is the latest log.

FFmpeg could not detect anything at all from the provided stream. I mean literally is could not find any streams at all. If SageDCT is working for you, you could try using the raw consumer instead, but that doesn't really answer why the stream was so devoid of content.

[videolog]

Originally Posted by EnterNoEscape

FFmpeg could not detect anything at all from the provided stream. I mean literally is could not find any streams at all. If SageDCT is working for you, you could try using the raw consumer instead, but that doesn't really answer why the stream was so devoid of content.

Okay, thanks. I will give the raw consumer a go.

[videolog]

Originally Posted by EnterNoEscape

FFmpeg could not detect anything at all from the provided stream. I mean literally is could not find any streams at all. If SageDCT is working for you, you could try using the raw consumer instead, but that doesn't really answer why the stream was so devoid of content.

I was just looking at the log again. I am trying to understand how this whole thing works. You said it could not find any streams. So if I see PES packet size mismatch in the logs it means I lost the stream? This error occurred approx 6 minutes into the show at 6:23.

Sorry if I am being a pain. Just trying to understand what I am reading in the log.

[EnterNoEscape]

Originally Posted by videolog

I was just looking at the log again. I am trying to understand how this whole thing works. You said it could not find any streams. So if I see PES packet size mismatch in the logs it means I lost the stream? This error occurred approx 6 minutes into the show at 6:23.

Sorry if I am being a pain. Just trying to understand what I am reading in the log.

You have my apologies. I did not notice that much further up in the log. What I was seeing was the buffer being starved after SageTV asked for OpenDCT to restart the recording. On the second round, you see a few entries like this:

INFO   | jvm 1    | 2017/02/14 06:26:19.644 | 06:26:19.564 [FFmpegTransSageTVConsumerImpl-292:DCT-Ceton InfiniTV PCIe (00-80-8c-0b) Tuner 4] INFO  FFmpegContext - Returning AVERROR_EOF in readCallback.call()
INFO   | jvm 1    | 2017/02/14 06:26:21.644 | 06:26:21.566 [FFmpegTransSageTVConsumerImpl-292:DCT-Ceton InfiniTV PCIe (00-80-8c-0b) Tuner 4] INFO  FFmpegContext - Returning AVERROR_EOF in readCallback.call()
INFO   | jvm 1    | 2017/02/14 06:26:23.644 | 06:26:23.567 [FFmpegTransSageTVConsumerImpl-292:DCT-Ceton InfiniTV PCIe (00-80-8c-0b) Tuner 4] INFO  FFmpegContext - Returning AVERROR_EOF in readCallback.call()
INFO   | jvm 1    | 2017/02/14 06:26:23.644 | 06:26:23.567 [FFmpegTransSageTVConsumerImpl-292:DCT-Ceton InfiniTV PCIe (00-80-8c-0b) Tuner 4] ERROR FFmpegContext - initTsStream: avformat_open_input returned error code -1094995529

The first three aren't always a problem, but the fact that we see nothing between those entries and avformat_open_input returned error code means each one of those was the buffer returning without any data at all because it waited a full 2 seconds and not even one byte rolled in. That's a lot of time missing for RTP. The buffer does try to wait for some data to accumulate before passing that data on to FFmpeg partly for efficiency purposes, but if it's left with less than that, it will proceed with what it has. This also makes things more responsive when changing channels a lot or when you tune in a channel that you don't get. This combination will only happen up to 3 times before it just gives up.

Edit: There are further entries supporting that RTP was configured correctly, so why nothing was coming in is the mystery.

[sflamm]

Use RawSageTVConsumerImpl for these kinds of streams. None of the exceptions you have listed are actually problems.

Thanks for the information. Will return it to a Raw Consumer (was using that previously). The BM3000-HDMI produces an FFmpeg stream (in the settings) so I understand it will not be further cleaned up.

Questions:

1. what does "DEBUG NULL - [0127d2e0] unknown SEI type 229" indicate?

2. I have also seen in the logs it can't find the audio stream properly at first... in the settings for the BM3000-HDMI the codec is producing aac+. The available options are: aac, aac+, aac++ and mp3. Which do you recommend?

3. The BM3000-HDMI codec also support different profile levels for the H264 Stream. The profiles differ practically only in compressions/size. Not sure all of the available options are handled properly downstream by OpenDCT or SageTV. The profile options are: high profile, main profile, and base profile.
Which profile is recommended? (high is currently selected)

Thanks in advance.

[EnterNoEscape]

Originally Posted by sflamm

Thanks for the information. Will return it to a Raw Consumer (was using that previously). The BM3000-HDMI produces an FFmpeg stream (in the settings) so I understand it will not be further cleaned up.

Questions:

1. what does "DEBUG NULL - [0127d2e0] unknown SEI type 229" indicate?

2. I have also seen in the logs it can't find the audio stream properly at first... in the settings for the BM3000-HDMI the codec is producing aac+. The available options are: aac, aac+, aac++ and mp3. Which do you recommend?

3. The BM3000-HDMI codec also support different profile levels for the H264 Stream. The profiles differ practically only in compressions/size. Not sure all of the available options are handled properly downstream by OpenDCT or SageTV. The profile options are: high profile, main profile, and base profile.
Which profile is recommended? (high is currently selected)

Thanks in advance.

1. I don't actually know. That's straight from the FFmpeg logging. That logging is routed through into the normal logs after some filtering because we have the FFmpeg logging set to debug which can return a lot more than we need if you let it.

2. I use AAC. Some Googling a little while ago turned up the differences between AAC, AAC+ and AAC++ on Wikipedia. I don't recall all of the details, but my own conclusion was that it wasn't worth using the other codecs and that AAC is the most compatible of the bunch when using an MPEG-TS container.

3. I use high profile. Most modern hardware can handle high profile. Beyond the bitrates they also include additional encoding methods that produce better quality at lower bitrates at the expense of higher decoding power. This is another thing you can read about on Wikipedia.

[Monedeath]

Originally Posted by ldl72

With the HDHR Primes, a 45min recording (MPEG-TS 4:3 480i@29.97fps)would be 1.5+/-gigs. With the Connects (MPEG-TS 16:9 1080i@29.97fps) I am seeing 7+gigs.

Originally Posted by amg

Do realize that the difference here is recording Standard Def vs HiDef.

Now that's just stating the obvious and being mean.

How could a 720x480interlaced picture (720x243 pixels= 174,960 pixels per frame) possibly create a much smaller file than a 1920x1080interlaced picture (1920x540 pixels= 1,036,800 pixels per frame) would? It only has roughly 5.93 times as many pixels to keep track of.