OpenDCT: An open source digital cable tuner.

[Fuzzy]

Originally Posted by sflamm

The motivation is:

1. Setup a SageTV Server that serves local tuners
- this allows any other SageTV Server to use this shared resource
- allows any program (not just SageTV) to use this shared resource
- also important when the hardware itself must be plugged in local (USB-UIRT)
- especially useful when the common resource can tune ALL channels (including protected) such as the BM3000-HDMIs (amazing)


2. Ability to upgrade / configure one without the other - split the dependency
- the Network Encoders are completely backward compatible, works with any SageTV version
- Can migrate configurations easily without reconfiguring tuners

3. Speed of changing channels for IR. (latest script attached)
- the script is very fast since it leverages the in-memory REST
- support multiple zones natively

I guess we aren't speaking the same language - because I see nothing in that list that is requiring sagetv to even be installed on that machine.

I'm asking this because you were requesting a capability to be changed in sagetv so you could share a network encoder through another network encoder. As was alluded to, this will result in the potential for a neverending loop during discovery if someone happens to turn the encoding server on on two servers on the network.

Server1 has 1 hardware encoder on it (called Encoder1), and has encoding server enabled. In it's source list it will show:
Encoder1

Server 2 is started up and has 2 hardware encoders on it (called Encoder2 and Encoder3), and get the encoding server turned on in it. In it's source list, it will now show:
Encoder2
Encoder3
Encoder1 on Server1

Server1 is later restarted at some point. It will go through discover of encoders out on the network, and populate any it finds in it's list, it will then have:
Encoder1
Encoder2 on Server2
Encoder3 on Server2
Encoder1 on Server2

If Server 2 is restarted, it will then have
Encoder2
Encoder3
Encoder1 on Server1
Encoder2 on Server1
Encoder3 on Server1
Encoder1 on Server1

and so on.

This, and the fact that there should be no architectural reason to require it, is why network encoder type capture devices are not shared on the encoding server. Capture devices are only shared from the server that is actually housing them directly.

[sflamm]

On the network encoder serving another network encoder... if we have the ability to have a Network Tuner to complement the Network Encoder then I don't think that is necessary. That's what the script is trying to achieve....

Network Encoder: OpenDCT
Network Tuner: Tune (script)

Together these do everything needed. Now the video sources are independent of location and can be shared everywhere.

Would be great to have a native implementation of the Network Tuner.. instead of a script. The native implementation could automatically pick out the correct tuning plugin for a video source ... the script has to be preconfigured.

[sflamm]

Another great use of the Network Tuner would be for an On-Screen remote control...

When using a Network Tuner to get on-demand channels from a cable/satellite provider - it is necessary to have a UI that opens to act as an On-Screen remote. The On-Screen remote can issue the remote tuning commands to the Network Tuner.
NOTE: the Network Tuner can issue any IR command not just channels.

In fact I was looking around to see if anyone has developed an On-screen remote control that can be plugged in an STV like SageMC. Is there one?

[EnterNoEscape]

Originally Posted by tmiranda

If you are updating the splitter please save me some time and make it 64 bit friendly (I think there are Windows native pieces).

That's not quite the same thing. The splitter I'm talking about is written in Java and uses the media server so we don't need to duplicate work between Windows and Linux to be able to write multiple files with one encoder stream. The end goal is to eventually get to a place where I can completely remove the native Windows capture device support from SageTV and place it in it's own fully native service that SageTV communicates with like a network encoder. I think that's our best bet to simplify and re-write the entire Windows piece with modern methods of accessing BDA capture devices. It also will make troubleshooting and debugging incredibly easier. Debugging native code in a JVM is not a good experience to say the least.

I want to do something similar with Linux too, however there isn't as much to update in the V4L2 code, so I probably would just leave it alone, separate it into it's own JVM or fully native network encoder. Separating the native encoder support makes SageTV incredibly more portable because you then have the option to install it on devices that only have the capacity to support network based devices without it blowing up from all of the native code that won't function in that environment. It might actually make the memory footprint slim enough to run the server on some cheapish NAS solutions like the MyCloud.

[Fuzzy]

Originally Posted by sflamm

On the network encoder serving another network encoder... if we have the ability to have a Network Tuner to complement the Network Encoder then I don't think that is necessary. That's what the script is trying to achieve....

Network Encoder: OpenDCT
Network Tuner: Tune (script)

Together these do everything needed. Now the video sources are independent of location and can be shared everywhere.

Would be great to have a native implementation of the Network Tuner.. instead of a script. The native implementation could automatically pick out the correct tuning plugin for a video source ... the script has to be preconfigured.

I think that may be part of the confusion. What are you calling a 'Network Tuner? What function does this bit have? SageTV only has the concept of a 'Network Encoder', which is simply something sage can tell to record a given channel, and it returns the stream from that channel - how it does that, and over what medium it is done, and even physically where it is done, sage could care less about.

[Monedeath]

Originally Posted by EnterNoEscape

That's not quite the same thing. The splitter I'm talking about is written in Java and uses the media server so we don't need to duplicate work between Windows and Linux to be able to write multiple files with one encoder stream. The end goal is to eventually get to a place where I can completely remove the native Windows capture device support from SageTV and place it in it's own fully native service that SageTV communicates with like a network encoder. I think that's our best bet to simplify and re-write the entire Windows piece with modern methods of accessing BDA capture devices. It also will make troubleshooting and debugging incredibly easier. Debugging native code in a JVM is not a good experience to say the least.

So almost having it run like it is a plug-in for Sage TV, except it runs in its own memory space/container?
Or more specifically, it (the "native code tuner" for non-networked tuners) operates much like Opern DCT does now. Only as an Official sub-stream of SageTV itself?

Makes sense to me, probably would help considerably with maintainability of some of the code. It definitely should speed along deployment of a 64bit Windows Server. Probably should give Narflex a heads up on it so he can point out any likely outstanding issues with such an idea.

tmiranda probably likes hearing this news in some ways, while it doesn't get him out of him helping getting a working 64 bit SageTV compile out there(and eventually getting that code to work anyhow--many users are going to want to use non-networked/native tuners). It could potentially enable that effort to create a prototype branch using the existing protocols, and complete part of that effort sooner rather than later.

There probably are users out there already who are only running networked tuners and could use such a thing now.

[sflamm]

The reality is that the reason there are so many tuning plugins is that there are many tuning options (different IR options, FireWire, serial, URI). Many of these tuning options require local tuning hardware (e.g. IR blaster) - local to whatever physical hardware is being tuned (e.g. Cable box)

The tuning plugins need to leverage a framework - such that regardless of their individual implementation details they can be invoked universally. The invocation should be a standard REST interface. That is the definition of a Network Tuner. SageTV already has both the framework and the REST interface. It can easily serve as a Network Tuner.

By definition a Network Encoder can provide either or both:

A) Network Video Capture (video capture without tuning) "Network" implies the source Video Capture ability can be accessed remotely by a standard protocol
B) Network Tuning (tuning without video capture). ""Network" implies the source Tuning ability can be accessed remotely by a standard protocol

Ideally a Network Encoder should do both - but it NOT a requirement. So a Network Encoder is really a combination of a Network Video Capture and a Network Tuner. How a Network Encoder does the Video Capture and Tuning is invisible to the upstream client.

Examples:

a) HDHR - directly attached to source (e.g. Cable card) and exposes it for capture and tuning. Provides Network Video Capture and Network Tuning
b) Cable box - directly attached to source and exposes Network Tuning only (for boxes that have URL tuning)
c) BM3000-HDMI - directly attached to source and exposes Network Video Capture only

SageTV does both when it is configured as a Network Encoder - providing unified capture and tuning. On the Video Capture front it has good support for local capture devices but no support for Network Video Capture devices (that's where SageDCT came in). On the tuning side SageTV has a mature framework and ecosystem of tuning options (existing plugins automatically configured upon selection)

OpenDCT on the other hand has Network Video Capture support only (but no local capture support) and very primitive Network tuning (providing limited command line tuning only - no plugin framework and limited manually configured tuning solutions..) NOTE: OpenDCT working with an HDHR is exactly this pattern - a) the HDHR is a Network Encoder capable of both Network Video Capture and Network Tuning b) OpenDCT exposes itself as the monolithic Network Encoder doing both (nothing upstream needs to know how it captures / tunes which is via the HDHR)


These could be unified by:

1) allowing SageTV to treat Network Encoders like OpenDCT as if it were a local capture source (my previous suggestion). In this regard the external Network Encoder to SageTV is allowed to be configured to serve only as a Network Video Capture device - in the end it is a video stream just like any local capture device. Some Encoders might be able to tune and some might not. For those like OpenDCT which don't have support for a variety of automated tuning plugins - SageTV can provide the tuning capability. When SageTV is configured as a Network Encoder it can provide capture (via external Network Encoder - e.g. OpenDCT or local capture device e.g. HDPVR) and tuning (via the tuning plugin e.g. USB-UIRT plugin). To its upstream clients the SageTV Network Encoder itself looks like a monolithic Network Encoder capable of BOTH capture and tuning.

Examples of how this would work in SageTV:

a) HDHR would be configured as a video source with tuning capability of its own (no plugin configuration needs to be selected)
b) HDPVR would be configured as a video source that does NOT do its own tuning (plugin would be selected)
c) OpenDCT could be configured either i) as a video source that cannot tune - a tuning plugin would need to be configured OR ii) a video source with tuning capability - no tuning plugin configuration necessary)

For all cases A - C SageTV configured as a Network Encoder presents itself to the outside world as a Network Encoder capable of BOTH capture and tuning

2) integrate OpenDCT into SageTV - essentially giving SageTV the ability to handle Network Video Capture directly.

3) Provide a Network Encoder (e.g. OpenDCT) which supports Network Video Capture but only partially supports tuning ability access to a Network Tuner (SageTV acting as Network Tuner). The script I wrote does basically this - it allows OpenDCT to invoke a standard Network Tuner. The Network Tuner is implemented against the uniform REST interface of SageTV whise framework abstracts the many possible plugin implementations that are loaded into its tuning framework.

[Fuzzy]

Originally Posted by Monedeath

So almost having it run like it is a plug-in for Sage TV, except it runs in its own memory space/container?
Or more specifically, it (the "native code tuner" for non-networked tuners) operates much like Opern DCT does now. Only as an Official sub-stream of SageTV itself?

Makes sense to me, probably would help considerably with maintainability of some of the code. It definitely should speed along deployment of a 64bit Windows Server. Probably should give Narflex a heads up on it so he can point out any likely outstanding issues with such an idea.

tmiranda probably likes hearing this news in some ways, while it doesn't get him out of him helping getting a working 64 bit SageTV compile out there(and eventually getting that code to work anyhow--many users are going to want to use non-networked/native tuners). It could potentially enable that effort to create a prototype branch using the existing protocols, and complete part of that effort sooner rather than later.

There probably are users out there already who are only running networked tuners and could use such a thing now.

Interestingly enough, SageTV used to HAVE a standalone network encoder for the native tuners a long time ago (called SageTVRecorder). That said, there are other native bits to be dealt with besides just the capture devices. Things like Power Management, as well as some other system management and monitoring tasks that hook into win32 api's as well will need ot be dealt with - either ported to 64-bit, but a different method to handle these bits.

[Fuzzy]

Originally Posted by sflamm

The reality is that the reason there are so many tuning plugins is that there are many tuning options (different IR options, FireWire, serial, URI). Many of these tuning options require local tuning hardware (e.g. IR blaster) - local to whatever physical hardware is being tuned (e.g. Cable box)

The tuning plugins need to leverage a framework - such that regardless of their individual implementation details they can be invoked universally. The invocation should be a standard REST interface. That is the definition of a Network Tuner. SageTV already has both the framework and the REST interface. It can easily serve as a Network Tuner.

By definition a Network Encoder can provide either or both:

A) Network Video Capture (video capture without tuning) "Network" implies the source Video Capture ability can be accessed remotely by a standard protocol
B) Network Tuning (tuning without video capture). ""Network" implies the source Tuning ability can be accessed remotely by a standard protocol

Ideally a Network Encoder should do both - but it NOT a requirement. So a Network Encoder is really a combination of a Network Video Capture and a Network Tuner. How a Network Encoder does the Video Capture and Tuning is invisible to the upstream client.

Examples:

a) HDHR - directly attached to source (e.g. Cable card) and exposes it for capture and tuning. Provides Network Video Capture and Network Tuning
b) Cable box - directly attached to source and exposes Network Tuning only (for boxes that have URL tuning)
c) BM3000-HDMI - directly attached to source and exposes Network Video Capture only

SageTV does both when it is configured as a Network Encoder - providing unified capture and tuning. On the Video Capture front it has good support for local capture devices but no support for Network Video Capture devices (that's where SageDCT came in). On the tuning side SageTV has a mature framework and ecosystem of tuning options (existing plugins automatically configured upon selection)

OpenDCT on the other hand has Network Video Capture support only (but no local capture support) and very primitive Network tuning (providing limited command line tuning only - no plugin framework and limited manually configured tuning solutions..) NOTE: OpenDCT working with an HDHR is exactly this pattern - a) the HDHR is a Network Encoder capable of both Network Video Capture and Network Tuning b) OpenDCT exposes itself as the monolithic Network Encoder doing both (nothing upstream needs to know how it captures / tunes which is via the HDHR)


These could be unified by:

1) allowing SageTV to treat Network Encoders like OpenDCT as if it were a local capture source (my previous suggestion). In this regard the external Network Encoder to SageTV is allowed to be configured to serve only as a Network Video Capture device - in the end it is a video stream just like any local capture device. Some Encoders might be able to tune and some might not. For those like OpenDCT which don't have support for a variety of automated tuning plugins - SageTV can provide the tuning capability. When SageTV is configured as a Network Encoder it can provide capture (via external Network Encoder - e.g. OpenDCT or local capture device e.g. HDPVR) and tuning (via the tuning plugin e.g. USB-UIRT plugin). To its upstream clients the SageTV Network Encoder itself looks like a monolithic Network Encoder capable of BOTH capture and tuning.

Examples of how this would work in SageTV:

a) HDHR would be configured as a video source with tuning capability of its own (no plugin configuration needs to be selected)
b) HDPVR would be configured as a video source that does NOT do its own tuning (plugin would be selected)
c) OpenDCT could be configured either i) as a video source that cannot tune - a tuning plugin would need to be configured OR ii) a video source with tuning capability - no tuning plugin configuration necessary)

For all cases A - C SageTV configured as a Network Encoder presents itself to the outside world as a Network Encoder capable of BOTH capture and tuning

2) integrate OpenDCT into SageTV - essentially giving SageTV the ability to handle Network Video Capture directly.

3) Provide a Network Encoder (e.g. OpenDCT) which supports Network Video Capture but only partially supports tuning ability access to a Network Tuner (SageTV acting as Network Tuner). The script I wrote does basically this - it allows OpenDCT to invoke a standard Network Tuner. The Network Tuner is implemented against the uniform REST interface of SageTV whise framework abstracts the many possible plugin implementations that are loaded into its tuning framework.

SageTV acting as a network encoder (meaning an instance of sagetv connecting to it's own capture devices and exposing those to be used by a master sagetv server that is handling the scheduling, recording, and ui presentation) has nothing to do with sagetv being able to connect to a capture device with a network port.

Additionally, SageTV already has the capability to use a local tuning plugin with a network encoder (implemented about 10 months ago), so you already should be able to use a usb-uirt connected to the main sagetv server to blast to a device that is recorded by a network encoder like openDCT, or another instance of sagetv.

There, essentially, should already be ways to use a BM3000-HDMI, for instance, with sagetv, without sage acting as a network encoder, or an external script calling back to sagex, etc. This is why I asked early on what your setup is, and why you were using a second sagetv server in the first place.

[EnterNoEscape]

Originally Posted by Fuzzy

Interestingly enough, SageTV used to HAVE a standalone network encoder for the native tuners a long time ago (called SageTVRecorder). That said, there are other native bits to be dealt with besides just the capture devices. Things like Power Management, as well as some other system management and monitoring tasks that hook into win32 api's as well will need ot be dealt with - either ported to 64-bit, but a different method to handle these bits.

I was wondering if SageTVRecorder was fully native or not. I also like the idea what the JVM could potentially be running 64-bit and the encoders could be 32-bit if it makes things more driver compatible. Though I have a hard time believe that modern BDA devices wouldn't have proper 64-bit drivers. If we extend the encoder protocol enough, the experience could be exactly the same as if it was still inside of the JVM. We don't necessarily need to use the network for this either, but it would probably be less confusing for some users if we did.

Power management for 64-bit should be fairly easy compared to the encoder architecture. I've had a taste of it in OpenDCT and it wasn't too hard to set up. I've played with converting to 64-bit and the biggest pain I saw was getting the encoder bits to play along. The remuxer comes over fine. The Sage.time() function is ok. I didn't test out any of the tuning plugins.

[Fuzzy]

Originally Posted by EnterNoEscape

I was wondering if SageTVRecorder was fully native or not. I also like the idea what the JVM could potentially be running 64-bit and the encoders could be 32-bit if it makes things more driver compatible. Though I have a hard time believe that modern BDA devices wouldn't have proper 64-bit drivers. If we extend the encoder protocol enough, the experience could be exactly the same as if it was still inside of the JVM. We don't necessarily need to use the network for this either, but it would probably be less confusing for some users if we did.

Power management for 64-bit should be fairly easy compared to the encoder architecture. I've had a taste of it in OpenDCT and it wasn't too hard to set up. I've played with converting to 64-bit and the biggest pain I saw was getting the encoder bits to play along. The remuxer comes over fine. The Sage.time() function is ok. I didn't test out any of the tuning plugins.

Ideally, the tuning plugins would move to the encoder piece anyway, so sage just has to send a channel to the encoder in the START command, so they can stay 32-bit just fine. Also, yes, the recorder was still at least partially java based. (min requirements list Java 1.4) I'm guessing it was actually still using Sage.jar, just with a different launch point (similar to how standalone, service, and client are today. I think the current launcher code on github still has the launch points in it for the recorder.

[sflamm]

Additionally, SageTV already has the capability to use a local tuning plugin with a network encoder (implemented about 10 months ago), so you already should be able to use a usb-uirt connected to the main sagetv server to blast to a device that is recorded by a network encoder like openDCT, or another instance of sagetv.

There, essentially, should already be ways to use a BM3000-HDMI, for instance, with sagetv, without sage acting as a network encoder, or an external script calling back to sagex, etc. This is why I asked early on what your setup is, and why you were using a second sagetv server in the first place.

A SageTV server can indeed add a video source using the OpenDCT for Video Capture and provide a Tuner.

However, once that is video source is setup - if you configure the same SageTV server as a network encoder it WILL NOT provide that video source as a Network Encoder to any other upstream SageTV

This is exactly what doesn't work. See my previous messages and photos.

Currently SageTV configured as a Network Encoder will not advertise to upstream clients any video source of the SageTV server which itself uses a Network Encoder

[sflamm]

As you said - SageTV used to have SageTVRecorder. This was folded into the SageTV server itself (due to licensing and convenience).

The idea was simple - a Network Encoder should advertise upstream the video stream and provide capture and tuning.

That is the model that SageTV uses (correct model) - properly when the video capture is local.

However, the addition of OpenDCT breaks that model - it does not advertise video sources upstream if the video capture is from another Network Encoder.
That is the issue that needs to be addressed.

The Network Encoder is a black box - with a standard API for tuning and managing capture of a video stream. How it implements itself should remain a black box.

A SageTV with ANY video source that is configured should be able to advertise that video source upstream to any client when the SageTV Server is configured as a Network Encoder.

[sflamm]

I detailed this in the previous posts:

https://forums.sagetv.com/forums/sho...postcount=1741
https://forums.sagetv.com/forums/sho...postcount=1742
https://forums.sagetv.com/forums/sho...postcount=1744

[Fuzzy]

Originally Posted by sflamm

A SageTV server can indeed add a video source using the OpenDCT for Video Capture and provide a Tuner.

However, once that is video source is setup - if you configure the same SageTV server as a network encoder it WILL NOT provide that video source as a Network Encoder to any other upstream SageTV

This is exactly what doesn't work. See my previous messages and photos.

Currently SageTV configured as a Network Encoder will not advertise to upstream clients any video source of the SageTV server which itself uses a Network Encoder

My question about that still stands as to WHY you have a separate server set up as a network encoder, just to connect to a different network encoder. What other function is this second server providing you? Why is your main server not connecting directly to openDCT itself?

[sflamm]

My main Server is connected to its own instance of OpenDCT which serves 6 tuners against 2x HDHRs.

The second machine is local to my STBs that are physically located elsewhere. The second Server (Network Encoder) is advertising the BM3000-HDMIs attached via HDMI to the STBs and being tuning the STBs in 3 zones of USB-UIRT. (These handle all protected content)

[sflamm]

This worked perfectly before when the STBs were connected to HDPVRs.
It should exactly the work the same with a different Video Capture device.

[EnterNoEscape]

Originally Posted by Monedeath

Makes sense to me, probably would help considerably with maintainability of some of the code. It definitely should speed along deployment of a 64bit Windows Server. Probably should give Narflex a heads up on it so he can point out any likely outstanding issues with such an idea.

It's pretty far out into the future at this time, so I will discuss it with Jeff when I get there/if I get there. I usually like to test the waters myself before I go creating an issue on GitHub because to me, creating a new feature issue to me is like saying that I will be carrying it out to completion or impassible failure. I don't like submitting them until I have reasonable confidence that I have the time and skill to get the job done right. I also prefer to show a detailed outline of what is about to be done so that people can appropriately point out things I may have missed.

[Fuzzy]

Originally Posted by sflamm

My main Server is connected to its own instance of OpenDCT which serves 6 tuners against 2x HDHRs.

The second machine is local to my STBs that are physically located elsewhere. The second Server (Network Encoder) is advertising the BM3000-HDMIs attached via HDMI to the STBs and being tuning the STBs in 3 zones of USB-UIRT. (These handle all protected content)

Get SageTV off of that remote system, install openDCT only, and use the batch file I posted called from openDCT, and you'd have a completely working network encoder for your BM3000-HDMIs, complete with tuning, all located next to the boxes they are tuning. This is what I was trying to get at. You are trying making the entire setup more complicated than it needs to be. This is the proper configuration for it.

[Fuzzy]

Originally Posted by sflamm

This worked perfectly before when the STBs were connected to HDPVRs.
It should exactly the work the same with a different Video Capture device.

It worked perfectly when the devices were being hosted by the sagetv instance - that is not the case anymore, as they are not hosted by openDCT, so openDCT is the network encoder to connect to.