SageTV Server Linux with Comskip for Docker (and unRAID) Support Thread

[stuckless]

Originally Posted by BobPhoenix

Been getting those myself. So far I've just ignored them. May have had a few videos without comskip info but haven't paid too much attention.

Originally Posted by alfi33

Are you processing live_tv or are you waiting for the recording to finish before starting Comskip processing?

Out of curiosity... I checked my syslog on unRAID, and sure enough, I have a few of these as well. I haven't noticed any recordings without comskip info, so maybe it's just restarting and then fine. I don't use livetv at all, so it could be a week or two before i get to watching a particular show.

I wonder if there is a way to debug "where" this is happening in comskip. There hasn't been a code update to comskip since April 9th... so I think I'm running the latest from the source code.

[alfi33]

Originally Posted by stuckless

There hasn't been a code update to comskip since April 9th... so I think I'm running the latest from the source code.

Possibly its just a coincidence, but not too long ago I had a very similar issue (comskip process crashes while processing live TV) with the latest Windows version of Comskip - 81.092. So on that server, I ended up rolling back to the 81.076 which does not have this issue for me.

[BobPhoenix]

Originally Posted by alfi33

Are you processing live_tv or are you waiting for the recording to finish before starting Comskip processing?

I never watch live TV any more. I even record the news and watch it later.

[alfi33]

Originally Posted by BobPhoenix

I never watch live TV any more. I even record the news and watch it later.

Yeh, me neither, for the most part. But every now and then I'll start something before the recording is complete, and it sure is nice to have Comskip already crunching away on it....so on my Windows server, I had left it on because I could.

It's not really a big deal. I certainly can live with out it...or just have another Windows machine process recordings with Dirmon2 (or something) and an older version of Comskip. I'm mostly just curious if live_tv processing works for anybody with this version of Comskip...or is there something different and/or unique about my environment that causes it to fail.

[Taddeusz]

I've been playing around with SageTV on an Ubuntu VM with the native comskip. While I'm not sure what magic sauce is being used for the Docker image to get the native comskip running I can still ssh in and run it manually.

I can start 4 individual instances (4 separate ssh sessions) of comskip running on live recordings and eventually all but one seems to get "Segmentation fault (core dumped)".

I've had it running via Wine quite successfully before. One would think the native version would run better but maybe there's something particular about the way Linux does file systems that causes comskip to have issues doing live detection?

I know comskip is more accurate when not run on live recordings but I like the convenience of live detection. There are instances where I run rerun comskip while we're watching something because the detection was bad. But for the most part it works well enough live. Also, other than while things are recording I don't have to worry about timing of maintenance waiting for the comskip queue to empty.

[Taddeusz]

Could also be the throughput, or lack thereof, of the integrated SATA controller that I currently have my unraid drives on. The IBM m1015 SAS card is MUCH faster. Prior to putting one of those cards in my Windows SageTV server I was having all sorts of performance problems doing live comskip while also watching something. That card cleared up all performance issues.

[stuckless]

Originally Posted by Taddeusz

I've been playing around with SageTV on an Ubuntu VM with the native comskip. While I'm not sure what magic sauce is being used for the Docker image to get the native comskip running I can still ssh in and run it manually.

I can start 4 individual instances (4 separate ssh sessions) of comskip running on live recordings and eventually all but one seems to get "Segmentation fault (core dumped)".

I've had it running via Wine quite successfully before. One would think the native version would run better but maybe there's something particular about the way Linux does file systems that causes comskip to have issues doing live detection?

I know comskip is more accurate when not run on live recordings but I like the convenience of live detection. There are instances where I run rerun comskip while we're watching something because the detection was bad. But for the most part it works well enough live. Also, other than while things are recording I don't have to worry about timing of maintenance waiting for the comskip queue to empty.

I heard people say they had similar problems with windows comskip on the latest version... ie crashes. I'm wondering if this is more to do with the fact that back at the end of march they moved to ffmpeg3... this required some code changes. I've been tempted to build using the code prior to march 31 to see if that fixes this issue... If you want to test it... you can try checking out the code by tag, 'v0.81.089' (which sept 2015 timeframe) or even by date, (look at the commits, and then find a date that looks stable before march 31)

[Fuzzy]

Working on UnRaiding my house - and found a problem in the docker (at least it seems to be). Using the java7 docker, placeshifting into it cannot get past the 2nd step of the wizard. It asks for language, then for time zone, but doesn't come up with and time zones in the list. The java8 docker seems to get through just fine. Not positive this is a docker issue, or some other problem in newer sage linux builds, but thought I'd bring it up here just in case.

[stuckless]

Originally Posted by Fuzzy

Working on UnRaiding my house - and found a problem in the docker (at least it seems to be). Using the java7 docker, placeshifting into it cannot get past the 2nd step of the wizard. It asks for language, then for time zone, but doesn't come up with and time zones in the list. The java8 docker seems to get through just fine. Not positive this is a docker issue, or some other problem in newer sage linux builds, but thought I'd bring it up here just in case.

pretty sure it's a java 7 issue several people have encountered this... My recommendation is to use the Java 8 container. This may actually be a Java 7 (OpenJDK) issue, since I'm using OpenJDK and not not Oracle JDK in the containers. I haven't encountered it, since I'm using Java 8 myself.

It would be interesting to see if there is an error in the sagetv_0.txt log file.

[Fuzzy]

Originally Posted by stuckless

pretty sure it's a java 7 issue several people have encountered this... My recommendation is to use the Java 8 container. This may actually be a Java 7 (OpenJDK) issue, since I'm using OpenJDK and not not Oracle JDK in the containers. I haven't encountered it, since I'm using Java 8 myself.

It would be interesting to see if there is an error in the sagetv_0.txt log file.

I'll have to do some checking later, once TV is restored to my abode.. hehe. I do know I've run a test linux system for a bit, and haven't noticed the issue, but I'm pretty sure I was using Oracle 7, not OpenJDK - so it does seem to make sense - I'm guessing the JDK installed is the only significant difference between your two containers, right?

[stuckless]

Originally Posted by Fuzzy

I'll have to do some checking later, once TV is restored to my abode.. hehe. I do know I've run a test linux system for a bit, and haven't noticed the issue, but I'm pretty sure I was using Oracle 7, not OpenJDK - so it does seem to make sense - I'm guessing the JDK installed is the only significant difference between your two containers, right?

Java is literally the only difference between the 2 containers... and to be clear, they both use OpenJDK... just one 7 and the other is 8. The reason for OpenJDK was because it worked (at least java 8 that I use) and it's faster to setup than the Oracle JDK... but I could probably setup a Java 7 using Oracle pretty easily.

[Fuzzy]

Originally Posted by stuckless

Java is literally the only difference between the 2 containers... and to be clear, they both use OpenJDK... just one 7 and the other is 8. The reason for OpenJDK was because it worked (at least java 8 that I use) and it's faster to setup than the Oracle JDK... but I could probably setup a Java 7 using Oracle pretty easily.

Honestly, I'm personally not that concerned, as I can live with OpenJDK 8 on the server. Has anyone else had this issue with the 7 container? or am i the only one who's tried to use it? :-)

[Taddeusz]

Originally Posted by Fuzzy

Honestly, I'm personally not that concerned, as I can live with OpenJDK 8 on the server. Has anyone else had this issue with the 7 container? or am i the only one who's tried to use it? :-)

I had the same trouble with it not allowing me to pick the time zone with the Java 7 version.

[BobPhoenix]

Originally Posted by Taddeusz

I had the same trouble with it not allowing me to pick the time zone with the Java 7 version.

Me too. I did try to downgrade the Java8 docker to Java7 once it was setup. But I switched back to Java8 since I could still get to the WebUI with 8 anyway.

But if anyone is interested here is what I did:

Open a putty session to my unRAID box that I installed the docker on. Then entered the following:

docker exec -ti stuckless-sagetv-server-java8 bash
sudo add-apt-repository ppa:openjdk-r/ppa
sudo apt-get update
sudo apt-get install openjdk-7-jdk
sudo update-java-alternatives
sudo update-alternatives --config java

I believe the last one brought up a menu where I could pick the default java instance and I selected the 7 jdk.

[Taddeusz]

It seems logical to me that the Java 7 version is not installing all the necessary Java libraries. Particularly the one that includes all the time zone definitions.

[tmiranda]

Originally Posted by Taddeusz

.. While I'm not sure what magic sauce is being used for the Docker image to get the native comskip running ...

It is using an updated CommercialDetector that I created for Sean. I haven't released it publicly yet because it hasn't been thoroughly tested.

[Taddeusz]

Originally Posted by tmiranda

It is using an updated CommercialDetector that I created for Sean. I haven't released it publicly yet because it hasn't been thoroughly tested.

Is there a beta entry I can put in my SageTVPluginsDev.xml?

[tmiranda]

Originally Posted by Taddeusz

Is there a beta entry I can put in my SageTVPluginsDev.xml?

Yes. The issue has not been the plugin but rather comskip. You need to make sure all of the comskip dependencies are inatalled and that's a manual process which varies according to the flavor of linux you are running.

If you don't mind manually installing dependencies I can send you the manifest.

[Taddeusz]

Originally Posted by tmiranda

Yes. The issue has not been the plugin but rather comskip. You need to make sure all of the comskip dependencies are inatalled and that's a manual process which varies according to the flavor of linux you are running.

If you don't mind manually installing dependencies I can send you the manifest.

Thought I had all the dependencies installed since it compiles and runs from bash. However, don't mind at all since for now this is strictly for testing.

[Taddeusz]

I've been messing around with the Linux native comskip today. I tried going back to the pre-ffmpeg3 version and still got seg faults with live detection. All my Linux native comskip testing thus far has been done on unRAID either through the Docker image or through an Ubuntu VM.

I have another Ubuntu VM on a Windows vmware workstation instance that I'm currently running a test on with the latest version. Running comskip live on 4 HDHR recordings and so far it's been running nearly 30 minutes without issue. All 4 comskip tasks are still cranking.

I wonder if there is something particular about the xfs file system, about how unRAID has configured drive caching, or a combination of the two that is causing comskip to be unable to correctly detect the end of the file?

I'm going to continue this test and let it run for at least an hour total. Then I think I'm going to see if I can re-format my recording drive to xfs and test again. BTW, it's currently formatted ext4.