SageTV Server Linux with Comskip for Docker (and unRAID) Support Thread

[SomeWhatLost]

fwiw, I never got external/network/smb paths to work, but as all my stuff is local/unraid it really wasn't an issue for me...

to do local paths I just use \unraid\[folder]

[SomeWhatLost]

also, not sure it matters any, or helps in anyway, but it does seem somewhat cleaner, it is easy enough to add additional "paths" to the container, but I wonder if those will get blown away if the container gets updated?
ie add a /var/Movies that maps to the movies dir/etc...

[stuckless]

Originally Posted by stanger89

Do we have to blow away the docker and re-add it? I'm still on unraid 6.1.7 and I don't see an update. Or does just restarting the docker (which seems to have updated Sage) do it?

I'm also having trouble getting my import dirs to "work". I can successfully browse to them as samba shares, and that all works well, but when Sage goes to scan them, it finds nothing, this is all I see in the log:

Sun 7/3 10:35:23.042 [LibraryImporter@6f683bdf] Checking video directories for new files
Sun 7/3 10:35:23.542 [LibraryImporter@6f683bdf] Seeker is starting the library import scan...
Sun 7/3 10:35:23.543 [LibraryImporter@6f683bdf] Starting to scan lib import root: /tmp/sagetv_shares/unraid/blu-ray
Sun 7/3 10:35:23.573 [LibraryImporter@6f683bdf] Starting to scan lib import root: /tmp/sagetv_shares/unraid/tv
Sun 7/3 10:35:23.603 [LibraryImporter@6f683bdf] Starting to scan lib import root: /tmp/sagetv_shares/unraid/media/Processed Pics
Sun 7/3 10:35:23.603 [LibraryImporter@6f683bdf] Skipping import directory because it's offline: /tmp/sagetv_shares/unraid/media/Processed Pics
Sun 7/3 10:35:23.603 [LibraryImporter@6f683bdf] Seeker has finished the library import scan.
S

Restarting the docker will update sagetv, but not the docker container. In 6.2, in the Docker tab, each docker has a "force update" link.

You can force the update as well by blowing away the container, and then re-adding it.

Not sure what /tmp/sagetv_shares are... but, all of you media is available to sagetv via the /var/media /var/mediaext/ or by using /unraid/PATH_TO_DATA...

Originally Posted by SomeWhatLost

also, not sure it matters any, or helps in anyway, but it does seem somewhat cleaner, it is easy enough to add additional "paths" to the container, but I wonder if those will get blown away if the container gets updated?
ie add a /var/Movies that maps to the movies dir/etc...

If you add your own paths, they should remain when the container is updated.

[stanger89]

So maybe it's a misunderstanding that needs to be cleared up, but for as long as I can remember, if you had plans of using Clients (not extenders), and wanted things like Comskip to work, it was best to use UNC paths, even for content local to the SageTV server.

The reason was, the paths were then universal throughout the network. If you used local paths, Clients would be able to play content (the server would stream it) but comskip wouldn't work, since the paths would be wrong.

Is that not the case with Linux?

I thought there were similar issues with Fanart.

That's why I've been trying to set things up with network/UNC paths.

Originally Posted by stuckless

Not sure what /tmp/sagetv_shares are... but, all of you media is available to sagetv via the /var/media /var/mediaext/ or by using /unraid/PATH_TO_DATA...

the /tmp/sagetv_shares is what you get when you configure a network location, it seems there's a permissions problem:

Sun 7/3 12:23:47.461 [EventRouter-305a3a536ac0@40cb6a51] Executing process: [sh, -c, mount -t smbfs | grep -i "/tmp/sagetv_shares/unraid/blu-ray"]
Sun 7/3 12:23:47.461 [Seeker@522a2cb0] Seeker waiting for 153722842821469 mins.
Sun 7/3 12:23:47.461 [EventRouter-305a3a536ac0@40cb6a51] Started process object: java.lang.UNIXProcess@17a2c44c
Sun 7/3 12:23:47.462 [AsyncPropSaver@5783494b] Done writing out the data to the properties file
Sun 7/3 12:23:47.463 [EventRouter-305a3a536ac0@40cb6a51] SMB Mount Path: //unraid/blu-ray /tmp/sagetv_shares/unraid/blu-ray
Sun 7/3 12:23:47.463 [EventRouter-305a3a536ac0@40cb6a51] Executing process: [mount.cifs, //unraid/blu-ray, /tmp/sagetv_shares/unraid/blu-ray, -o, guest,iocharset=utf8]
Sun 7/3 12:23:47.463 [EventRouter-305a3a536ac0@40cb6a51] Started process object: java.lang.UNIXProcess@52b69a4a
Sun 7/3 12:23:47.464 [ErrorStreamConsumer@7a7bb082] STDERR:java.lang.UNIXProcess@52b69a4a: mount.cifs: permission denied: no match for /tmp/sagetv_shares/unraid/blu-ray found in /etc/fstab
Sun 7/3 12:23:47.464 [EventRouter-305a3a536ac0@40cb6a51] SMB Mount Failed
Sun 7/3 12:23:47.464 [LibraryImporter@5783494b] Checking video directories for new files
Sun 7/3 12:23:47.964 [LibraryImporter@5783494b] Seeker is starting the library import scan...
Sun 7/3 12:23:47.965 [LibraryImporter@5783494b] Starting to scan lib import root: /tmp/sagetv_shares/unraid/blu-ray
Sun 7/3 12:23:47.995 [LibraryImporter@5783494b] Seeker has finished the library import scan.

Not sure if the docker is ubuntu based (actually it seems it's a cifs-utils thing), but it looks like at some point ubuntu changed mount.cifs so that by default only root can use it:
https://bugs.launchpad.net/ubuntu/+s...ls/+bug/657900

[stuckless]

Originally Posted by stanger89

So maybe it's a misunderstanding that needs to be cleared up, but for as long as I can remember, if you had plans of using Clients (not extenders), and wanted things like Comskip to work, it was best to use UNC paths, even for content local to the SageTV server.

The reason was, the paths were then universal throughout the network. If you used local paths, Clients would be able to play content (the server would stream it) but comskip wouldn't work, since the paths would be wrong.

Is that not the case with Linux?

I thought there were similar issues with Fanart.

That's why I've been trying to set things up with network/UNC paths.

the /tmp/sagetv_shares is what you get when you configure a network location, it seems there's a permissions problem:

Sun 7/3 12:23:47.461 [EventRouter-305a3a536ac0@40cb6a51] Executing process: [sh, -c, mount -t smbfs | grep -i "/tmp/sagetv_shares/unraid/blu-ray"]
Sun 7/3 12:23:47.461 [Seeker@522a2cb0] Seeker waiting for 153722842821469 mins.
Sun 7/3 12:23:47.461 [EventRouter-305a3a536ac0@40cb6a51] Started process object: java.lang.UNIXProcess@17a2c44c
Sun 7/3 12:23:47.462 [AsyncPropSaver@5783494b] Done writing out the data to the properties file
Sun 7/3 12:23:47.463 [EventRouter-305a3a536ac0@40cb6a51] SMB Mount Path: //unraid/blu-ray /tmp/sagetv_shares/unraid/blu-ray
Sun 7/3 12:23:47.463 [EventRouter-305a3a536ac0@40cb6a51] Executing process: [mount.cifs, //unraid/blu-ray, /tmp/sagetv_shares/unraid/blu-ray, -o, guest,iocharset=utf8]
Sun 7/3 12:23:47.463 [EventRouter-305a3a536ac0@40cb6a51] Started process object: java.lang.UNIXProcess@52b69a4a
Sun 7/3 12:23:47.464 [ErrorStreamConsumer@7a7bb082] STDERR:java.lang.UNIXProcess@52b69a4a: mount.cifs: permission denied: no match for /tmp/sagetv_shares/unraid/blu-ray found in /etc/fstab
Sun 7/3 12:23:47.464 [EventRouter-305a3a536ac0@40cb6a51] SMB Mount Failed
Sun 7/3 12:23:47.464 [LibraryImporter@5783494b] Checking video directories for new files
Sun 7/3 12:23:47.964 [LibraryImporter@5783494b] Seeker is starting the library import scan...
Sun 7/3 12:23:47.965 [LibraryImporter@5783494b] Starting to scan lib import root: /tmp/sagetv_shares/unraid/blu-ray
Sun 7/3 12:23:47.995 [LibraryImporter@5783494b] Seeker has finished the library import scan.

Not sure if the docker is ubuntu based (actually it seems it's a cifs-utils thing), but it looks like at some point ubuntu changed mount.cifs so that by default only root can use it:
https://bugs.launchpad.net/ubuntu/+s...ls/+bug/657900

Yeah, if you are using Clients (which I don't use), then you have all sorts of path related issues, as you mentioned.

As for the cifs error is that ONLY since I changed sagetv to run as a non-root user? Or has it always bee n this way, for your docker container?

[stanger89]

It might be? But sadly I didn't check. I haven't really played with the docker much, my VM has been running fine and I haven't taken any time to get the docker really set up. I had some free time this weekend and thought I'd take another look.

From reading the bug thread, it sounds like if you were running as root before, it wouldn't have been a problem, but I couldn't get network shares to work before I updated today either....

I guess I do have a log for that, but it looks like I must never have updated the docker since before you added mount.cifs:

Sun 7/3 10:30:52.364 [main@76ed5528] Error executing process mount.cifs : java.io.IOException: Cannot run program "mount.cifs": error=2, No such file or directory

So that would explain why they didn't work before.

[alfi33]

Originally Posted by stanger89

So maybe it's a misunderstanding that needs to be cleared up, but for as long as I can remember, if you had plans of using Clients (not extenders), and wanted things like Comskip to work, it was best to use UNC paths, even for content local to the SageTV server.

The reason was, the paths were then universal throughout the network. If you used local paths, Clients would be able to play content (the server would stream it) but comskip wouldn't work, since the paths would be wrong.

With the latest version of the Comskip Playback plugin, this is not required. It seems to be working fine on the Windows Client I've been testing with.

Fanart also does not seem to need UNC paths by my Client. Although I've had a little weirdness going on with Fanart (and/or Gemstone), I don't think it's because of pathing since it's working just fine for the most part.

[EnterNoEscape]

Originally Posted by alfi33

Fanart also does not seem to need UNC paths by my Client. Although I've had a little weirdness going on with Fanart (and/or Gemstone), I don't think it's because of pathing since it's working just fine for the most part.

Fanart doesn't need to be shared. From what I have observed, the client will get it's own if it's not pointed to the same fanart share as the server.

[stanger89]

So, fanart... if I customize it for a show (pick something other than default) will that be picked up by a client too?

[Fuzzy]

The clients need to be configured to point to the Central Fanart Folder, wherever that might be, but as far as i know, it doesn't have to be the same path as the server sees the files at. I think within phoenix, it addresses the files relative to whatever path is set in that client context's central fanart folder.

[stuckless]

I;m pretty sure that in BMT when you set a default, it's relative to the central fanart folder, and not an absolute path.

[EnterNoEscape]

Originally Posted by Fuzzy

The clients need to be configured to point to the Central Fanart Folder, wherever that might be, but as far as i know, it doesn't have to be the same path as the server sees the files at. I think within phoenix, it addresses the files relative to whatever path is set in that client context's central fanart folder.

I think that's closer to what actually happens. Ignore me.

[BobPhoenix]

I had this in my go file on unRAID to mount my network path on boot:

mkdir -p /mnt/media2

mount -t cifs -o rw,nounix,iocharset=utf8,_netdev,file_mode=0777,dir_mode=0777,username=guest,password=******* //Media2/ATSC /mnt/media2

And then if I needed to remount it because I stopped and restarted the docker or the array (without rebooting) I would uses this script to remount it.

umount /mnt/media2
samba stop

mkdir -p /mnt/media2

samba start

mount -t cifs -o rw,nounix,iocharset=utf8,_netdev,file_mode=0777,dir_mode=0777,username=guest,password=******* //Media2/ATSC /mnt/media2

This was when it was running as root don't know if changes are needed or if it is needed at all now that it is nobody.

[m1k3g]

Originally Posted by m1k3g

Hey thanks for that info! I've played around with the net=host and of course it never worked. I'm on DSM 5.2 so I'll have to upgrade to 6 at some point to get this working, it looks like. Since I'm running Xpenology rather than Synology software, it'll probably be awhile before I can do that. May try just upgrading the Docker package only first.

Thanks again.

I'm going to get this working yet! I've got DSM6 running in a VM with Docker installed and Sage installed in host mode. After banging my head against the wall several times, I still don't seem to be able to connect to it with a client.
Once host mode is enabled, there are no options for opening external/internal ports available, so I assumed that the SageTV client should just be pointed to the IP of the DSM6 volume. No luck. Where am I going wrong? Thanks.

[stuckless]

Originally Posted by m1k3g

I'm going to get this working yet! I've got DSM6 running in a VM with Docker installed and Sage installed in host mode. After banging my head against the wall several times, I still don't seem to be able to connect to it with a client.
Once host mode is enabled, there are no options for opening external/internal ports available, so I assumed that the SageTV client should just be pointed to the IP of the DSM6 volume. No luck. Where am I going wrong? Thanks.

My guess is that nesting a docker container inside a VM is the network issue here... The VM network would have to be configurated to expose all ports to the world and obtain an ipaddress that is the same as the host computer... and then the docker would have to be setup to do the same. Otherwise, I don't think the discover packets are going to escape both the VM and the Docker container.

Is there a reason why you are not just running the Docker container on the host directly? And or... just setting up the VM to run sagetv directly and forget about docker?

[m1k3g]

Originally Posted by stuckless

My guess is that nesting a docker container inside a VM is the network issue here... The VM network would have to be configurated to expose all ports to the world and obtain an ipaddress that is the same as the host computer... and then the docker would have to be setup to do the same. Otherwise, I don't think the discover packets are going to escape both the VM and the Docker container.

Is there a reason why you are not just running the Docker container on the host directly? And or... just setting up the VM to run sagetv directly and forget about docker?

Stupidity, most likely.. :-) I did kind of suspect that the 'network within a network within a network' probably had something to do with it. My ultimate goal is to have everything running on one box, since I have Xpenology (open source version of Synology v5) running on a nice small box with all the other HTPC utils, I had hoped to run Sage there as well, in Docker. But since v5 doesn't support Docker very well, I decided to run v6 in the VM. Ridiculously complicated, I know. I probably just need to wait for a bootable version of Synology DSM6.

[stuckless]

Originally Posted by m1k3g

Stupidity, most likely.. :-) I did kind of suspect that the 'network within a network within a network' probably had something to do with it. My ultimate goal is to have everything running on one box, since I have Xpenology (open source version of Synology v5) running on a nice small box with all the other HTPC utils, I had hoped to run Sage there as well, in Docker. But since v5 doesn't support Docker very well, I decided to run v6 in the VM. Ridiculously complicated, I know. I probably just need to wait for a bootable version of Synology DSM6.

I've recently become a fan of unRAID, myself I've run a number of different NAS solutions over the years, but, nothing compares to unRAID in terms of each of setup and maintenance, and not to mention the rich community that has created 100s of docker containers and plugins.

It's free to try (for 30 days)... and for $59 you can support up to 6 drives which fine for most people... or get support for 12 devices for $89.

[Taddeusz]

Originally Posted by stuckless

I've recently become a fan of unRAID, myself I've run a number of different NAS solutions over the years, but, nothing compares to unRAID in terms of each of setup and maintenance, and not to mention the rich community that has created 100s of docker containers and plugins.

It's free to try (for 30 days)... and for $59 you can support up to 6 drives which fine for most people... or get support for 12 devices for $89.

For me it would be nice to move everything to a single system but would mean a lot of time I really don't have with 2 wee ones.

I currently have 2 "server" systems. One is running vmware esxi, dual quad core 2.66Ghz Xeon LGA 771 system maxed out at 24GB RAM. Plenty fast but really kind of weak in the disk department. Running 2 Windows vm's, 3 Ubuntu vm's, and a single OS X vm.

My SageTV server is on a separate quad core machine with an 8 port SAS card with SATA drives.

I've toyed with the idea of moving SageTV to a vmware vm and doing drive pass-through. However, I learned that feature requires SCSI or SAS drives.

So, I'm really kind of stuck with this really kind of old but still fast enough Xeon system that runs mostly domain and other network stuff. Really kind of underutilized. Actually, with a family all my stuff is old but fast enough. Really have to scrape by with what I have and hope nothing too major fails.

Wish I could strip it all down and start over from scratch but I just don't have the time.

Enough with my sob story. How difficult is it to migrate NTFS drives to unRAID?

[stanger89]

Originally Posted by Taddeusz

Enough with my sob story. How difficult is it to migrate NTFS drives to unRAID?

Don't, don't run NTFS on linux, it's not stable. The only thing I'd "recommend" doing with NTFS on linux is read-only copying to a native linux FS.

As for unRAID, it supports reiserFS, XFS, and BTRFS for array/pool file systems.

[Taddeusz]

Originally Posted by stanger89

Don't, don't run NTFS on linux, it's not stable. The only thing I'd "recommend" doing with NTFS on linux is read-only copying to a native linux FS.

As for unRAID, it supports reiserFS, XFS, and BTRFS for array/pool file systems.

Of course. My issue is that I have almost 7.5TB of music & video across 4 1TB and 3 2TB drives. I could probably scrape up enough to buy a single 2TB drive to migrate data over but the process would be time consuming and cause downtime for the current SageTV server.

I might also be able to do migration in "blocks". Do the imported video & music first separately. Then do the recordings only after things are set up correctly.