Moving Recordings From Windows to Linux

[Taddeusz]

Has anyone done the move from Windows to Linux? I have a weird issue where the existing recordings show they don't have a length and comskip breaks aren't positioned correctly most of the time. Any ideas why?

Attached are a few examples between my Windows and Linux servers.

[Monedeath]

http://forums.sagetv.com/forums/showthread.php?t=62662

Sounds comparable.

Was SageTV running in Linux while you moved the recordings?

[Taddeusz]

I didn't move recordings. I simply added the Windows shares, which are fstab mounted, as recording directories.

[Skirge01]

I've had this happen on my setup, but, as Monedeath said, only when the files were moved/copied to a new folder while Sage was doing a scan. I do believe the fix in the thread Monedeath pointed to is how I fixed it, though. You could at least give it a try.

As to why it happened... maybe it's a mismatch between some combination of wiz.bin/epg/properties files and their moving between 2 Sage setups and so the new Sage server got confused?

[Taddeusz]

I didn't even copy the Wiz.bin from Windows. All I did was added the shares that hold my recording directories to my Linux server. That's it.

It appears that it is probably somehow misreading the metadata that's at the end of each file and getting the duration wrong.

[turak]

Was SageTV running on the Windows server and generating new recordings while SageTV was running on the Linux server. I don't think that SageTV plays nice with files that are growing in it's TV directories which are not generated by that instance of SageTV.

[turak]

For instance...

Windows SageTV starts recording a show. Linux SageTV scans the shared TV directories 5 minutes after the recording on Windows started. Now the Linux SageTV finds the file and adds it to it's Wiz.bin as a 5 minute long recording.

That recording will always be a 5 minute recording as far as the Linux SageTV server is concerned.

The only way I know of to fix this is to shut down Linux SageTV. Remove the file from it's TV directory (or unmount the share). Start up Linux SageTV. Shutdown Linux SageTV. Put the file back (or remount share). Start up SageTV.

[stuckless]

Originally Posted by turak

Was SageTV running on the Windows server and generating new recordings while SageTV was running on the Linux server. I don't think that SageTV plays nice with files that are growing in it's TV directories which are not generated by that instance of SageTV.

This is actually a "feature" of network mounted filesytems Any time you mount a network filesystem, the OS (linux) will cache the filesystem for performance reasons. In NFS (and likely smbmount), you can disable such caching mechanisms, and the filesystem becomes slower, but, it will always be consistent to both platforms. So, my suggestion is disable caching of the network filesystem, and you'll likely ok.

[Taddeusz]

Originally Posted by turak

Was SageTV running on the Windows server and generating new recordings while SageTV was running on the Linux server. I don't think that SageTV plays nice with files that are growing in it's TV directories which are not generated by that instance of SageTV.

No, my Windows SageTV was not recording at the time. It doesn't really do much doing the day.

[Taddeusz]

Originally Posted by stuckless

This is actually a "feature" of network mounted filesytems Any time you mount a network filesystem, the OS (linux) will cache the filesystem for performance reasons. In NFS (and likely smbmount), you can disable such caching mechanisms, and the filesystem becomes slower, but, it will always be consistent to both platforms. So, my suggestion is disable caching of the network filesystem, and you'll likely ok.

While I would like to say this is the case I can actually physically copy files over to the Linux filesystem and the same thing happens.

[stuckless]

Originally Posted by Taddeusz

While I would like to say this is the case I can actually physically copy files over to the Linux filesystem and the same thing happens.

yeah, but do physically copy them through the mounted filesystem? As opposed to using FTP, or a USB drive.

[Taddeusz]

Originally Posted by stuckless

yeah, but do physically copy them through the mounted filesystem? As opposed to using FTP, or a USB drive.

I guess I fail to understand why that would make a difference.

[stuckless]

Originally Posted by Taddeusz

I guess I fail to understand why that would make a difference.

If the file was first accessed, while it was growing/changing, then the NFS "cached" portion will only reflect a portion of that file. Eventually the network mount might reconcile the file, but, if you've already copied it, then you might not have the entire file.

So, when you copy it, the linux os might think that it has copied the entire file, but in fact it is missing a part. This is only really an issue for files that are growing, and when you access the file via the network mount, during the time in which was growning/changing.

I think this would be the case for sagetv, for example, if you started recording it on windows, and then have a separate sagetv server looking at the same file. SageTV will read what it "knows" about that file and process it. Because the file was "accessed" not sure when it would be fully reconciled.

This might have nothing to do with your issues, but a FYI, when using a network filesystem. (I personally never use them, because I've run into these issues way to many times)

[Taddeusz]

Originally Posted by stuckless

If the file was first accessed, while it was growing/changing, then the NFS "cached" portion will only reflect a portion of that file. Eventually the network mount might reconcile the file, but, if you've already copied it, then you might not have the entire file.

So, when you copy it, the linux os might think that it has copied the entire file, but in fact it is missing a part. This is only really an issue for files that are growing, and when you access the file via the network mount, during the time in which was growning/changing.

I think this would be the case for sagetv, for example, if you started recording it on windows, and then have a separate sagetv server looking at the same file. SageTV will read what it "knows" about that file and process it. Because the file was "accessed" not sure when it would be fully reconciled.

This might have nothing to do with your issues, but a FYI, when using a network filesystem. (I personally never use them, because I've run into these issues way to many times)

That's not the problem at all. These are all recordings that are all already completed. No recordings are currently occurring.

[Taddeusz]

Originally Posted by stuckless

yeah, but do physically copy them through the mounted filesystem? As opposed to using FTP, or a USB drive.

Just to test I enabled the IIS FTP server on my Windows machine and copied a couple recordings over that way. It didn't make any difference at all.

[lgkent]

This appears to be a bug adding any 'new' file to the tv recording directory regardless of which system recorded it. Smaller files <1GB tend to work. Here is debug log example of adding 2 new files, 'ChristmasontheDanube' (2.91GB) has a 30 minute length, but adds with segment length < 1 second. 'NaziMegaWeapons' (735.7MB) has 11 minute length, and correctly adds with segment length of 11 minutes. This log file came from SageTV V9.0.3.178 on Ubuntu 14.04

Fri 12/25 12:08:05.966 [main@3764951d] Checking video directories for new files
Fri 12/25 12:08:05.987 [main@3764951d] Using internal format detector first for: /var/media/recordings/ChristmasontheDanube-2777762-0.mpg
Fri 12/25 12:08:05.995 [main@3764951d] File Format Parsed-2a /var/media/recordings/ChristmasontheDanube-2777762-0.mpg=MPEG2-PS 0:00:01 2815 kbps [#0 Video[MPEG2-Video 29.97003 fps 1920x1080 16:9 interlaced id=e0]#1 Audio[AC3 48000 Hz 6 channels 384 kbps MAIN idx=1 id=bd-80010000 eng]#2 Audio[AC3 48000 Hz 1 channels 128 kbps idx=3 id=bd-82010000 eng]#3 Audio[AC3 48000 Hz 1 channels 128 kbps idx=2 id=bd-81010000 spa]]
Fri 12/25 12:08:06.009 [main@3764951d] File Format Parsed-2b /var/media/recordings/ChristmasontheDanube-2777762-0.mpg=MPEG2-PS 0:00:01 2815 kbps [#0 Video[MPEG2-Video 29.97003 fps 1920x1080 16:9 interlaced id=e0]#1 Audio[AC3 48000 Hz 6 channels 384 kbps MAIN idx=1 id=bd-80010000 eng]#2 Audio[AC3 48000 Hz 1 channels 128 kbps idx=3 id=bd-82010000 eng]#3 Audio[AC3 48000 Hz 1 channels 128 kbps idx=2 id=bd-81010000 spa]{Description=Christmas markets come alive in Austria's Vienna and Salzburg, Hungary's Budapest, and Switzerland's Oberdorf; Melk Abbey., AiringTime=1451003400000, ParentalRating=TVG, Genre=Special;Documentary, ForcedUnique=true, OriginalAirDate=1384646400000, AiringDuration=1800000, ExternalID=SH018094060000, Title=Christmas on the Danube, CC=true, Stereo=true}]
Fri 12/25 12:08:06.010 [main@3764951d] Increased StringBuffer pool to size=1
Fri 12/25 12:08:06.015 [main@3764951d] Added:MediaFile[id=98898 A[98899,34283,"Christmas on the Danube",0@1224.19:30,30,T V] mask=TV host=sagetv1 encodedBy= format=MPEG2-PS 0:00:01 2815 kbps [#0 Video[MPEG2-Video 29.97003 fps 1920x1080 16:9 interlaced id=e0]#1 Audio[AC3 48000 Hz 6 channels 384 kbps MAIN idx=1 id=bd-80010000 eng]#2 Audio[AC3 48000 Hz 1 channels 128 kbps idx=3 id=bd-82010000 eng]#3 Audio[AC3 48000 Hz 1 channels 128 kbps idx=2 id=bd-81010000 spa]] /var/media/recordings/ChristmasontheDanube-2777762-0.mpg, Seg0[Thu 12/24 19:59:58.489-Thu 12/24 20:00:00.000]] num=3944
Fri 12/25 12:08:06.016 [main@3764951d] Using internal format detector first for: /var/media/recordings/NaziMegaWeapons-2764929-0.mpg
Fri 12/25 12:08:06.019 [main@3764951d] File Format Parsed-2a /var/media/recordings/NaziMegaWeapons-2764929-0.mpg=MPEG2-PS 0:10:58 2815 kbps [#0 Video[MPEG2-Video 29.97003 fps 1920x1080 16:9 interlaced id=e0]#1 Audio[AC3 48000 Hz 6 channels 384 kbps MAIN idx=1 id=bd-80010000 eng]#2 Audio[AC3 48000 Hz 1 channels 128 kbps idx=3 id=bd-82010000 eng]#3 Audio[AC3 48000 Hz 1 channels 128 kbps idx=2 id=bd-81010000 spa]]
Fri 12/25 12:08:06.037 [main@3764951d] File Format Parsed-2b /var/media/recordings/NaziMegaWeapons-2764929-0.mpg=MPEG2-PS 0:10:58 2815 kbps [#0 Video[MPEG2-Video 29.97003 fps 1920x1080 16:9 interlaced id=e0]#1 Audio[AC3 48000 Hz 6 channels 384 kbps MAIN idx=1 id=bd-80010000 eng]#2 Audio[AC3 48000 Hz 1 channels 128 kbps idx=3 id=bd-82010000 eng]#3 Audio[AC3 48000 Hz 1 channels 128 kbps idx=2 id=bd-81010000 spa]{Description=The engineering secrets of six structures that changed the face of warfare., AiringTime=1450152000000, ParentalRating=TVPG, Genre=Documentary;Science, OriginalAirDate=1374105600000, AiringDuration=660000, ExternalID=SH017381900000, Title=Nazi Mega Weapons, CC=true, Stereo=true}]
Fri 12/25 12:08:06.039 [main@3764951d] Added:MediaFile[id=98900 A[98901,7183,"Nazi Mega Weapons",0@1214.23:00,11,T V] mask=TV host=sagetv1 encodedBy= format=MPEG2-PS 0:10:58 2815 kbps [#0 Video[MPEG2-Video 29.97003 fps 1920x1080 16:9 interlaced id=e0]#1 Audio[AC3 48000 Hz 6 channels 384 kbps MAIN idx=1 id=bd-80010000 eng]#2 Audio[AC3 48000 Hz 1 channels 128 kbps idx=3 id=bd-82010000 eng]#3 Audio[AC3 48000 Hz 1 channels 128 kbps idx=2 id=bd-81010000 spa]] /var/media/recordings/NaziMegaWeapons-2764929-0.mpg, Seg0[Mon 12/14 23:00:01.673-Mon 12/14 23:11:00.000]] num=3945

[Narflex]

Yeah, I can see the format is mis-detecting the duration on those files. Can you upload an example file somewhere so this can be reproduced and looked into?