Randomly deleted recordings?

[pilotguy7ca]

I've noticed in the past that occasionaly some of my saved recording were getting deleted and I thought it was due to either it thinking some needed to be deleted to make space or it was enforcing the "keep # of recordings" that I had set. But I checked this morning and during the night Sage had automatically deleted a bunch of recent recordings and with each one it had 3 lines like this in the log:

Sun 1/21 2:49:21.262 MediaFile failing verify testFile=\\mediasvr\d\media\sagetv\LateNightWithConanOBrien-1704923-0.mpg isFile=false len=2645884928 isTV=true isRecording=false
Sun 1/21 2:49:21.263 Removing MediaFile because it failed verification:MediaFile[id=1836852 A[1704923,1703730,"Late Night With Conan O'Brien",11387@0116.22:37,59] host=mediasvr encodedBy=SC-HD505A-2 on 127.0.0.1:6968 MPEG2 Max Quality format=MPEG2-TS 0:04:59 6000 kbps [#0 Video[MPEG2-Video 29.97003 fps 528x480 4:3 2581 kbps progressive]#1 Audio[AC3 48000 Hz 2 channels 192 kbps idx=1 id=14]] \\mediasvr\d\media\sagetv\LateNightWithConanOBrien-1704923-0.mpg, Seg0[Tue 1/16 22:37:00.011-Tue 1/16 23:36:00.002]]
Sun 1/21 2:49:21.276 Deleted media segment file \\mediasvr\d\media\sagetv\LateNightWithConanOBrien-1704923-0.mpg dtime=13 len=2645884928

It did this to about 6 recordings all from the past couple of days but left earlier ones alone. I've checked and it has nothing to do with space (over 250GB free) and for some of them I had "keep until space is needed". Anyways from the look of the log file Sage was trying to verify file but couldn't, whatever that means?

Any ideas? What sucks is that only about half the deleted shows I had been able to watch.

[Opus4]

You could send the logs to SageTV to see if they can tell part of verification is failing.

It is possible that there is some problem with the tuner or hard drive & the files are corrupted in some way.

- Andy

[gplasky]

Originally Posted by pilotguy7ca

I've noticed in the past that occasionaly some of my saved recording were getting deleted and I thought it was due to either it thinking some needed to be deleted to make space or it was enforcing the "keep # of recordings" that I had set. But I checked this morning and during the night Sage had automatically deleted a bunch of recent recordings and with each one it had 3 lines like this in the log:

Sun 1/21 2:49:21.262 MediaFile failing verify testFile=\\mediasvr\d\media\sagetv\LateNightWithConanOBrien-1704923-0.mpg isFile=false len=2645884928 isTV=true isRecording=false
Sun 1/21 2:49:21.263 Removing MediaFile because it failed verification:MediaFile[id=1836852 A[1704923,1703730,"Late Night With Conan O'Brien",11387@0116.22:37,59] host=mediasvr encodedBy=SC-HD505A-2 on 127.0.0.1:6968 MPEG2 Max Quality format=MPEG2-TS 0:04:59 6000 kbps [#0 Video[MPEG2-Video 29.97003 fps 528x480 4:3 2581 kbps progressive]#1 Audio[AC3 48000 Hz 2 channels 192 kbps idx=1 id=14]] \\mediasvr\d\media\sagetv\LateNightWithConanOBrien-1704923-0.mpg, Seg0[Tue 1/16 22:37:00.011-Tue 1/16 23:36:00.002]]
Sun 1/21 2:49:21.276 Deleted media segment file \\mediasvr\d\media\sagetv\LateNightWithConanOBrien-1704923-0.mpg dtime=13 len=2645884928

It did this to about 6 recordings all from the past couple of days but left earlier ones alone. I've checked and it has nothing to do with space (over 250GB free) and for some of them I had "keep until space is needed". Anyways from the look of the log file Sage was trying to verify file but couldn't, whatever that means?

Any ideas? What sucks is that only about half the deleted shows I had been able to watch.

I had the same mysterious problem that I finally figured out. I don't know if this would apply to you.

I had setup my HD Homerun. Last week I could of sworn my wife's programs were recording. Cold Case Files, American Justice, etc. But the next day they were not there. I saw them recording but they were not there. It appears my digital lineup at some point in time had all the analog channels enabled. Since HDTV is preferred it would try to use the HD Homerun to record the analog channels and of course nothing recorded. But it looked like it was recording. Unless you actually went in and tried to play it while it was recording. I'm not sure why those channels became enabled. It wasn't all the channels and the wife never tries to change that stuff. Once I disabled the channels everything recorded correctly and it was back to normal. Fluke or bug? I don't know and didn't file one.

Gerry

[Narflex]

That's quite interesting. According to the log:

isFile=false len=2645884928

The file did NOT exist (java.io.File.isFile() returns false) but the length is 2.6GB. So SageTV thought the file wasn't there because isFile returned false. (the length was just for debugging to try to figure out why stuff sometimes failed a verify like this even when it exists).

So I guess the best thing to do is update SageTV to use file size as a secondary check (because it'll be zero for files that don't exist).

This is probably related to some windows file networking bug since you're using UNC paths. But now we'll have a workaround for it. (I already submitted a bug report for you)

[pilotguy7ca]

Originally Posted by Narflex

That's quite interesting. According to the log:

isFile=false len=2645884928

The file did NOT exist (java.io.File.isFile() returns false) but the length is 2.6GB. So SageTV thought the file wasn't there because isFile returned false. (the length was just for debugging to try to figure out why stuff sometimes failed a verify like this even when it exists).

So I guess the best thing to do is update SageTV to use file size as a secondary check (because it'll be zero for files that don't exist).

This is probably related to some windows file networking bug since you're using UNC paths. But now we'll have a workaround for it. (I already submitted a bug report for you)

Thanks for the quick reply and to know that I wasn't going crazy and the files are disappearing without my doing!

Also I don't know if it would make a difference as well but I use a couple of r5000 moded boxes to do the recording, althought I've never had issues with playing any of the recordings and I double checked and some of the recording deleted were from a number of months ago, like a bunch of my LOST recordings . So I don't know how they would all of a sudden get corrupted.

Thanks again.

[Opus4]

I would ask if there could be some sort of permissions issue or temporary network unavailability, but then I don't know how it would report that it isn't a file but get a length for it...

- Andy

[owilsky]

I have a similar problem. Nearly 80% of my recordings are gone. I get some of them back by undeleting them, but some are gone for good.

I copied the restored files back and turned on logging.
They get deleted again, even new recorded shows!!!!

I find this in the logs:

So 5/13 21:27:10.191 Checking video directories for new files
So 5/13 21:27:10.200 Diskspace checking is running
So 5/13 21:27:10.201 Verifying existence of all TV media files in database fixDurs=false avoidArchive=true
So 5/13 21:27:10.202 MediaFile failing verify testFile=D:\Videos\DieRockies-DerRufdereisigenWildnis-6961630-0.mpg isFile=false len=0 isTV=true isRecording=false
So 5/13 21:27:10.203 Removing MediaFile because it failed verification:MediaFile[id=6982958 A[6961630,6961129,"Die Rockies",11589@1227.21:45,45] host=oscar encodedBy=Hauppauge WinTV PVR PCI II Capture  format=MPEG2-PS 0:00:00 0 kbps [#0 Video[MPEG2-Video 25.0 fps 720x576 4:3 3800 kbps interlaced]#1 Audio[MP2 48000 Hz 2 channels 384 kbps]] D:\Videos\DieRockies-DerRufdereisigenWildnis-6961630-0.mpg, Seg0[Mi 12/27 21:40:00.016-Mi 12/27 22:35:00.075]]
So 5/13 21:27:10.203 Deleted media segment file D:\Videos\DieRockies-DerRufdereisigenWildnis-6961630-0.mpg dtime=0 len=0
So 5/13 21:27:10.258 MediaFile failing verify testFile=D:\Videos\DieRockies-SehnsuchtnachdemaltenWesten-6961667-0.mpg isFile=false len=0 isTV=true isRecording=false
So 5/13 21:27:10.259 Removing MediaFile because it failed verification:MediaFile[id=6986263 A[6961667,6961131,"Die Rockies",11589@1228.21:45,45] host=oscar encodedBy=Hauppauge WinTV PVR PCI II Capture  format=MPEG2-PS 0:00:00 0 kbps [#0 Video[MPEG2-Video 25.0 fps 720x576 4:3 3800 kbps interlaced]#1 Audio[MP2 48000 Hz 2 channels 384 kbps]] D:\Videos\DieRockies-SehnsuchtnachdemaltenWesten-6961667-0.mpg, Seg0[Do 12/28 21:40:00.016-Do 12/28 22:35:00.006]]
So 5/13 21:27:10.260 Deleted media segment file D:\Videos\DieRockies-SehnsuchtnachdemaltenWesten-6961667-0.mpg dtime=0 len=0

This continues for a lot of files!

Always isFile=false len=0 isTV=true isRecording=false,
but the files sizes were NOT 0!!!! they are real recording files, >4GB!

Even in my imported files folder a lot of files got deleted! Mostly AVI (DIVX/XVid) files

I submitted a bug report, but does anybody have an idea?

Oliver

[nyplayer]

You can add me to the list of deleted recordings. I had recordings deleted from Series that I had not watched at all lost half of my Jerichos and Eureka that I had not even watched yet.

I noticed my space left was increasing then I looked and I had over 90 GIG free and and recordings were gone. Even lost some of my imported media.. there definitely is a problem here. I am really dreading the next time this is going to happen. I no longer feel 100% comfortable with SageTV maintaining the space.

[BobPhoenix]

Wild guess:

Try with a new wiz.bin so that all recordings are imported. Does it still want to delete your recordings imported or otherwise? If not then it could be a corruption in wiz.bin. Obviously keep a backup copy so you can go back.

But like I said - this is a wild guess.

[nyplayer]

I always backup my wiz.bin but it is of no use if the recordings are gone. I thought that under no circumstances would Sage delete imported media. All my files that were gone were also AVI files.

???????
What would happen If i have an 80 gig drive with say 70 gig left and I then tell SageTV to leave 40 gig free will it delete to accommodate the 40 Gig free right away or will it delete the watched etc.....??? And stop when all watched are deleted?

[BobPhoenix]

Originally Posted by nyplayer

I always backup my wiz.bin but it is of no use if the recordings are gone. I thought that under no circumstances would Sage delete imported media. All my files that were gone were also AVI files.

I agree that deleting imported files should be considered a bug. Sage should just skip them not delete them. My suggestion was only because owilsky said he was able to get some back. Obviously if you can't get them back it would be of no use.

[Opus4]

Enable debug logging and contact SageTV about it.

A couple things that I know of that messes with recordings are 1) having some sort of undelete progam active (such as Norton Utilities) & then SageTV keeps trying to delete more files because the 'deleted' files still take up space, so it deletes more to make more room for new recordings or 2) it has an error getting the file size, so it thinks the file is size 0 and might as well be deleted. #1 should not affect imported videos, but I'm not sure if #2 would.

For #1, stop using the undelete utility on the drive where SageTV recordings are stored, or at least exempt the recording dir or the video file types.

For #2: check to make sure SgaeTV has permission to access the recoridng dir. If it is a UNC network path & you are using service mode, make sure you have reconfigured the user + password used by the service. That gets cleared during an upgrade install.

- Andy

[owilsky]

Originally Posted by Opus4

Enable debug logging and contact SageTV about it.

I did.

Originally Posted by Opus4

[post=149076]
For #1, stop using the undelete utility on the drive where SageTV recordings are stored, or at least exempt the recording dir or the video file types.

I don't have anything like that. My setup is the same since SageTV V2. Only Service Packs, patches from MS and newer Java VMs (XPSP2).

Originally Posted by Opus4

[post=149076]
For #2: check to make sure SgaeTV has permission to access the recoridng dir. If it is a UNC network path & you are using service mode, make sure you have reconfigured the user + password used by the service. That gets cleared during an upgrade install.

SageTV Service is running as LocalService, Client has Admin rights. No permission problems there. No UNC usage. My Videos are in D:\Videos and my Imports in D:\Imports (NTFS drive). chkdsk says that there are no problems.

After I saw that recordings dissappeared, I stopped Sage and all processes writing to partition D:. I started R-Studio and undeleted what was still intact to an external USB Drive. I copied those files back to D:\Videos, turned on logging and restarted sage. I has >150 GB free and after some hours again shows have been deleted. The Shows were perfectly ok before and could be played by WMP.

I suggest everybody contact support about it.

I guess the main problem - at least for me - is that SageTV thinks that there are files with a size of 0 byte. I understand that SageTV deletes those orphaned files. But my files are not 0 bytes.....

Oliver

[nyplayer]

I filed a bug report but this really has me pissed now the Wife went to play an Archived version of You've got mail and poof it was not there.

What I don't get is that Sage is not supposed to delete Imported files or archived files and it is doing just that.

[nyplayer]

I had not even watched it and it got deleted.

Wed 5/16 0:10:09.021 MediaFile failing verify testFile=\\janedserver\sageuncm\Jake20-TrainingDay-3259005-0.mpg isFile=true len=0 isTV=true isRecording=false

Wed 5/16 0:10:09.021 Removing MediaFile because it failed verification:MediaFile[id=3308958 A[3259005,3258971,"Jake 2.0",24533@0513.23:00,60] host=JANEDSERVER encodedBy=Hauppauge WinTV PVR2 USB2 Encoder DVD Long Play format=MPEG2-PS 0:00:00 0 kbps [#0 Video[MPEG2-Video 29.97 fps 720x480 4:3 4800 kbps interlaced]#1 Audio[MP2 48000 Hz 2 channels 384 kbps]] \\janedserver\sageuncm\Jake20-TrainingDay-3259005-0.mpg, Seg0[Sun 5/13 23:00:00.049-Mon 5/14 0:00:00.002]]

Wed 5/16 0:10:54.422 RecordingXmlGenerator: Deleting: \\janedserver\sageuncm\Jake20-TrainingDay-3259005-0.mpg.xml


Wed 5/16 0:10:09.050 MediaFile failing verify testFile=\\janedserver\sageuncn\Heroes-Landslide-3261676-0.mpg isFile=true len=0 isTV=true isRecording=false
Wed 5/16 0:10:09.050 Removing MediaFile because it failed verification:MediaFile[id=3317683 A[3261676,3261665,"Heroes",10483@0514.21:00,61] host=JANEDSERVER encodedBy=Hauppauge WinTV PVR2 USB2 Encoder DVD Long Play format=MPEG2-PS 0:00:00 0 kbps [#0 Video[MPEG2-Video 29.97 fps 720x480 4:3 4800 kbps interlaced]#1 Audio[MP2 48000 Hz 2 channels 384 kbps]] \\janedserver\sageuncn\Heroes-Landslide-3261676-0.mpg, Seg0[Mon 5/14 21:00:00.175-Mon 5/14 22:01:00.004]]
Wed 5/16 0:10:09.055 Deleted media segment file \\janedserver\sageuncn\Heroes-Landslide-3261676-0.mpg dtime=4 len=0

Wed 5/16 3:15:44.271 MediaFile failing verify testFile=\\janedserver\sageuncm\DeadliestCatch-NewBeginnings-3268971-0.mpg isFile=true len=0 isTV=true isRecording=false
Wed 5/16 3:15:44.272 Removing MediaFile because it failed verification:MediaFile[id=3323740 A[3268971,3268950,"Deadliest Catch",12500@0516.01:00,60] host=JANEDSERVER encodedBy=Hauppauge WinTV PVR2 USB2 Encoder DVD Long Play format=MPEG2-PS 0:00:00 0 kbps [#0 Video[MPEG2-Video 29.97 fps 720x480 4:3 4800 kbps interlaced]#1 Audio[MP2 48000 Hz 2 channels 384 kbps]] \\janedserver\sageuncm\DeadliestCatch-NewBeginnings-3268971-0.mpg, Seg0[Wed 5/16 1:00:00.066-Wed 5/16 2:00:00.003]]
Wed 5/16 3:15:44.277 Deleted media segment file \\janedserver\sageuncm\DeadliestCatch-NewBeginnings-3268971-0.mpg dtime=5 len=0

[owilsky]

Yes, I see this also. I recorded soccer this weekend and wanted to watch it yesterday..... nope!

[owilsky]

Maybe we can help each other.
Please list all your plugins and extensions. Maybe we will find some similarities.
SageTV also deletes some of my imported videos, support says it shouldn't do that at all...

My hdd is perfectly ok, chkdsk and SMART does not give me any errors.

My setup:

Background Tools for comskip:
Dirmon2 0.6.3.46
ShowAnalyzer - 0.8.11.115 registered

EPG Source:
LM Gestion's STV XMLTV Importer 2.6

STVis: (I simply post my plugins.cfg)
dynamic_menu_import.stvi
comskip_playback.stvi
recpadding_import.stvi
autoaspect_import.stvi
MPTimeJumpV0.6.stvi
miniguide_plugin_0_96.stvi
imdb_search_import.stvi
RW_Metadata.stvi
favorite_excluder_2_5.stvi
SageTV_DVD_Burning.stvi
SageTV_otherPrograms_v1_2_2.stvi
VideoEditV0_2_5.stvi
Auto_Compress.stvi
webradio_import_1_1.stvi

runnable classes (see sage.properties)
load_at_startup_runnable_classes=
net.sf.sageplugins.webserver.StartServer;
net.sf.sageplugins.sagexmlinfo.RecordingXmlGenerator

[owilsky]

BTW: Do any of you possibly have Acronis True Image installed? Just a thought... I have read, that the internal Acronis service can hide parts of a directory and if SageTV does not see the files.....

Or any other tool that has a driver that digs deep into the system? I recently installed AVG Anti-Rootkit Free and AVG Anti-Spyware, I will check that....

I also have Avast Anti Virus Personal Edition, but I have it installed for months now without a problem.