![]() |
|
SageTV v7 Customizations This forums is for discussing and sharing user-created modifications for the SageTV version 7 application created by using the SageTV Studio or through the use of external plugins. Use this forum to discuss plugins for SageTV version 7 and newer. |
![]() |
|
Thread Tools | Search this Thread | Display Modes |
#21
|
|||
|
|||
Quote:
![]() |
#22
|
|||
|
|||
ALL files missing
Not sure what the root cause of this was, but I will tell you what I did and what I found afterwards. I am running SageTV 7.0.11 with multiple plugins installed, among them Jetty, BMT, and SAD. From the BMT web interface this morning I removed a few folders from the video sources in the BMT web interface. When I came home tonight, all the video files in the folders and the artifact files had been deleted. The folders are still intact, but something triggered the deletion of the video files themselves. The sad.log file has a record of all the file deletions along with the artifact file deletions.
I don't know if there is an explanation for the behavior or if it is a bug in one of the 2 plugins. If no one here has any ideas, I'll inquire in the BMT area. This is an excerpt from sad.log Code:
2010-06-17 07:21:36,541 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Serenity (2005)\Serenity (2005).avi 2010-06-17 07:21:36,555 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Serenity (2005)\Serenity (2005).avi.properties 2010-06-17 07:21:36,570 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Serenity (2005)\Serenity (2005).dvdid.xml 2010-06-17 07:21:36,586 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Serenity (2005)\Serenity (2005).jpg 2010-06-17 07:21:36,603 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Serenity (2005)\Serenity (2005).nfo 2010-06-17 07:21:36,618 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Serenity (2005)\Serenity (2005).tbn 2010-06-17 07:21:36,640 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\The Day the Earth Stood Still (1951)\The Day the Earth Stood Still (1951).avi 2010-06-17 07:21:36,654 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\The Day the Earth Stood Still (1951)\The Day the Earth Stood Still (1951).avi.properties 2010-06-17 07:21:36,670 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\The Day the Earth Stood Still (1951)\The Day the Earth Stood Still (1951).dvdid.xml 2010-06-17 07:21:36,694 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\The Day the Earth Stood Still (1951)\The Day the Earth Stood Still (1951).jpg 2010-06-17 07:21:36,717 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\The Day the Earth Stood Still (1951)\The Day the Earth Stood Still (1951).nfo 2010-06-17 07:21:36,731 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\The Day the Earth Stood Still (1951)\The Day the Earth Stood Still (1951).tbn 2010-06-17 07:21:36,751 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\TVShows\Doctor Who\Doctor Who - 22x2 - Attack of the Cybermen (2).avi 2010-06-17 07:21:36,769 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\TVShows\Doctor Who\Doctor Who - 22x2 - Attack of the Cybermen (2).avi.properties 2010-06-17 07:21:36,787 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Sky Captain and the World of Tomorrow (2004)\Sky Captain and the World of Tomorrow (2004).avi 2010-06-17 07:21:36,806 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Sky Captain and the World of Tomorrow (2004)\Sky Captain and the World of Tomorrow (2004).avi.properties 2010-06-17 07:21:36,820 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Sky Captain and the World of Tomorrow (2004)\Sky Captain and the World of Tomorrow (2004).dvdid.xml 2010-06-17 07:21:36,835 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Sky Captain and the World of Tomorrow (2004)\Sky Captain and the World of Tomorrow (2004).jpg 2010-06-17 07:21:36,848 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Sky Captain and the World of Tomorrow (2004)\Sky Captain and the World of Tomorrow (2004).nfo 2010-06-17 07:21:36,861 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Sky Captain and the World of Tomorrow (2004)\Sky Captain and the World of Tomorrow (2004).tbn 2010-06-17 07:21:36,895 WARN [FileDeleteTask]: File deleted: \\rangerone\VideoI\TVSeries\Monty Python's Flying Circus\Monty Python's Flying Circus - 3x6 - The War Against Pornography.jpg 2010-06-17 07:21:36,911 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Another Thin Man (1939)\Another Thin Man (1939).avi 2010-06-17 07:21:36,925 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Another Thin Man (1939)\Another Thin Man (1939).avi.properties 2010-06-17 07:21:36,938 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Another Thin Man (1939)\Another Thin Man (1939).dvdid.xml 2010-06-17 07:21:36,955 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Another Thin Man (1939)\Another Thin Man (1939).jpg 2010-06-17 07:21:36,971 WARN [FileDeleteTask]: File deleted: \\rangerone\Media\Movies\Another Thin Man (1939)\Another Thin Man (1939).nfo |
#23
|
|||
|
|||
It would have been better had you had debug level on for the SAD logging, but based on that log output, something was deleted and SAD found all of those artifacts for whatever was deleted. They all look like valid artifacts except for the avi files, I would assume? However, for some reason Sage told SAD that the avi file was NOT a registered SageTV object. SAD only deletes files if they aren't registered as a SageTV object (and to be extra safe, I do that check twice before deleting the file).
Code:
public void run() { for(File f : files) { // files is a collection of discovered artifacts if(fake) LOG.warn("File skipped because SAD is in pretend mode: " + f); // This is the second check with the API before deleting else if(API.apiLocalUI.mediaFileAPI.GetMediaFileForFilePath(f) == null) { boolean result; if(rmNonEmptyDirs) result = FileUtils.deleteQuietly(f); else result = f.delete(); if(result) LOG.warn("File deleted: " + f); else LOG.error("Delete FAILED: " + f); } else LOG.warn("File skipped because it is a registered SageTV media object: " + f); } }
__________________
Twitter: @ddb_db Server: Intel i5-4570 Quad Core, 16GB RAM, 1 x 128GB OS SSD (Win7 Pro x64 SP1), 1 x 2TB media drive Capture: 2 x Colossus STB Controller: 1 x USB-UIRT Software:Java 1.7.0_71; SageTV 7.1.9 Clients: 1 x HD300, 2 x HD200, 1 x SageClient, 1 x PlaceShifter Plugins: Too many to list now... |
#24
|
|||
|
|||
Thanks for taking a look. I am not sure what triggered this off either. If I find something that makes sense, I'll post back, but as you say, the log doesn't have enough detail to tell us why.
|
#25
|
|||
|
|||
Quote:
__________________
Channels DVR UBUNTU Server 2 Primes 3 Connects TVE SageTV Docker with input from Channels DVR XMLTV and M3U VIA Opendct. |
#26
|
|||
|
|||
Ok, before you do any more testing, let me add some more logging output to SAD and put a new build in the repository. Basically, I want to log more details of the event that's triggering the delete. Will try to do it today, but may not have time. You may have to nudge me on the weekend if I forget.
__________________
Twitter: @ddb_db Server: Intel i5-4570 Quad Core, 16GB RAM, 1 x 128GB OS SSD (Win7 Pro x64 SP1), 1 x 2TB media drive Capture: 2 x Colossus STB Controller: 1 x USB-UIRT Software:Java 1.7.0_71; SageTV 7.1.9 Clients: 1 x HD300, 2 x HD200, 1 x SageClient, 1 x PlaceShifter Plugins: Too many to list now... |
#27
|
|||
|
|||
Slugger, I just installed SAD based on your mention in the SJQ thread that the SJQ File Cleaner will eventually be replaced with SAD. After installing and configuring, I picked a few items to test with the artifact finder to see how it works. I picked 5 mpg files where 4 were recordings and 1 was an imported video. The SAD log (below) gives me an error for all items telling me they are not registered SageTV media objects. All five items show up in SageTV, with the 4 recordings showing up in the Recordings area and the 1 imported video showing up in the Videos area. I am assuming the mpg files are what you should be picking as a media object, is that correct? I am also assuming that if SAD says all of my items are not registered SageTV media objects, that it will not delete the artifacts, is that correct? Any thoughts as to why my Recorded TV items are not registered SageTV media objects?
Code:
2010-06-20 12:34:48,453 ERROR [Plugin]: Selected file is not a registered SageTV media object! [D:\shares\SageTV\SageTV Recordings\AngelinaBallerina-863707-0.mpg] 2010-06-20 12:37:20,421 ERROR [Plugin]: Selected file is not a registered SageTV media object! [D:\shares\SageTV\SageTV Recordings\TopGear-1258684-0.mpg] 2010-06-20 12:47:01,906 ERROR [Plugin]: Selected file is not a registered SageTV media object! [D:\shares\SageTV\SageTV Recordings\AngelinaBallerina-830251-0.mpg] 2010-06-20 12:50:11,359 ERROR [Plugin]: Selected file is not a registered SageTV media object! [D:\shares\SageTV\Movies and Videos\Movies Recorded On TV\Armageddon.mpg] 2010-06-20 12:51:09,562 ERROR [Plugin]: Selected file is not a registered SageTV media object! [D:\shares\SageTV\SageTV Recordings\Up-2547602-0.mpg]
__________________
Server: OS: MS WHS v1; MB: GIGABYTE GA-EP45-UD3R; CPU: Intel Q9400 2.66GHz; RAM: G.SKILL 4GB (4x1GB); HDs: 1x80GB, 7x1.5TB; Graphics: EVGA GeForce 9500 GT 512MB Capture/Content: HD-PVR, 2xHDHRP (CC), Comcast STBs (Controllers): RNG110 (Firewire ChCh) Clients: 2xHD300, 2xHD200, 2xPS, Client SageTV/Plugins: v7.1.9, SageDCT, SRE, Comm Det, Jetty, Web Server, Mob Web, PlayOn, E/D Fav, Fav Ex, MF Stop, Enc Names |
#28
|
|||
|
|||
This appears to be a bug with the Sage API calls. You can test this by launching studio and going to the expression evaluator and running the following:
GetMediaFileForFilePath("D:\\shares\\SageTV\\SageTV Recordings\\AngelinaBallerina-863707-0.mpg") Simply use each file path printed in your sad log output. If studio returns null then Sage believes the files aren't media objects (at least via that API call). And if that's the case then that's a bug with the core API. If studio doesn't return null then Greg's wrappers are inadvertently returning null and if that's the case then I'll need to talk with him. The problem in SAD is that the API call is returning null so SAD then believes the file isn't a media object. I'm not able to reproduce that here, but that's basically what's going on. If studio does not return null with the above expression then we need to know which version of SAD and all its listed dependencies you're running in order to be able and track this down.
__________________
Twitter: @ddb_db Server: Intel i5-4570 Quad Core, 16GB RAM, 1 x 128GB OS SSD (Win7 Pro x64 SP1), 1 x 2TB media drive Capture: 2 x Colossus STB Controller: 1 x USB-UIRT Software:Java 1.7.0_71; SageTV 7.1.9 Clients: 1 x HD300, 2 x HD200, 1 x SageClient, 1 x PlaceShifter Plugins: Too many to list now... Last edited by Slugger; 06-21-2010 at 07:31 AM. Reason: Escape backslashes |
#29
|
|||
|
|||
Quote:
I can definitely see the act of removing video source folders unregistering those video objects from the db... Could SAD be getting confusing and thinking that those artifacts should be deleted? Did your testing include removing an import directory and see what SAD did?
__________________
Server 2003 r2 32bit, SageTV9 (finally!) 2x Dual HDHR (OTA), 1x HD-PVR (Comcast), 1x HDHR-3CC via SageDCT (Comcast) 2x HD300, 1x SageClient (Win10 Test/Development) Check out TVExplorer |
#30
|
|||
|
|||
Quote:
__________________
Twitter: @ddb_db Server: Intel i5-4570 Quad Core, 16GB RAM, 1 x 128GB OS SSD (Win7 Pro x64 SP1), 1 x 2TB media drive Capture: 2 x Colossus STB Controller: 1 x USB-UIRT Software:Java 1.7.0_71; SageTV 7.1.9 Clients: 1 x HD300, 2 x HD200, 1 x SageClient, 1 x PlaceShifter Plugins: Too many to list now... |
#31
|
|||
|
|||
Quote:
I tried all five of them in the expression evaluator in studio and they all returned null. So, from your statement above, it looks like SageTV believes all of my recorded TV and imported videos are not media objects. ![]() ![]() Thanks.
__________________
Server: OS: MS WHS v1; MB: GIGABYTE GA-EP45-UD3R; CPU: Intel Q9400 2.66GHz; RAM: G.SKILL 4GB (4x1GB); HDs: 1x80GB, 7x1.5TB; Graphics: EVGA GeForce 9500 GT 512MB Capture/Content: HD-PVR, 2xHDHRP (CC), Comcast STBs (Controllers): RNG110 (Firewire ChCh) Clients: 2xHD300, 2xHD200, 2xPS, Client SageTV/Plugins: v7.1.9, SageDCT, SRE, Comm Det, Jetty, Web Server, Mob Web, PlayOn, E/D Fav, Fav Ex, MF Stop, Enc Names |
#32
|
|||
|
|||
Quote:
This is one of those cases where I really wish it was a SAD bug that was causing this, but I'm just not seeing it. SAD listens for MediaFileRemoved events, searches for artifacts of the media file it's told was deleted, filters out any artifacts that are registered media objects, schedules the remaining files to be deleted, then the files are deleted only after asking the API one more time if it's not a registered media object. If someone sees a bug in my code, I'd love to hear about it (and fix it). EDIT: Right, you were just testing with those mpgs and SAD told you they weren't registered. You went to studio and ran the expression evaluator and it returned null for each of those API calls (you remembered to double up on the backslashes?). Assuming that's all true, then, yes, Sage believes those files aren't registered media objects and an issue should be opened with Sage.
__________________
Twitter: @ddb_db Server: Intel i5-4570 Quad Core, 16GB RAM, 1 x 128GB OS SSD (Win7 Pro x64 SP1), 1 x 2TB media drive Capture: 2 x Colossus STB Controller: 1 x USB-UIRT Software:Java 1.7.0_71; SageTV 7.1.9 Clients: 1 x HD300, 2 x HD200, 1 x SageClient, 1 x PlaceShifter Plugins: Too many to list now... Last edited by Slugger; 06-22-2010 at 07:47 AM. Reason: Reread original post |
#33
|
|||
|
|||
Quote:
__________________
Twitter: @ddb_db Server: Intel i5-4570 Quad Core, 16GB RAM, 1 x 128GB OS SSD (Win7 Pro x64 SP1), 1 x 2TB media drive Capture: 2 x Colossus STB Controller: 1 x USB-UIRT Software:Java 1.7.0_71; SageTV 7.1.9 Clients: 1 x HD300, 2 x HD200, 1 x SageClient, 1 x PlaceShifter Plugins: Too many to list now... |
#34
|
|||
|
|||
I would agree with that statement. If the file exists, then the artifacts probably shouldn't be deleted either. At least I cannot think of a situation where you would want to delete those files.
__________________
i7-6700 server with about 10tb of space currently SageTV v9 (64bit) Ceton InfiniTV ETH 6 cable card tuner (Spectrum cable) OpenDCT HD-300 HD Extenders (hooked to my whole-house A/V system for synched playback on multiple TVs - great during a Superbowl party) Amazon Firestick 4k and Nvidia Shield using the MiniClient Using CQC to control it all |
#35
|
|||
|
|||
The weirdness has been solved, I believe. Turns out core 7.0.11 added some new, undocumented features, which would explain all the behaviour described here.
The fix will be to only delete artifacts if the media file object SAD is being told about was actually deleted (and not just unregistered from wiz.bin), otherwise the event will be ignored and nothing will be deleted. Fix might be a day or two if I don't have time before I play baseball tonight. I'm on the road to Toronto for work tomorrow, Thursday is busy, and I'm in Ottawa on Friday so if I can't find the time before 6pm tonight then it might be the weekend before it's fixed, but at least we now know the issue and the fix. Issue 134 has been opened for this. If you want to be notified as soon as the fix is checked in then you'll want to star this issue at the project site.
__________________
Twitter: @ddb_db Server: Intel i5-4570 Quad Core, 16GB RAM, 1 x 128GB OS SSD (Win7 Pro x64 SP1), 1 x 2TB media drive Capture: 2 x Colossus STB Controller: 1 x USB-UIRT Software:Java 1.7.0_71; SageTV 7.1.9 Clients: 1 x HD300, 2 x HD200, 1 x SageClient, 1 x PlaceShifter Plugins: Too many to list now... |
#36
|
|||
|
|||
I love it when fixes are easier than you expect. Just a few lines (5 to be exact) needed to clean up this issue. A new version of SAD is now in the repository with this fix.
__________________
Twitter: @ddb_db Server: Intel i5-4570 Quad Core, 16GB RAM, 1 x 128GB OS SSD (Win7 Pro x64 SP1), 1 x 2TB media drive Capture: 2 x Colossus STB Controller: 1 x USB-UIRT Software:Java 1.7.0_71; SageTV 7.1.9 Clients: 1 x HD300, 2 x HD200, 1 x SageClient, 1 x PlaceShifter Plugins: Too many to list now... |
#37
|
|||
|
|||
I haven't tried installing this but am following along as I think it is a great plugin (just like all of Sluggers work).
Would it be possible to move the files that are being triggered for delete to a common folder where you can confirm deletion? Kind of a "Recycle Bin", it would add an extra step, but I know I would feel a lot better about using the plugin if I new I could recover from an accidental delete.
__________________
Sage Server: i5-2500K 8 GB DDR, 6000gb HDD, 4xHD-PVR < 4xBell 6141, Win7 x64 Client 1: HD-200, Panasonic 42PX75 Client 2: HD-300, Samsung LCD Client 3: HD-300, Samsung PN50C550 Client 4: MS Surface Pro |
#38
|
|||
|
|||
Quote:
|
#39
|
|||
|
|||
Someone will have to report it to Sage because I am not able to reproduce this bug.
__________________
Twitter: @ddb_db Server: Intel i5-4570 Quad Core, 16GB RAM, 1 x 128GB OS SSD (Win7 Pro x64 SP1), 1 x 2TB media drive Capture: 2 x Colossus STB Controller: 1 x USB-UIRT Software:Java 1.7.0_71; SageTV 7.1.9 Clients: 1 x HD300, 2 x HD200, 1 x SageClient, 1 x PlaceShifter Plugins: Too many to list now... |
#40
|
|||
|
|||
Quote:
__________________
Twitter: @ddb_db Server: Intel i5-4570 Quad Core, 16GB RAM, 1 x 128GB OS SSD (Win7 Pro x64 SP1), 1 x 2TB media drive Capture: 2 x Colossus STB Controller: 1 x USB-UIRT Software:Java 1.7.0_71; SageTV 7.1.9 Clients: 1 x HD300, 2 x HD200, 1 x SageClient, 1 x PlaceShifter Plugins: Too many to list now... |
![]() |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
|
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
VRM9 Artifact Problem/FF/RW Freezes Video | OldPCGUY | SageTV Software | 6 | 10-15-2008 08:26 PM |
Am I the only sad one? | grooves12 | SageMC Custom Interface | 30 | 07-10-2008 04:19 PM |
How to delete multiple recordings or set it to auto delete? | batigol9 | SageTV Software | 24 | 11-10-2006 07:57 PM |
undesired artifact present on PBS' HD shows | stevech | SageTV Software | 2 | 04-13-2006 08:06 PM |
Any TimeWarner Cable user here have you seen this odd artifact | SHS | Hardware Support | 4 | 04-22-2005 02:57 PM |