Plugin: Sage Artifact Delete (SAD)

[Slugger]

Just added this plugin to the repository. It's a simple plugin that listens for MediaFileRemoved events and deletes any artifacts (edl, txt, log, etc.) it can find for the media file just deleted. The current version has the following features:

• Deletion of artifacts is delayed (30 seconds by default, can be configured); this allows other processes to move/copy the artifacts before SAD deletes them
• SAD will never delete a file if it is registered as a SageTV media file object
• Algorithm is simple: If SAD is told that myshow-0.mpg was deleted then SAD looks for files named myshow-0.* in the same directory and will delete any that it finds
• All operations performed are logged to SageTV\plugins\sad\sad.log

Feedback, etc. welcomed in this thread.

[GKusnick]

Originally Posted by Slugger

Algorithm is simple: If SAD is told that myshow-0.mpg was deleted then SAD looks for files named myshow-0.* in the same directory and will delete any that it finds

What about directories? If there's a directory named myshow-0.whatever, can/should that be deleted as well?

Also I foresee issues with converted files in which people might want to delete the original but keep the conversion (even if it's not a MediaFile).

So my suggestion would be to have a configurable list of extensions that SAD is allowed to delete (be they files or directories). The default could be *, but users could make it more restrictive if they like.

[doc]

The old file cleaner didn't work with xml files generated with Nielm's XML generator.

This one seems to work great, I can now remove a script that I run every day.

Cheers

[Slugger]

A new beta will be in the repository shortly. Lots of new features added:

* Add ability to restrict deleted artifacts by file extension
* Add ability to ignore artifacts by file extension
* Add ability to delete artifact directories with extension filter and ignore lists
* Add artifact finder (mainly for testing/debugging purposes, but may be helpful to others)

[Slugger]

The new beta build promised earlier has been delayed as I investigate a possible bug in the core. Though SAD can handle the multiple copies of the event, I'd rather wait and get confirmation of the bug before releasing anything else.

[stuckless]

That's cool... any thoughts on adding in fanart removal as well? I have it planned, but it's not on my radar... but at some point it has to be done. Thing is, i'll probably have to duplicate your efforts unless you add support for it

[PLUCKYHD]

Originally Posted by stuckless

That's cool... any thoughts on adding in fanart removal as well? I have it planned, but it's not on my radar... but at some point it has to be done. Thing is, i'll probably have to duplicate your efforts unless you add support for it

Sean if you wanted me to I could port my old advanced fanart cleanup tool from sagemc. It seemed to work pretty well and would be pretty easy to automate with the same listener this is using. Of course I would need to check that no other shows exist with the title and that it is not a favorite (no need to delete if it is just going to be recorded again.)

[Slugger]

Originally Posted by stuckless

That's cool... any thoughts on adding in fanart removal as well? I have it planned, but it's not on my radar... but at some point it has to be done. Thing is, i'll probably have to duplicate your efforts unless you add support for it

If someone wants to describe the algorithm for finding fanart dirs then I don't think it'd be much effort to add support for it to SAD. My only requirement is that it doesn't add a dependency on another (standard) plugin. I don't mind adding a dependency on a library plugin.

Originally Posted by PLUCKYHD

Sean if you wanted me to I could port my old advanced fanart cleanup tool from sagemc. It seemed to work pretty well and would be pretty easy to automate with the same listener this is using. Of course I would need to check that no other shows exist with the title and that it is not a favorite (no need to delete if it is just going to be recorded again.)

Even better! Plucky, if you wanted to write a patch for SAD to add fanart support then I'd be more than willing to add it in. Might be a better alternative than to add another plugin that basically does the same kind of thing. Code is in svn and is available for checkout at:

http://sagetv-addons.googlecode.com/svn/trunk/sad/

[PLUCKYHD]

Originally Posted by Slugger

Even better! Plucky, if you wanted to write a patch for SAD to add fanart support then I'd be more than willing to add it in. Might be a better alternative than to add another plugin that basically does the same kind of thing. Code is in svn and is available for checkout at:

http://sagetv-addons.googlecode.com/svn/trunk/sad/

Okay I will look at doing that since I already pretty much have the algorithm done. It will rely on phoenix api (library plugin). Let me dig up that old code and see what I can come up with for you and add to your code.

I am guessing this fires on movies and tv correct?

[Slugger]

Originally Posted by PLUCKYHD

I am guessing this fires on movies and tv correct?

Haven't actually tested this, but I would assume so. It should fire when any media file is deleted (tv, imported video, music, pic, etc.). Be aware that there appears to be a bug where the core fires this event twice. I've already submitted a bug report and am awaiting confirmation. Assuming the bug is in the core then I'll release the latest version to the repository later today. I say this because the version of the plugin in the repository right now is not even close (feature-wise) to what's in svn.

[PLUCKYHD]

Originally Posted by Slugger

Haven't actually tested this, but I would assume so. It should fire when any media file is deleted (tv, imported video, music, pic, etc.). Be aware that there appears to be a bug where the core fires this event twice. I've already submitted a bug report and am awaiting confirmation. Assuming the bug is in the core then I'll release the latest version to the repository later today. I say this because the version of the plugin in the repository right now is not even close (feature-wise) to what's in svn.

Okay just a fair warning you code better than I do So you won't hurt my feelings if you cleanup what I submit.



edit: Another caveat is I use Sagex and you use greg's api. I have never coded with his api but if you are against me adding a sagex dependency I suppose I can figure it out quick enough.

[gibsonpa]

Originally Posted by Slugger

A new beta will be in the repository shortly. Lots of new features added:

* Add ability to restrict deleted artifacts by file extension
* Add ability to ignore artifacts by file extension
* Add ability to delete artifact directories with extension filter and ignore lists
* Add artifact finder (mainly for testing/debugging purposes, but may be helpful to others)

This sounds really nice. Can't wait to try it out....

[Slugger]

Originally Posted by PLUCKYHD

edit: Another caveat is I use Sagex and you use greg's api. I have never coded with his api but if you are against me adding a sagex dependency I suppose I can figure it out quick enough.

Not against it, per se. Depends how you're going to do things. If you're going to isolate your code into a separate class then feel free to use sagex. If you're going to add directly to my code then I'd rather you use Greg's API (b/c it gets too confusing to read obj access through two different APIs in the same class file).

[PLUCKYHD]

Originally Posted by Slugger

Not against it, per se. Depends how you're going to do things. If you're going to isolate your code into a separate class then feel free to use sagex. If you're going to add directly to my code then I'd rather you use Greg's API (b/c it gets too confusing to read obj access through two different APIs in the same class file).

Okay I will probably make my own class and just add into your collection of files to be deleted as well as a couple of config options. should be pretty straight foward...should being the key word

edit: apache delete quietly very very nice never knew that existed would have saved me allot of code in the past of looping through files then deleting the directory. Plus if the setting is write I can just call the main fanart folder know for the known title and it will delete all directories. Wow learn something new everyday.

[Slugger]

I've confirmed the bug where the media file event is fired twice is indeed a bug with the core/default STV. Therefore I'm confident releasing the next version of this plugin. Plucky is working on adding fanart support and says he'll have something ready soon so I figure I'll wait for that patch before releasing the updated version. Should appear in the repository sometime today.

[GKusnick]

Originally Posted by Slugger

Algorithm is simple: If SAD is told that myshow-0.mpg was deleted then SAD looks for files named myshow-0.* in the same directory and will delete any that it finds

Say I have both a foo.mpg and a foo.jpg in the same folder, and they both have artifacts (e.g. foo.mpg.properties and foo.jpg.properties). Will SAD be able to tell them apart, and not delete the foo.mpg.* artifacts when foo.jpg goes away? Or must I set up inclusion/exclusion rules for the cross-product of media type and artifact type?

[Slugger]

Originally Posted by GKusnick

Say I have both a foo.mpg and a foo.jpg in the same folder, and they both have artifacts (e.g. foo.mpg.properties and foo.jpg.properties). Will SAD be able to tell them apart, and not delete the foo.mpg.* artifacts when foo.jpg goes away? Or must I set up inclusion/exclusion rules for the cross-product of media type and artifact type?

Currently, no. I thought about this for awhile the other night. What constitutes an artifact? If the object is named 'foo.mpg' then should artifacts be foo.mpg.* or foo.*? Ideally, it'd be the former, but comskip/showanalyzer/comskip playback stvi all expect and use foo.edl so my hands are kind of tied and therefore an artifact for foo.mpg is anything matching foo.*

I could recognize that there are two media objects starting with foo. in the current dir, but that's all I can do. Reasonably, I suppose, I could assume that foo.jpg.* actually belongs to foo.jpg, but that's only a (logical) guess. There's no reason it couldn't be a generated jpg artifact from the .mpg object. There's just no end to the possibilities. To keep my sanity, the rule (with this plugin) is an object named foo.mpg has artifacts named foo.*. For tv/video I don't think this will be a problem. I suppose it could be a problem for import directories if one were to use the same dir for videos and pictures and then had a video named vacation.mpg and a pic named vacation.jpg.

The best solution may be to optionally restrict the types of media this will delete artifacts for (basically introduce a media mask to prevent it from deleting artifacts for music, pics, etc.)?

[Slugger]

v0.0.2 (beta) added to the repository. All the changes listed above are included except for fanart removal support. It's coming, but will be another day or two before it's ready.

NOTE: There is a known issue where deleting media objects in certain ways can cause the MediaFileRemoved event to fire twice. In this case SAD will queue the artifacts for deletion twice. Not a big deal as the first time it will delete them and the second time it will simply fail (because they're already deleted). Presumably, a future SageTV build will fix this issue.

Enjoy.

[KJake]

I think it would be a good idea to pre-configure everything for the user. If I'm installing this via my TV, there's no way that I want to put in all the extensions that should be cleaned up via the remote.

[Slugger]

Originally Posted by KJake

I think it would be a good idea to pre-configure everything for the user. If I'm installing this via my TV, there's no way that I want to put in all the extensions that should be cleaned up via the remote.

It is preconfigured. By default it'll delete every extension (blank means delete them all).