Plugin: CommercialDetector

[tmiranda]

Originally Posted by Sambo555

Upshot is that the .ts file was deleted successfully but the comskip files are still there. Maybe the log will shed more light.

Now I understand the issue and this log file helps a lot. Thanks.

Edit: Can you look into your properties file and see if you have progressive_delete set to true? I forget the exact name of the property, it's some thing like "use_progressive_delete". Try setting that to false t osee if that helps.

[tmiranda]

I just released a new UI Mod that address the issue of the CS icon in the header not always appearing. (As was noted above if no other icons were being displayed in the header the CS icon would not be displayed.)

Tom

[Sambo555]

Originally Posted by tmiranda

Can you look into your properties file and see if you have progressive_delete set to true? I forget the exact name of the property, it's some thing like "use_progressive_delete". Try setting that to false t osee if that helps.

I just looked and enable_progressive_delete is already set to false.

[tmiranda]

Originally Posted by Sambo555

I just looked and enable_progressive_delete is already set to false.

It was worth a shot. I looked at the code last night and thanks to your logs I know what is happening and I have a fix but I need to test it before I release an update.

[Tomahawk51]

Hi, I had a slightly off-topic question to ask:

It seems to me that I have a number of orphaned show.txt, *.edl, *.properties, *.logo.txt, *.log files hanging around for shows that I've already deleted (or SageTV has automatically deleted).

So my questions are:
1) I assume that in the prefs when you specify the files to delete, you don't need to type the period (ex: "edl, log, txt, properties,logo.txt" is what I show now). Correct?

2) Should I expect the plugin to clean up these files for shows that I process with Comskip Monitor before I migrated to this (very nice) plugin? Are those not going to be processed due to being missed in plugin logs or something?

3) If the answer to 2) is NO - THESE FILES WON'T GET CLEANED UP, then does anyone know a good way to "clean house"? Would I have to install comskip monitor again to do this or can I run a complex search in Vista that would help me filter to this?

thanks!

[tmiranda]

Originally Posted by Tomahawk51

Hi, I had a slightly off-topic question to ask:

It seems to me that I have a number of orphaned show.txt, *.edl, *.properties, *.logo.txt, *.log files hanging around for shows that I've already deleted (or SageTV has automatically deleted).

So my questions are:
1) I assume that in the prefs when you specify the files to delete, you don't need to type the period (ex: "edl, log, txt, properties,logo.txt" is what I show now). Correct?

2) Should I expect the plugin to clean up these files for shows that I process with Comskip Monitor before I migrated to this (very nice) plugin? Are those not going to be processed due to being missed in plugin logs or something?

3) If the answer to 2) is NO - THESE FILES WON'T GET CLEANED UP, then does anyone know a good way to "clean house"? Would I have to install comskip monitor again to do this or can I run a complex search in Vista that would help me filter to this?

thanks!

1. Correct. No "." needed.
2. Yes, it should but there is a bug in there that prevents cleanup in some cases if you have it set to start comskip immediately.
3. The next update will have the ability to cleanup these orphaned files.

[Tomahawk51]

awesome! not that the <10k files really matter...but a tidy server is a happy server.

[tmiranda]

Originally Posted by Tomahawk51

awesome! not that the <10k files really matter...but a tidy server is a happy server.

Not to mention that my plugin needs to function properly and cleanup after itself.

I've got a few other enhancements and fixes in the works. Once they are ready I'll release an new version with the fix (so no future orphans are created) and a manual cleanup procedure.

[Sambo555]

Originally Posted by tmiranda

I looked at the code last night and thanks to your logs I know what is happening and I have a fix but I need to test it before I release an update.

That's awesome! If you need anyone to test on.....

BTW, the CS icon is working good for me now. Looking forward to the new additions to the update. Thank you.

Now I just have to figure out my major problem of comskip taking hours to complete. Ha, the tweaking never ends...

[tmiranda]

Originally Posted by Sambo555

That's awesome! If you need anyone to test on.....

PM me your email address.

[tmiranda]

Here is a version for forums users that have been experiencing problems. I believe it corrects:

- toricred's issue of some files not getting processed (fingers crossed.)
- Sambo555's issue of some files not getting cleaned up.
- Tomahawk51's issue of being able to manually scan for and delete orphans (Look in advanced settings for that. Note that it will not show up on SageClients.)

It also adds Intelligent Scheduling but I have not tested that thoroughly. If you try this and it does not work the the worst thing that will happen is that files will not get processed. In that case, turn the feature off and then restart. Any files that were not processed should be picked up. The option is listed in the Advanced Options section.

I also added more debug info so if things do not work as expected please set debug log level to Trace and attach a log foe me to look at.

To install: download the attached zip file and extract to the install directory of your sage server. When you go to the plugin manager you will see an "update available" message next to the CommercialDetector General plugin. No changes were made to the UI Mod.

[bcjenkins]

Will we be able to disable intelligent scheduling? B

[tmiranda]

Originally Posted by bcjenkins

Will we be able to disable intelligent scheduling? B

Yes, it's actually disabled by default. If you don't want to use it you'll never even know it's there.

[ldavis]

I was just playing around the forced "run comskip". It appears to work but subject to time and channel restrictions. Maybe that is the behavior everyone wants but I would think this should override those restrictions. Is that something that is in the queue to be implemented or does everyone like it the way it is?

[tmiranda]

I had not thought about changing the behavior of the "Run Comskip" button, but if it's something people want I can probably make it happen in a future release.

[tmiranda]

I see a few people downloaded the "Forum Beta", any feedback? I do know it broke the "restricted times" functionality, but I've already fixed that.

toricred? Sambo555? Tomahawk51?

[ptzink]

Tmiranda, first of all I want to say thanks for the great plugin. I was a former Comskip Monitor user. I must admit when I first saw this plugin in STV 7 I discarded it because I thought it was essentially a simple alternative to CM. Then I wanted more control of which channels comskip was processed on, so I gave SJQ a try. That was a little too involved for my needs, so I finally took another look at your plugin - and I am glad I did. It does just what I want and is dead simple to setup. Plus, I get the added bonus of being able to skip commercials while the show is still recording! Awesome. Enough about that. Here is my question/issue:
I think you are aware of this issue and have either fixed it or are working on it, but I wanted to make you aware anyway. I recently noticed some issues with cleaning up .edl, .log, etc. files. It happened on a manual recording that we started to watch while the show was still recording. Of course, when you do this, Sage also records the following programs on that station until you stop playback at which time it deletes the programs that followed your recording. Com Detector failed to delete the orphaned files for these auto-deleted recordings in this case. Otherwise, file cleanup seems to be working well (especially now that I have added txt and incommercial to the cleanup list). Is there any reason that txt and incommercial are not default cleanup values when you install the plugin? Just curious. Again, thanks for the great work.

[HelenWeathers]

Let me add one more to the cleanup - and not sure if it's even possibile.

I had an ATSC station transmission error for about 3 minutes a few nights ago which resulted in a programname-#####-0.mpg and a programname-#####-1.mpg (two mpg files).

The -0 edl, txt etc files were deleted when the recording was deleted but the -1 files were not.

I discovered them the next day while browsing and deleted them so not sure if they would have been deleted eventually if I had left them.

[tmiranda]

Originally Posted by ptzink

I think you are aware of this issue and have either fixed it or are working on it, but I wanted to make you aware anyway. I recently noticed some issues with cleaning up .edl, .log, etc. files. It happened on a manual recording that we started to watch while the show was still recording. Of course, when you do this, Sage also records the following programs on that station until you stop playback at which time it deletes the programs that followed your recording. Com Detector failed to delete the orphaned files for these auto-deleted recordings in this case. Otherwise, file cleanup seems to be working well (especially now that I have added txt and incommercial to the cleanup list). Is there any reason that txt and incommercial are not default cleanup values when you install the plugin? Just curious. Again, thanks for the great work.

Yes, I am aware of the orphan issue and I have a fix for that. It's actually in the "Forum Beta" located a few posts back (caveat is that the Forum Beta breaks restricted times functionality).

txt and incommercial are now included in the default values, but if you had CommercialDetector previously installed it does not automatically add them.

[tmiranda]

Originally Posted by HelenWeathers

I had an ATSC station transmission error for about 3 minutes a few nights ago which resulted in a programname-#####-0.mpg and a programname-#####-1.mpg (two mpg files).

The -0 edl, txt etc files were deleted when the recording was deleted but the -1 files were not.

I'll look into this. If you were watching live TV while this occured it is probably a manifestation of the same issue others have mentioned.

Maybe an explanation will help clarify the issue. I believe the these orphans are being created when:
- CommercialDetector (actually comskip) is processing the file and Sage tries to delete it.
- Sage puts the .mpg in a queue of files that could not be deleted because they were locked. (Sage comes back to these files later to delete them.)
- Sage issues a plugin event saying the .mpg was deleted (even though it was not.)
- CommercialDetector tries to cleanup the files, but does not because it finds that a corresponding .mpg file still exists.

This situation happens often, especially if you have Start Immediately set to True and you are watching live TV. It can also happen in other situations because Sage tries to delete the file faster than CommercialDetector can stop comskip.