Plugin: CommercialDetector

[valnar]

Originally Posted by tmiranda

It might actually be comskip producing them during detection on live TV. I thought those files were automatically deleted when comskip found that the show being scanned was no longer "in a commercial". Are you using the default .ini that comes with CD or are you using an altered version?

Everything is default. I edited nothing. That's why I thought it was your plugin.

[gplasky]

.incommercial was added to Comskip a while ago.

Changes in 0.80.025
- Added parameter: incommercial_frames, default is 1000. When live_tv is set comskip also outputs a file with the extension ".incommercial" that contains a single line with a "0" when outside a commercial and a "1" when the current processed frame is less then incommercial_frames after the last detected commercial.

Gerry

[tmiranda]

gplasky is correct, it is CD creating those files. I had not noticed because on my systems I use a modified version of the .ini that does not set the live_tv flag.

You must manually add incommercial to the list of files to get cleaned up. I'll make it the default in the next update.

[tmiranda]

I figured I post a little about what I'm currently working on to see what people think about this idea and if you have any suggestions. I'm working on implementing "Intelligent Comskip Scheduling".

What this will do is only run comskip when nothing is recording and the job(s) will likely finish before any upcoming recordings. In order to do this it will dynamically keep track of how long recordings take to process (on a channel-by-channel basis) so if it finds a "gap" in the recording schedule it will launch just enough queued jobs so that they all finish before the next scheduled recording.

It will still honor any times that you have restricted comskip from running but if this works as I hope it should eliminate the need for using that setting.

[toricred]

I'm hoping you're going to make this optional. I can see a lot of value in it, but I for one like my detection done live. I still love this plugin after getting used to it.

I'm having one problem, but for now I have a workaround for it. When I have more recordings running than the number of threads for SAV, the extra recordings never get analyzed. For now I've set the number of threads to the same number as I have tuners so this can't be a problem. Any ideas?

[tmiranda]

Originally Posted by toricred

I'm hoping you're going to make this optional. I can see a lot of value in it, but I for one like my detection done live. I still love this plugin after getting used to it.

I'm having one problem, but for now I have a workaround for it. When I have more recordings running than the number of threads for SAV, the extra recordings never get analyzed. For now I've set the number of threads to the same number as I have tuners so this can't be a problem. Any ideas?

Intelligent Scheduling will be optional.

Regarding the issue you you are having - I don't know what you mean by "SAV". What this plugin should do if you have Maximum Concurrent Jobs set to something less than the number of recordings in progress (including live tv), is to queue them up until some of the in-process jobs complete. I'll look into it to see why that is not happening for you.

Can you attach a log with the debug level set to Trace?

Thanks,

Tom

[toricred]

Sorry, SAV=ShowAnalyzer. I'll work on getting a log for you tomorrow night when I next have a bunch of recordings at the same time.

[tmiranda]

I completely bypass ShowAnalyzer's threading mechanism which is why you need to point my plugin at ShowAnalyzerEngine.exe. ShowAnalyzer's threading settings will not effect my plugin at all.

[toricred]

I understand that and I'm talking about the number of threads setting in your plugin. I am calling ShowAnalyzerEngine.

[tmiranda]

OK, just wanted to make sure. How many jobs are you launching at once? I want to try to reproduce your problem.

Tom

[Sambo555]

First I'd like to say a big thanks tmiranda for this plugin.

I'm just having a couple of minor problems with it.

My main problems is that the cleanup of comskip files is hit and miss. Would it be possible to have it look for orphaned comskip files and clean those up, either manually or at user set intervals or whenever CD does it's thing?

I think my file cleanup issues are stemming from the fact that comskip on my machine takes about 5 hours on an hours worth of .ts file. Maybe this is preventing sage or the plugin from deleting the ts right away. I can then look back in the log later and find the ts has been deleted by Sage but the CD cleanup never seems to do it's thing after that. This happens mostly when watching live tv or deleting a recording that has not finished comskipping yet.

The other minor problem is about the CS icon on main menu. On my HD200, it will only appear when something else is active in that same menu section, for example, 'REC' or JavaHeapMonitor is visible then CS appears. It seems the CS will not activate that menu area on it's own even when comskip is still running. I see other people having CS icon related problem and maybe this observation will help in resolution.

[toricred]

Originally Posted by tmiranda

OK, just wanted to make sure. How many jobs are you launching at once? I want to try to reproduce your problem.

Tom

I'm running 3 threads at once. I only recall the issue when I'm recording several recordings at the same time. It might require 7 or more recordings to cause the problem.

[tmiranda]

Originally Posted by Sambo555

My main problems is that the cleanup of comskip files is hit and miss. Would it be possible to have it look for orphaned comskip files and clean those up, either manually or at user set intervals or whenever CD does it's thing?

The other minor problem is about the CS icon on main menu. On my HD200, it will only appear when something else is active in that same menu section, for example, 'REC' or JavaHeapMonitor is visible then CS appears. It seems the CS will not activate that menu area on it's own even when comskip is still running. I see other people having CS icon related problem and maybe this observation will help in resolution.

Thanks for the observation on the CS Icon, I think that will help. The STV will only display the icon area if certain things are in progress but I thought I'd added a check in there for the CS icon. Maybe I missed it.....

I don't know why you are having problems with cleanup. The cleanup happens when you manually delete the recording (or Sage automatically deletes it) so it should not matter how long comskip takes. I don't quite understand your comment about "preventing Sage or the plugin from deleting the ts right away". The plugin will never delete the ts, it will only delete the corresponding edl (and other extensions you specify) after you or Sage deletes the ts.

Furthermore, it will not delete the edl if any valid video file still exists. So if you have MyShow-XXXX.mpg, MyShow-XXXX.ts, and MyShow-XXXX.avi, it will only delete the edl when all three of the video files are deleted.

I had thought about putting in an option to manually cleanup any orphaned edl files but never got around to it. I'll see if I can get that into a future release.

Thanks for the feedback,

Tom

[jerryt]

Is it possible to use CommercialDetector to run CCextractor.exe along with comskip.exe?

Or do I need Dirmon2...

[electronics4lif]

Originally Posted by tmiranda

I figured I post a little about what I'm currently working on to see what people think about this idea and if you have any suggestions. I'm working on implementing "Intelligent Comskip Scheduling".

What this will do is only run comskip when nothing is recording and the job(s) will likely finish before any upcoming recordings. In order to do this it will dynamically keep track of how long recordings take to process (on a channel-by-channel basis) so if it finds a "gap" in the recording schedule it will launch just enough queued jobs so that they all finish before the next scheduled recording.

It will still honor any times that you have restricted comskip from running but if this works as I hope it should eliminate the need for using that setting.

I think this option would be great for those of us that have a large number of tuners with only a small number of disks to record to! I am in this group with 8 tuners recording to 3 drives. I currently have comskip monitor set up with black out periods to avoid stuttering during 'prime time' recording. I am eagerly awaiting the addition of your intelligent recording. I know SJQ can do this, but it had too high of a learning curve for me to bother with it.

[tmiranda]

Originally Posted by jerryt

Is it possible to use CommercialDetector to run CCextractor.exe along with comskip.exe?

Or do I need Dirmon2...

I think you will be better off using Dirmon2 or SJQ.

[tmiranda]

Originally Posted by electronics4lif

I think this option would be great for those of us that have a large number of tuners with only a small number of disks to record to!

I am actually in the same boat and that is one of my main drivers for implementing this. If you are interested in helping me test please PM me your email address and I'll send you somethig when it's ready. I would like more data than just my system can provide so I can "tune" things to work as smooth as possible. The way it's architected right now it will try to estimate how long jobs take to finish and as more jobs are run it should get better at estimating. Of course this is all theory until I can get enough data samples.

There is already the ability to totally block comskip from running during certian periods, I added that in the last update.

[toricred]

@tmiranda OK, I had the problem with some shows not getting analyzed last night and I had the debug log set to trace. Which log file should I grab and provide?

[Sambo555]

Originally Posted by tmiranda

I don't know why you are having problems with cleanup. The cleanup happens when you manually delete the recording (or Sage automatically deletes it) so it should not matter how long comskip takes. I don't quite understand your comment about "preventing Sage or the plugin from deleting the ts right away". The plugin will never delete the ts, it will only delete the corresponding edl (and other extensions you specify) after you or Sage deletes the ts.

Furthermore, it will not delete the edl if any valid video file still exists. So if you have MyShow-XXXX.mpg, MyShow-XXXX.ts, and MyShow-XXXX.avi, it will only delete the edl when all three of the video files are deleted.

Oops, what I meant to say was 'preventing Sage from deleting the ts file, thus preventing CD from deleting the comskip files. That'll teach me to not try to troubleshoot this stuff when I'm sick.

I just tried a test tonight. I watched LiveTV, which started off the comskip process. I then stopped watching the show and looked at sage log. Sage had not deleted the file yet but had moved it to the partial recordings. I don't know if this was because comskip is still running or because that is the way Sage handles deletions normally. I then deleted the show in the partial recordings menu to see what would happen and have attached an excerpt from the log file to show what occurred after that.

Upshot is that the .ts file was deleted successfully but the comskip files are still there. Maybe the log will shed more light.

Kelly

[tmiranda]

Originally Posted by toricred

@tmiranda OK, I had the problem with some shows not getting analyzed last night and I had the debug log set to trace. Which log file should I grab and provide?

The sage log file from the server: sagetv_0.txt.