Plugin: CommercialDetector

[dealsdyker]

Hi Tom,

I'm "kind of" sure I do have the UI Mod Plugin installed on the server where I ran the test but will confirm when I get home. The reason I'm pretty sure is because I went to the "All" tab and I'm pretty sure I saw it there. I also recall seeing the button in Sage/Setup for comskip playback options. Looked at the settings but did not change any. Also, on some shows commercials are found. On some they are not. But all the shows play fine in both Sage and Windows Media player.

I watched the recordings directory through file manager. When I and pushed the button to delete the comsip files I verified through file manager they disappeared. Additionally I also tried manually deleting the comskip files and pushing "run comskip" on the file. In all cases the CS support files disappeared and then reappeared when I pushed "run comskip".

And when I do this the "CS" icon appears in the header area the in the list of files that particular file has the CS icon in a slightly "off" color (maroon?) which I assume to mean it is processing. After about 6-8 minutes the CS in the header disappears and the color of CS on that show goes to the "normal CS" color. So I assumed I waited long enough. Additionally, all the files show as "existing" on the directory again with the date/time when I ran it, with the EDL empty and the one file containing
FILE PROCESSING COMPLETE 107838 FRAMES AT 2997
-------------------
107837 107838

What is odd is some shows do CS just fine, while some have zero length EDL files. And if I re-process a file that processed OK before it processes OK again (I can watch the files get deleted and recreated on those too).

Any ideas? Maybe I should just delete all the comskip support files where the EDL is zero length, uninstall and reinstall commercial skip on the server and client, push the button again to recreate all the CS files (in the settings of the plug in) and let it run overnight tonight. One more thing, I also have tried pushing the "reset all settings" button on the Comskip server options.

[Fuzzy]

Sounds like comskip is definitely running on the files fine, and there is no problem with CommercialDetector. It looks like comskip is having some problem with the recordings themselves. You'd need to look at the comskip log files (you'll have to turn logging on in the comskip.ini). It may point to a problem in the recordings themselves, that comskip is hanging up on.

[tmiranda]

As Fuzzy said, it appears that comskip is running because thte "off color" CS icon indicates that the file is in-process and the normal color indicates it has completed.

Edit the comskip.ini file and change verbose=0 to verbose=10. Make sure the CommercialDetector loglevel is set to Verbose. Send me the logfile after doing that and processing one or two of these troublesome files.

[95TBird]

I saw the "empty EDL" issue on Jeopardy Monday & Tuesday & I was able to determine that it was caused by the "ticker bar" that was running on the bottom of the screen (they were announcing school closures) Apparently the bar renders comskip unable to properly detect the logo.

-Bruce

[tmiranda]

Originally Posted by 95TBird

I saw the "empty EDL" issue on Jeopardy Monday & Tuesday & I was able to determine that it was caused by the "ticker bar" that was running on the bottom of the screen (they were announcing school closures) Apparently the bar renders comskip unable to properly detect the logo.

-Bruce

Yes, Something like this is the most likely culprit. Comskip is very good, but it's not infalible.

[dealsdyker]

That might explain Tuesday's Biggest Loser... I recall the local network had "shrunk" the video some and put a bunch of closings along the bottom. I will look closer at the recordings and in light of that, see if there are still recordings that cannot be explained.

[jlmdxtv]

Hi Tom,

I perform post-recording comskip processing only.

Is it possible to have comskip wait a few (eg. 10 or 20) seconds after a show is finished before starting its commercial detection?

I'm asking this because I use use the "multifunction stop plugin" often to "stop recoding and delete now" news and another shows that don't keep my interest, but as soon as I stop a recording comskip takes over, locks up the file while it starts its thing, and the show isn't easily deleted and/or edl files are left behind as orphans.

thank, john

[tmiranda]

John,

Are you getting an error or warning from Sage saying the file can't be deleted or does it appear to successfully delete and then the .mpg and .edl are left behind?

I thought I had put in code to handle this situation so I guess I need to make sure it's working as advertised.

Tom

[jlmdxtv]

Tom,

The "stop" plugin always stops the recording when I ask it to, but typically doesn't delete the recording file as expected - no error messages in the UI ever about it being in use.

If the recording is still there (almost always), comskip starts right up and I delete the recording file from the recordings list a few seconds later. The recording file is deleted, comskip stops, and the orphans are left behind.

Comskip likely starts the moment the recording is stopped and before the command to delete it takes place. That's why it's there on the "Recordings" list moments later. If comskip wasn't so quick to open the file and process it, I think the "stop" plugin would be able to delete it properly and the orphan issue would go away.

john

[dealsdyker]

It seems that some of the files do, in fact have school closings etc. But here is some weirdness. On the server I record all to e:\tvrecs\*.*. And prior to earlier this week, I had "e:\tvrecs\" as the recordings directory. But I recently (to use this plug-in) changed to UNC \\servername\tvrecs which seemed to work fine. I'd say 80-90% of the files, when I display "view recording detail" show the UNC path now. But on some of the problem files the file location still says e:\tvrecs\Programname_... under the "View Recording Detail" in Sage. I'm assuming that commercial skip looks at this field to determine where the comskip support files are. It sees "E:" on the client machine (on which there is no E: drive). And then comskip can't work on that file. Sage still serves the file cuz on the server E: exists, but commskip on the client cannot resolve E:\. On new recordings it seems to be storing the file location as \\servername\tvrecs\programname correctly.

I thought by changing to UNC that Sage would change all the file location references to the UNC for any existing recording/file. I looked and the media imports to not reference e specifically, nor does the recorded TV directory. So why do a few files still have a hard-code reference to e:\... this might be why some files don't seem to be able to CS from the TV computer.

So more importantly how do I fix so that all recordings metadata switches over to UNC instead of e:\tvrecs\... (or is this not a problem and I'm down the wrong path). I can post in another thread if that is best.

[tmiranda]

comskip.exe always runs on the Server so it can always find the file. If it can't you have bigger problems

To get all of the files updated to UNC paths Sage must "see" that e:\tvrecs is empty. There are loads of discussons on this on the forums, I'd suggesting searching for "UNC path".

[dealsdyker]

Thanks Tom. What I noticed is the files still coded with E:\TVRECS\... is that from the Client Machine (which has no E: drive) when I select the show instead of "Delete Comskip Files" I get the "Run Comskip" selection, even though those comskip files actually exist out on the server. From the server itself I get "Delete Comskip files" on the same popup, I'm guessing, because the server can see them either way.

I wonder if I go to E:\ and rename the directory tvrecs to something else, will the UNC path stay "tvrecs"? I guess I better search because I don't want to mess things up more.

[tmiranda]

Originally Posted by dealsdyker

What I noticed is the files still coded with E:\TVRECS\... is that from the Client Machine (which has no E: drive) when I select the show instead of "Delete Comskip Files" I get the "Run Comskip" selection, even though those comskip files actually exist out on the server.

That's totally normal because your Client can't see E:\tvrecs. Double check these instructions but I believe doing this will get all the paths changed on the server:


- Stop sage.
- Rename the e:\tvrecs directory to something else like e:\tvrecs2.
- Make your share point to tvrecs2
- Make an empty directory e:\tvrecs
- Restart Sage

Sage will scan e:\tvrecs and when it finds it is empty it will update all of the recordings to use the share name.

That should solve the last of your problems.

[rockinray]

Hey Tom,

This looks to be a fantastic setup.... Except I cannot for the life of me get it to work!

I am running 64 Bit server, running Sage as "root", I know....

I have installed wine without issue and I have installed the plug in...

When I go to set things up, I ask to scan the shows that have already been recorded. It finds them (in my case 9) and it appears the job starts. The icon is displayed on the screen, it is purple... It appears to run for a bit and if I arrow down to see the shows that are currently being processed, the icon disappears and when I land on the current processing shows, it will go from say 2 to 0. Then the program appears to die. I have also tried to kick off the scan in the recored shows menu (note the CS icon does appear next the the show with a red - slash) and it does not appear to do anything.

So, I am kind of stuck here and wondering what I could try next? I do have a powerful machine running this so that should not be a problem.

I was thinking maybe a permissions issue??? Not sure.

Thanks for any advice you could provide!

--Ray

[stuckless]

Originally Posted by rockinray

Hey Tom,

This looks to be a fantastic setup.... Except I cannot for the life of me get it to work!

I am running 64 Bit server, running Sage as "root", I know....

I have installed wine without issue and I have installed the plug in...

When I go to set things up, I ask to scan the shows that have already been recorded. It finds them (in my case 9) and it appears the job starts. The icon is displayed on the screen, it is purple... It appears to run for a bit and if I arrow down to see the shows that are currently being processed, the icon disappears and when I land on the current processing shows, it will go from say 2 to 0. Then the program appears to die. I have also tried to kick off the scan in the recored shows menu (note the CS icon does appear next the the show with a red - slash) and it does not appear to do anything.

So, I am kind of stuck here and wondering what I could try next? I do have a powerful machine running this so that should not be a problem.

I was thinking maybe a permissions issue??? Not sure.

Thanks for any advice you could provide!

--Ray

Just to confirm... you did follow the linux specific configuration...

If your Sage server is running on linux do the following:

You MUST install wine on your linux server under a normal user account (not root). CommercialDetector uses comskip to scan the recordings and comskip is a native Windows program. comskip does work quite well under wine and in some cases actually runs faster under wine than under Windows. Do NOT install wine from the root account.
If you are running the Sage server as root I suggest making a special account solely for the purpose of running comskip. If you call the account comskip, or anything that has "commercial", "sage", "wine" or "detect" in it, the plugin should find that account and use it as the default.
If you are not running Sage under the root account, install wine under the same user you are using for sage.
Go to the plugin manager->All Installed Plugins and select the "General" section.
Select the CommercialDetector plugin, then select "Configure".
Navigate down the list of options until you find "SageTV is Running as Root". Set this to True or False according to your setup.
If the Sage server is running as root you must select a user account that will be used to run comskip. This should be the user account where you installed wine in step 1.
If the Sage server is not running as root, enter the user account that is running Sage.
If you installed wine in the normal way the Home directory for wine default will be OK. If you did something funky you can enter the appropriate Home directory for wine.

And if you have HD recordings you'll need to donate to get the HD version of comskip.

[rockinray]

I think that is the issue. All of my recordings are HD! I was unaware of the donation needed.

I will do some further reading on this and donate.

So, questions:

1. What is the best approach to install the software since I will not "package" them together?
2. Do I simply put Comskip in a seperate directory and move the files to the correct directory once I install the CommercialDetector Plug In???

Just looking for best approach on this!

Thanks!

--Ray

[stuckless]

Originally Posted by rockinray

I think that is the issue. All of my recordings are HD! I was unaware of the donation needed.

I will do some further reading on this and donate.

So, questions:

1. What is the best approach to install the software since I will not "package" them together?
2. Do I simply put Comskip in a seperate directory and move the files to the correct directory once I install the CommercialDetector Plug In???

Just looking for best approach on this!

Thanks!

--Ray

Both of those questions are answered in the comskip wiki (which I copied from in my post above). The wiki is linked in the first post.

[cjbojrab]

I installed the CommercialDetector plugin and it in turn installed comskip, dated 8/15/2010. But I notice there is a later version of comskip available here: http://www.kaashoek.com/files/. Is it safe to download the latest version of comskip and use it in place of the 8/15/2010 version? I don't know what's different, but just wanted to update to the latest and greatest.

[gplasky]

If you want to be able to comskip any recrodings in h.264 format you need to donate $10 for the special donators version that the later versions are. You are then provided a login and password to download the file from.

Gerry

[tmiranda]

Originally Posted by cjbojrab

I installed the CommercialDetector plugin and it in turn installed comskip, dated 8/15/2010. But I notice there is a later version of comskip available here: http://www.kaashoek.com/files/. Is it safe to download the latest version of comskip and use it in place of the 8/15/2010 version? I don't know what's different, but just wanted to update to the latest and greatest.

You can use whatever version of comskip you like. Unzip comskip to the location of your choosing and then go to the CommercialDetector setup options and point it at comskip.exe and comskip.ini. You should proabably tweak the comskip.ini to make sure the follwoing are set:

live_tv=1
output_default=0
output_edl=1
verbose=0
delete_logo=1 (check this one, I'm going from memory.)