Commercial Advance Beta Release

[ajuhawk]

Heh, figures I'd look in the wrong spot. Sorry - what I get for staying up so late.

Ok - and I'm sorry if this too is in the FAQ - but I set the score_percentile even lower (0.60) to try and make it work better on an hour long show and now it detects fewer commercials on a half hour show - any reason for this that you know of?

and if it's in your sig, just say so and I'll go digging...

-AJ

[Jere_Jones]

Originally Posted by ajuhawk

Heh, figures I'd look in the wrong spot. Sorry - what I get for staying up so late.

No big deal.

Originally Posted by ajuhawk

Ok - and I'm sorry if this too is in the FAQ - but I set the score_percentile even lower (0.60) to try and make it work better on an hour long show and now it detects fewer commercials on a half hour show - any reason for this that you know of?

You have hit on the major problem of detecting commercials. It's easy to say "this segment is MORE like a commercial" or "this segment is MORE like a show". The catch comes in where do you draw the line. That's what the score_percentile does. It sorts all the frames by score and draws a line where ever score_percentile is set. Everything above the line is show and everything below is commercial. But this doesn't set EXACTLY that percentage as show since score adjustments are made on blocks of frames. What happened in your case was probably the segments were large enough to make for funny results. That's why the most overlooked but most important setting is the black_percentile setting. It really determines how accurately the segments are divided.

End answer: I'd need to see the log file to give you a definitive answer, but it shouldn't work that way.

Jere

[Pumpkinhead]

A lot of the settings are missing from the included ini file. I got the missing ones off the website, although some of those differ than the settings included in the included ini file, such as max_avg_brightness.

[ajuhawk]

Ok, I've been messing with settings off and on today and I've come to the conclusion that it's impossible to have settings that work for both half hour shows and hour+ shows. Has this also been the experience of others?

I've managd to find a "happy" medium. On the few test files that I was running it seemed to work well for the half hour shows and ok on the hour shows. That is that it detects the commercials on the half hour show correctly but misses some on the hour show - oddly enough one of those was a welbutran commercial that had the keywords from the dictionary (it also called part of the show commercial). The problem I'm running into, some of which I mentioned previously, is that an hour show tends to be only 60% show whereas a half hour show tends to be 70% show.

Would a possible solution be to have 2 settings, one for half hour shows and one for hour+ shows? Perhaps others have found other work arounds.... I'll attach my .ini for comparison.

-AJ

[Deadbolt]

Every once in awhile I'll have a recording that is split into two pieces. This messes up my comskip...is there anyway to have the auto comskip look for _0,_1,..._n files and make .txt files for each?

Thanks!

[davey_fl]

I've tried everything but commskip doesn't work for me. I can drang and drop a file onto it and it works, but it won't work from within sage. I get an immediate "Commecrial Detection Failed" message everytime. All paths are default, just the standard install of sage 2.1. Any suggestions?

thx

[JasonJoel]

Personally, I got better commercial detection results overall BEFORE the %-of-show setting was put in at all!

I've messed with it for probably 30 different settings, and gave up. No matter what it would not work on some shows - specifically because of this setting. Some shows simply have more/less show than others! So I think this setting does more harm than good... But that is me. I could be wrong.

I still wish the setting could be completely bypassed...

Jason

Originally Posted by ajuhawk

The problem I'm running into, some of which I mentioned previously, is that an hour show tends to be only 60% show whereas a half hour show tends to be 70% show.

-AJ

[Jere_Jones]

Originally Posted by JasonJoel

Personally, I got better commercial detection results overall BEFORE the %-of-show setting was put in at all!

I've messed with it for probably 30 different settings, and gave up. No matter what it would not work on some shows - specifically because of this setting. Some shows simply have more/less show than others! So I think this setting does more harm than good... But that is me. I could be wrong.

I still wish the setting could be completely bypassed...

Jason

I hope you'll forgive the shortness of my answer, but I answered over a month ago that you CAN bypass the setting. See here.

Of course if you don't want to go that route (I expect that it will give horrible results), you can always use detect_method=3. That will remove all the "scoring" detection methods.

I am also open for ideas. If you (or anyone else, I'm not proud) has an idea for how to incorporate sources of information that aren't black and white (i.e. certainly commercial or certainly show) without sometype of sliding scale, please share it and I will do my best to code it up.

In the mean time, I recommend that you look at your black_percentile setting. Good results are more sensitive to that setting than any other. You (JasonJoel in this case) would probably benefit from reducing it SLIGHTLY. Set verbose=1 and reduce the setting until the numbers under "sbf" (Starting Black Frames) are reasonable (i.e. appropriate for the duration of the black gap). Then see how it works.

The other option is set verbose=10 and send me your log file. I'll take a look at it and make recommendations to you or I'll fine tune the code. I've offered this to anyone and continue to offer it.

Well, I guess my answer wasn't so short.

Jere

[JasonJoel]

I remembered you telling me that, and I tried it. Honestly at the time bypassing it didn't seem to work. Haven't tried since then though. On the couple of files I tried, I seemed to get the EXACT same detection. I should have mentioned that it didn't seem to work for me then...

But, I guess it is possible (likely?) that I messed something up when I tried it.

And don't get me wrong, OVERALL it is working pretty well for me. For some reason it seems to have gotten somewhat worse from the late 0.3x/early 0.4x builds to now.

But that could just be a function of my settings. Maybe it is time to make another stab at it.

As always, thanks for the help. I would never get rid of it at this point, whether it never got any better from here on or not!

Jason Bottjen

Originally Posted by Jere_Jones

I hope you'll forgive the shortness of my answer, but I answered over a month ago that you CAN bypass the setting. See here.

Of course if you don't want to go that route (I expect that it will give horrible results), you can always use detect_method=3. That will remove all the "scoring" detection methods.

I am also open for ideas. If you (or anyone else, I'm not proud) has an idea for how to incorporate sources of information that aren't black and white (i.e. certainly commercial or certainly show) without sometype of sliding scale, please share it and I will do my best to code it up.

In the mean time, I recommend that you look at your black_percentile setting. Good results are more sensitive to that setting than any other. You (JasonJoel in this case) would probably benefit from reducing it SLIGHTLY. Set verbose=1 and reduce the setting until the numbers under "sbf" (Starting Black Frames) are reasonable (i.e. appropriate for the duration of the black gap). Then see how it works.

The other option is set verbose=10 and send me your log file. I'll take a look at it and make recommendations to you or I'll fine tune the code. I've offered this to anyone and continue to offer it.

Well, I guess my answer wasn't so short.

Jere

[sleonard]

OK, I really have no clue as to if this could work but I'm wondering if it would be possible to create profiles for the most popular shows that would have information relevant for commercial detection that is specific to that show. Comskip could then look up the profile based on the filename. These profiles could be generated by comskip itself with some of the same info put into the log files. If a profile were off a bit, it could be tweaked manually until it was perfect. Alternatively (and probably far more complicated), there could be some way to give feedback on the errors and rerun comskip until the profile was perfect. As profiles were generated by the community they could be shared and eventually we would have one for most everything on TV.

What do you think? Is this an idea worth thinking about or should I lay off the crack pipe

[Cayars]

Originally Posted by sleonard

OK, I really have no clue as to if this could work but I'm wondering if it would be possible to create profiles for the most popular shows that would have information relevant for commercial detection that is specific to that show. Comskip could then look up the profile based on the filename. These profiles could be generated by comskip itself with some of the same info put into the log files. If a profile were off a bit, it could be tweaked manually until it was perfect. Alternatively (and probably far more complicated), there could be some way to give feedback on the errors and rerun comskip until the profile was perfect. As profiles were generated by the community they could be shared and eventually we would have one for most everything on TV.

What do you think? Is this an idea worth thinking about or should I lay off the crack pipe

This is quite possible to do and it doesn't need a change to the ComSpec program. What you do is create different INI files based on Channel or Show and use this specific INI file when processing that particular show. This same technique could be used for 30 minute or =>60 minute shows.

When you start getting to this level, I would stop having your STV doing the processing and have this controlled by scripting outside of SageTV so you can tweek it and control the settings.

Carlo

[Jere_Jones]

For those of you that have wondered if comskip will ever reach a final version, well here it is. I will not be developing this GPL version of comskip anymore (at least until I get ahold of Studio). I, of course, will continue to try to fix any bugs.

This version includes the following changes:

• Will do realtime commercial detection (this is up to the Studio-haves to implement)
• Added aggressive_logo_detection property. This will use some extra logic to reject logos that it isn't positive about.
• Added edge_radius and edge_level_threshold properties. These will modify how sharp and drastic an edge must be for logo detection. (Hint: the more transparent the logo, the lower edge_level_threshold must be to detect the logo)
• Adjustments were made to the aspect ratio code to determine Anamorphic or Widescreen instead of just a number.

If there are any bugs, let me know and I'll try to correct them.

Jere

[ToxMox]

Wow Jere! Thats some nice work there. Keep up the great job you've been doing

[lovingHDTV]

Jere,
Just a thanks for all your work. After I enabled CC the commercial detect has been right on for all my shows (in fact I can't remember the last time it actually incorrectly marked a commercial). It has become a must have in our SageTV viewing.

thanks,

[sleonard]

Originally Posted by Jere Jones

I will not be developing this GPL version of comskip anymore .

/putting on tin foil conspiricy hat

Are you going to be developing a non-GPL version? Did you have a comskip revelation and realize it was too valuable for the free community?
Or, alternatively, did you get a visit from the FBI regarding the upcoming legislative ban on commercial skipping?

/Removing tin foil conspiricy hat

OK, just in case the tin foil hat statements weren't enough clue the above was pure humor.


Or was it. Hmmmmm.

Seriously though, Thanks for all the hard work. You have taken this wonderful little program to levels barely even dreamed of by mere mortals such as I. Great job!

(at least until I get ahold of Studio)

I hereby formally nominate you for the Studio beta testers club
Can I get a 2nd on that? (preferably from one of the two people who actually count)

Will do realtime commercial detection (this is up to the Studio-haves to implement)

Studio-haves, it's time to step up to the plate. Maldude 1st since I use his STV

[justme]

I hereby formally nominate you for the Studio beta testers club
Can I get a 2nd on that? (preferably from one of the two people who actually count)

For what it's worth(almost certianly nothing), I'll second you motion. Few have given as much as Jere Jones, to the Sage community. I can't say how much I appreciate comskip.

As for the Studio haves, adding the support for Live detection... I'm sure Cayars will be first. He's on a huge roll recently.

[Cayars]

Originally Posted by justme

For what it's worth(almost certianly nothing), I'll second you motion. Few have given as much as Jere Jones, to the Sage community. I can't say how much I appreciate comskip.

As for the Studio haves, adding the support for Live detection... I'm sure Cayars will be first. He's on a huge roll recently.

Well actually if truth be know, Jere has given me a couple of advanced copies of it and made a change or two at my suggestions so it's probably a safe bet to assume the next version my STV will support real-time ComSkip.

[ToxMox]

Originally Posted by Cayars

Well actually if truth be know, Jere has given me a couple of advanced copies of it and made a change or two at my suggestions so it's probably a safe bet to assume the next version my STV will support real-time ComSkip.

Let me be the first to say. Woohoo!

[turak]

Originally Posted by lovingHDTV

Jere,
Just a thanks for all your work. After I enabled CC the commercial detect has been right on for all my shows (in fact I can't remember the last time it actually incorrectly marked a commercial). It has become a must have in our SageTV viewing.

thanks,

by enabling CC do you mean the registry key?

[mightyt]

Originally Posted by justme

For what it's worth(almost certianly nothing), I'll second you motion. Few have given as much as Jere Jones, to the Sage community. I can't say how much I appreciate comskip.

I nominate Jere as well ...

Studio would be a good reward for his great effort to provide a highly desired Sage feature ...

T.