Commercial Advance Beta Release

[mightyt]

Originally Posted by Jere_Jones

At this point comskip really WOULD work on LiveTV.

Jere

Next you'll be telling us you can Comskip shows before they air!!!!

[ajuhawk]

Originally Posted by mightyt

Next you'll be telling us you can Comskip shows before they air!!!!

Now there's the best idea yet!! *Thinking out loud* I wonder if a "database" could be set up that stored comskip files for reruns of shows...the problem would be sinking it up with where the show started and ... well, maybe someday

-AJ

[Deadbolt]

Originally Posted by Jere_Jones

Ok, I've got comskip scanning a file as it's being recorded. I've had it running on three shows as I was recording them with 15% cpu (after the logo had been identified). I also watched one of the shows as it was being recorded and comskip scanned (in addition to the other two shows being recorded at this time. As far as I could tell there was NO disk thrashing going on. Maybe comskip was getting it's info from a cache or something since it had just been written to the disk. :shrug:

Anyway, I'd like some ideas from the StudioHaves on how to notify sage that it needs to reload the .txt file. Right now, comskip acts as normal, just that the file is processed as it is recorded. Eventually, I plan on comskip writing identified commercials to disk as soon as it can. But to be useful, sage would have to reload the txt file and I don't think there is any reason to reload it every second. Maybe use one of the custom commands to reload the file. Then if comskip wrote a new commercial break to the file, then comskip would send an event to sage to say "HEY, there's new information!!! Come and get it!!!!" or something similar. At this point comskip really WOULD work on LiveTV.

I'll release it as soon as I get Deadbolt's log so I can figure out why it's crashing on him.
Jere

Maybe it was a fluke...I've been logging verbose ever since and I haven't had another crash.

[Cayars]

I'm on it. It's funny because I already thought of running it on the live recordings. I was going to modify the source to do so but I'd much rather have Jere Jones do the work so it's part of the base and one less thing for me to do.

Jere, what I'd suggest is changing the program so it writes the file as it finds the commercials. Keep the file closed and only open it when you are going to write to it and then close it right away, etc... What I'd suggest is adding a "File Processing Complete" message at the bottom of the text file when you are done processing it.

From an STV standpoint what can be done is load the file. If we see the "File Processing Complete" type flag message then we know to treat it exactly as it is today. If it's currently being processed (no flag) then we have a couple of choices.

We could re-load the file every 30 seconds/minute etc until we see the flag and know it's complete or we could build in a little smarts. For example if we know this is a commercial break in the file, wait until the user gets to this, then when we get to the commercial break and the code in the OSD is going to skip it we reload the file again. If another break has been found we'de wait until the break to reload the file. If there is no break found then we switch to re-loading every XX seconds, etc...

I will surely support this in the Cayars STV as I was planning on doing it anyway!

[ajuhawk]

Okay, I've had one of those *standing in the shower* thoughts, so forgive me if this sounds unreasonable. If I understand (from having read all 544 previous posts) correctly, Malore and Jere have been working mostly on the comskip program. For detection purposes you've been mostly concentrating on finding the commercials - why not reverse this? That is, why not try to find the show, if it's not part of the show, skip it. Other then the end bits after the final commercial you normally have 5-10 mins of show with minimal fade to black or black frames present. Maybe you all ready take this into account in some way....but I'd be interested in finding out.

-AJ

[abasu2003]

That is, why not try to find the show, if it's not part of the show, skip it.

Interesting "thiinking oustside the box" concept but could you elaborate more onthis? They do already look for the show a little bit when they do logo detection since logos only usually show up as part of the show.

[abasu2003]

I might have missed this but if I only wanted logo detection and CC in my com processing, would this be possible with a detect_method value of 18 (16+2)?

[ajuhawk]

Originally Posted by abasu2003

Interesting "thiinking oustside the box" concept but could you elaborate more onthis? They do already look for the show a little bit when they do logo detection since logos only usually show up as part of the show.

The problem I'm finding - and it well could just be in my area, I don't know - is that the logos DO show up during commercials as well. So on some channels I find that 1 or 2 commercials are left at the end or beginning of a show segment. I've also been finding that it'll detect, say, 3 commercials, then miss 1, and detect the following 3 before the show starts.

I guess I'm grasping for how exactly to explain what I mean by "detect the show" as there's no simple solution that's going to work in every case. But maybe if there was a min_show_seg_length setting or something - in addition to the all ready present stuff. However, if you do that then within the final, oh, 4-5 mins of the show it would have to be excluded as a lot of shows have a final 1-2 min part after a commercial break. Also, if you record a football game or a race or something you'll run into problems since you never know when they'll go to break (especially with NASCAR races) and how long they'll be in before the next break.

Does this make any sense?

-AJ

[Jere_Jones]

Originally Posted by abasu2003

I might have missed this but if I only wanted logo detection and CC in my com processing, would this be possible with a detect_method value of 18 (16+2)?

Very close. You need to include black frame detection as well (comskip won't run without it). So in theory, 19 would be correct. Unfortunately, I don't have the slightest idea of how to use the CC data without using the fuzzy logic. So you would need to use 27.

Jere

[Jere_Jones]

Originally Posted by ajuhawk

But maybe if there was a min_show_seg_length setting or something - in addition to the all ready present stuff.

Oddly enough, I included this in my last update (version 0.61). Except the actual property is min_show_segment_length. It defaults to 30 seconds.

Jere

[ajuhawk]

Originally Posted by Jere_Jones

Oddly enough, I included this in my last update (version 0.61). Except the actual property is min_show_segment_length. It defaults to 30 seconds.

Jere

Great minds think alike - or something along those lines.

Might be good to have a user adjustable setting... also a setting to exclude this setting towards the end?

-AJ

[Jere_Jones]

Originally Posted by Cayars

I'm on it. It's funny because I already thought of running it on the live recordings. I was going to modify the source to do so but I'd much rather have Jere Jones do the work so it's part of the base and one less thing for me to do.

It was an easy modification. I basically told the decoder that if it ran out of information, wait a second and try again. Amazingly effective for such a small change.

Originally Posted by Cayars

Jere, what I'd suggest is changing the program so it writes the file as it finds the commercials. Keep the file closed and only open it when you are going to write to it and then close it right away, etc... What I'd suggest is adding a "File Processing Complete" message at the bottom of the text file when you are done processing it.

I don't see a problem with that. I'm curious how the stv will react to non-digits, but I'm sure you'll take care of that. I'll leave implementation up to you, but I will make sure that I close it after each write.

Originally Posted by Cayars

From an STV standpoint what can be done is load the file. If we see the "File Processing Complete" type flag message then we know to treat it exactly as it is today. If it's currently being processed (no flag) then we have a couple of choices.

We could re-load the file every 30 seconds/minute etc until we see the flag and know it's complete or we could build in a little smarts. For example if we know this is a commercial break in the file, wait until the user gets to this, then when we get to the commercial break and the code in the OSD is going to skip it we reload the file again. If another break has been found we'de wait until the break to reload the file. If there is no break found then we switch to re-loading every XX seconds, etc...

I will surely support this in the Cayars STV as I was planning on doing it anyway!

I'll go ahead an release a version that will run on currently recording files. It won't output any commercials until the end though. I'll work on that next.

Food for thought: Scanning a file while it is recording, watching another or the same file, doing other things, etc, can get VERY processor intensive, so I recommend making LiveDetection(c) an option.

Jere

[Jere_Jones]

Originally Posted by ajuhawk

Might be good to have a user adjustable setting...

I'm confused. What do you mean? It is already user adjustable because it is a potential ini file property.

Originally Posted by ajuhawk

also a setting to exclude this setting towards the end?

I'll look into this. Of course, unless you make min_show_segment_length something really long like 90 or larger, it shouldn't make a difference. It'll probably be something like min_show_segment_exclude=.25 so that during the last 25% of the show, the setting is ignored. Is that what you were looking for?

Jere

[ajuhawk]

Originally Posted by Jere_Jones

I'm confused. What do you mean? It is already user adjustable because it is a potential ini file property.

My mistake - I didn't look at the file.

Originally Posted by Jere_Jones

I'll look into this. Of course, unless you make min_show_segment_length something really long like 90 or larger, it shouldn't make a difference. It'll probably be something like min_show_segment_exclude=.25 so that during the last 25% of the show, the setting is ignored. Is that what you were looking for?

Jere

I guess that's what I'm thinking - set the min_show_segment_length higher so that you don't get the false positives (i.e. a commercial as a show) earlier in the show and then have a way that it won't look for this towards the end of the show - or a secondary setting that it uses in the last segment.... so, yes, what you describe is basically what I'm thinking of.

-AJ

Edit: One other problem I just found with my idea is that some shows also have the small intro bit before the first commercial. So in this case you'd also need a beginning ignore period. So maybe this idea of having a large show segment isn't so good...

[Cayars]

Originally Posted by Jere_Jones

<snip>
I don't see a problem with that. I'm curious how the stv will react to non-digits, but I'm sure you'll take care of that. I'll leave implementation up to you, but I will make sure that I close it after each write.
I'll go ahead an release a version that will run on currently recording files. It won't output any commercials until the end though. I'll work on that next.
<snip>

Yep, might break some other STV's that aren't setup for it. Tell you what. Instead of "text" could you right a "cut point" that has the exact same starting and ending times?

Example:
1 1571
40198 42000
42902 45644
64711 67416
70113 73290
92341 95050
97299 99555
120397 126249
154369 158222
159110 161850
181602 189290
208892 216567
240266 244816
278566 278566 <-- Just use the last from of the file but make them the same.

[Jere_Jones]

There is a way already coded. According to malore, the file parser scans through the file until it finds a line containing only dashes then begins parsing the lines. Kinda like this:

All the
information in the world
here doesn't make a difference...
--------
1002 5003
15004 22002

I could easily put something before hand like this:

File Processing Complete
--------
1002 5003
15004 22002

This wouldn't break anything and would still be useful. But, the whole process during recording is going to be a setting. It will be turn-off/on-able. So any stv that doesn't account for this will just need to not set the live_tv setting.

I think this approach may be better because it is kinda "self-documenting". Doing a 0 frame break will surely confuse someone who comes on the scene later.

Jere

[ajuhawk]

Jere,

I was looking through your code and noticed that there's a long list of variables that it is possible for you to import from the .ini file. However, a lot of them are ones that I don't recall having been described before - is there a list of what they are all for somewhere?

Secondly - I was messing with 0.61 last night trying to get it to detect all the commercials. After many variations I finally gave up, then this morning it dawned on me that the score_percentile variable wasn't in the latest .ini. So I did a calculation and changed/added this to 0.70 and behold it detected all the commercials fabulously. How do you use this value in your code? Does the show HAVE to be 70% or more of the file (if I have it set to 70%, obviously the variable itself can be changed)?

Thanks,

-AJ

[Cayars]

That would work just fine.

[Jere_Jones]

Originally Posted by ajuhawk

is there a list of what they are all for somewhere?

Take a look at my signature.

Jere

[Cayars]

Originally Posted by Jere_Jones

Take a look at my signature.

Jere

That's too funny!