Commercial Advance Beta Release

[malore]

This is intended for users who are familiar with changing .stv files and know how SageTV works. It also won't work,as is, on the SageTV Client because of file access issues

Note: Several other custom STV files also have added support for comskip and Jere_Jones has added several different detection methods and features to the comskip program. His latest version can usual be found near the end of this thread.

I've updated my custom .stv file to implement commercial advance, by reading a .txt file that has a list of the start and end frames for commercials. You'll need to assign buttons for the custom 4 and 5 commands. I suggest using "[" and "]" but anything will work and of course you can also use girder or something else to send these commands. Custom 5 skips ahead to the beginning of the next non-commercial segment. Custom 4 skips back to the previous segment, commercial or non-commercial. Commercials are shown in purple.


Note: At the moment the skips aren’t frame accurate, so you may see a couple seconds of commercial.

The commercial break .txt files are generated by running the enclosed comskip.exe file on the desired mpeg files. I have also added the ability to do this from the options menu of my custom SageTV Recordings. The process runs in the background, so you can do other things while it is working. You will receive a pop up message in SageTV when commercial detection is completed, I would suggest you not start commercial detection on more than one file at a time, but you call run commercial detection on an entire group at once. On my 2.4 GB P4 computer, commercial detection runs at around 200 frames per second (just under 7x normal speed) at full CPU usage. Comskip.exe runs at normal class/idle priority, so you shouldn’t notice a major system slow down. I have also implemented running commercial detection automatically when SageTV is asleep and no programs are scheduled to be recording in the next 15 minutes. This can be enabled on the Settings menu in the Customs Menu, but I haven’t had time to fully test it.

The commercial detection program will be open source. It uses libmpeg2 to read the mpeg2 files and I added commercial detection by rewriting and tweaking some of the commercial detection code from MythTV. Currently, I'm just using the blank scene detection, because the accuracy of the scene change detection was somewhat poor and the CPU usage was high. Any Visual C programmers interested in working on this?

Installation: After downloading, install malore.stv as normal and put comskip.exe and comskip.ini in the main SageTV folder where wiz.bin is.


Update August 2:
• Added Commercial Auto Skip. Turn it off or on in the media player's option menu. If it skips something that isn't a commercial, uses the Custom4 command to jump back. It won't skip the same section twice in a row.
• Background commercial detection (if enabled) will also run when the screen saver is active.
• The delete commands in my custom SageTV Recordings and the Recording Detail Screen will also attempt to delete the supplemental files (.xml,.smi,.txt) for that file.
• If the now_playing_file property is set to a file path, the media player will output the name, episode, duration, and current play position to the file indicated.
• Added [CA] and [CC] after the show name in the description frame to indicate that the commercial advance or closed captioning (.smi) files are available.

Update August 3:
• Stopped showing the time bar after auto skipping, because it wouldn't auto close.
• Added carriage return to the line breaks of the now playing file.
• The now playing file is deleted after leaving SageTV's media player.

Update August 4:
• The SageTV client is no longer prevented from processing commercial in the background.
• Added two settings to the sage.properties file.
- Set background_commercial_processing_awake=true to allow background commercial processing while SageTV is awake.
- Set background_commercial_processing_recording=true to allow background commercial processing while SageTV is recording another show.
• In the Custom Menus menu, the info command (ctrl-I), will bring up an about box with the date my STV was last updated.
• I've made a couple adjustments to the commercial detection settings that I found to work a little better, but rather than spending all my time trying to figure out the perfect, one-size-fits-all settings, I've made most of the black frame detection settings configurable with a settings file (comskip.ini) I also included a file that has close to the original settings.

Update August 14:
• Added adjustment to SageTV's playing time to hopefully better align the marked breaks with playback.
• You can now have background commercial processing ignore shows recorded on specific channels. Edit the sage.properties file and set background_commercial_processing_ignore= a list of channel number seperated by commas (ex. 78,11)
• Changed how the commercial autoskip function was implemented to hopefully work better with pvr-350's.
• The custom 5 command will now start commercial processing on the currently selected episode in my version of SageTV Recordings.
• Added additional levels to the verbose setting of the comskip.ini file.
verbose=1 shows building of commercial breaks on screen
verbose=2 writes building of commercial breaks to file
verbose=3 writes list of black frame breaks
verbose=4 writes list of every black frame detected

1) Download malorebeta.zip for SageTV 2.0 or
Go to Malore's STV Homepage for the latest SageTV 2.1 compatable version.
2) Backup your sage.properties and wiz.bin files, just to be safe.
3) Follow the instructions on Malore's STV Homepage

enjoy,
malore

[ErsatzTom]

I'll help in any way I can.

t

[malore]

Had to make a minor update.

Anyone wanting to help with my scene detection program should download libmpeg2 and be able to compile the mpeg2dec test program which is the basis of my program.

[Deadbolt]

very very cool! can't wait to try it!

Edit: Malore: Where in the options is the selection to do this? All I see is "create .xml for mpeg" and "create shows.xml"

[Fluffdaddy]

malore

Thanks for your time and effect. Commercial Advance is the only thing about Sage I miss and want dearly...............Yes even e-nuff to pay for it

[malore]

Deadbolt: It should show up right after the .xml option when viewing the options menu in my custom version of SageTV Recordings (to set this, Custom Menus --> Settings -> Sage Recordings Style -> malore) If you switched the STV file without restarting SageTV, see if restarting it helps. Also be sure to download from the link on the first post and not from my usual site.

[Deadbolt]

I had switched to the Maldude .stv, but switched back to pure Malore goodness afterward. Now I have upgraded to your beta. I don't see the commercial detect item. I have restarted SageTV, and even rebooted. No help. I deleted the mldude stuff and still no help. Any ideas? I have run comskip manually on a file, and have the .txt file, but it is not recognized in SageTV.

Is there a good way to clear all .stv stuff except originalv2.stv and then reload your beta? Maybe this is the best option.

Again, thanks for you efforts! Your skin alone has enhanced my (and my wife's) Sage experience imeasurably. Now, this commerical skip is another revolutionary step. I bet someone automates it soon..auto process new recordings, and also auto skip (so you don't even need to hit Custom 5) --- an we will be in heaven.

[malore]

Deadbolt, if one of the options in Custom Menus --> Settings isn't Background Commercial Processing then for some reason you're not using the beta version. Having other .stv files around shouldn't matter as long as the sage.properties file points to the correct .stv file. You might want to extract malore.stv from the malorebeta.zip file again and replace the one in the originalv2 folder again while SageTV isn't running.

[mightyt]

Hope this is not uncool to ask ...

Is this implimentable witrh MlbDude?

[chrysek]

Sorry for this but I cant get this to work Ok, I did put malore.stv and png file into C:\Program Files\Frey Technologies\SageTV\STVs\OriginalV2 directory.
comskip.exe is in C:\Program Files\Frey Technologies\SageTV directory
when I go to commercial detect it does came up with a window saying "Commercial Detection Failed". I think I did everything ok. I am trying it on a client system, not on sage server, maybe that is the problem? but I did put those files in the exactly same location on the server pc.

Any ideas why it would fail? Is there a way to see some kind of debuging logs for it? Do I need to install anything else except those 3 files?

This is great stuff I am so ancious to try it.

Chris

PS: just a little update:
it does not work from sage client machine, but it does work on sageserver.
on the client machine I did run in the dos window command:
C:\Program Files\Frey Technologies\SageTV>com
087-0.mpg
mpeg2dec-0.4.0 - by Michel Lespinasse <walken
20921 frames in 334.77 sec (62.49 fps), 60 la
20951 frames decoded in 335.29 seconds (62.49
1 - start: 853 end: 7626

C:\Program Files\Frey Technologies\SageTV>
and it did work, so I guess this function was not implemented on a SageTV client box but on the server only.

Love this tool...

[Opus4]

Originally posted by chrysek
I am trying it on a client system, not on sage server, maybe that is the problem?

Please note the 2nd sentence of malore's original post, which says:

Originally posted by malore
It also won't work,as is, on the SageTV Client because of file access issues

Edit: See malore's comments, below. Maybe I should remove this post now that malore has said how to use it on the client... but then the next post would seem strange.

- Andy

[chrysek]

Ah, sorry about that, I did miss that statement But I love this feature. It is the coolest thing..

[malore]

You can make playback on the client work, if the client has a copy of the .txt file in the same location on the client machines as it appears on the server. So if on the server the file for the show being played is C:/videos/show-0.mpg and you've already generated the C:/videos/show-0.txt commercial file. Just copy all the .txt files to C:/videos on the client machine. If your server stores everying on a drive other than C:, you might be able to just setup a mapped drive with the same drive letter and have everything work without any additional effort.

[insomniac]

Originally posted by malore
You can make playback on the client work, if the client has a copy of the .txt file in the same location on the client machines as it appears on the server. So if on the server the file for the show being played is C:/videos/show-0.mpg and you've already generated the C:/videos/show-0.txt commercial file. Just copy all the .txt files to C:/videos on the client machine. If your server stores everying on a drive other than C:, you might be able to just setup a mapped drive with the same drive letter and have everything work without any additional effort.

When I setup my Video directories, I configured them by unc (even on the server). e.g.

drive 1 = \\server\sage
drive 2 = \\server\sage1
drive 2 = \\server\sage2

Wouldnt this work as a permanent solution? Or do you think this is broke and will require copying the txt files over still?

I would just install it and try it tonight, but my WAF is going down from being on the PC too much

I.

[malore]

insomniac, that sounds like that would work to me. I don't have the SageTV client, but I can run comskip.exe using a unc path without any problems.

[Deadbolt]

Malore, got it working!

Beautiful! Thanks!

[Jere_Jones]

Very impressive. So far I've tried it on a variety of shows and when it works, it works great! It seems there is no middle ground. IOW, it either does a great job or a crappy job depending on the show. BTW, it works much more often than not.

I think that signal quality makes a large difference too. I recorded a couple shows that were very noisy and comskip didn't write a single frame number. But that was to be expected because there were NO black frames. There were noisy black frames, but not entirely black.

I think I have found a bug though. If you start to watch the show before the commercial detection is complete then the marked commercials don't show up and you can't skip using custom4 or 5.

Overall this is a GREAT JOB!!

Question: When custom4 says jump to the next non-commercial, is it jumping to a frame number or a time derived from the frame number?

Jere

[ToxMox]

Malore,
Great work!
So the background commercial processing only goes when sage is sleeping? I never sleep my Sage. How about if the Sage screensaver kicks in and there isn't anything scheduled in the next 15 minutes that it also starts background commercial detection.
So far I've only tested it on World Series of Poker and it works great. I'm waiting for it to process some more files.

[kny3twalker]

Hey can we get an icon for when a show has been commercial skipped so we know that it has been done or after it has been started

really enjoying this feature by the way


also since it has been a while since I have used your STV
when I choose preview window rather than video background
I get video on top of the menu screens where there is no place for it
it sometimes overlaps menu titles and the blue bar dissappears behind it

and is there a reason that you made it so we can only use this option with the one view of the recording?

[malore]

Originally posted by Jere_Jones
I think that signal quality makes a large difference too. I recorded a couple shows that were very noisy and comskip didn't write a single frame number. But that was to be expected because there were NO black frames. There were noisy black frames, but not entirely black.

See if the attached version works better, run it manually on the problem file.

Originally posted by Jere_Jones
I think I have found a bug though. If you start to watch the show before the commercial detection is complete then the marked commercials don't show up and you can't skip using custom4 or 5.

That's no bug, that's a feature. It only reads the commercial skip file when the show is first loaded, so to get it to load the .txt file again, you would have to switch to another show and back again.

Originally posted by Jere_Jones
Question: When custom4 says jump to the next non-commercial, is it jumping to a frame number or a time derived from the frame number?

It jumps to a time derived from the frame number and SageTV's times don't seem to be 100% accurate, so my commercial skipping looks less accurate.

Edit:
Removed the attachment, because it is no longer needed.