Comskip & SageMC ?

[raoul]

I am trying to get comskip functioning properly. I have it sort of working, but it wasn't even close to marking the commercials on the few recordings I had saved. It wasn't just off a few seconds here and there, but missed entire commercials and cut a few minutes into the program. It is like it got the block of commercials, but offset the block by a couple of minutes in the recording.

All I have done so far is install Comskipmonitor and the lastest version of comskip. I am using SageMC, so I did not install the comskip playback as it appeared that it had it built in (???) and the playback stated it was for the standard STV.

What are my next steps in getting this working at an acceptable level? I know that it can't always be perfect, but it was really bad on all the recordings, so it makes me wonder if I missed a step? I see there is a comskip varient on the "available customizations" page, which I assume is a tweaked .ini file for comskip. Do I need to install this? I figured it would be included in the latest and greatest comskip. Do I need to install a playback even though I am using SageMC?

[MeInMaui]

Hi,

The only thing I can suggest is to make sure you generate .edl files with comskip instead of .txt files. They are much more accurate. (.edl files are time based and .txt files are frame based, which requires a conversion) SageMC will automatically recognize and use either (.edl files are chosen if both exist).

Aloha,
Mike

[mtyme]

I did a fresh install with everything you mentioned above and have the same issue. I'm thinking maybe the new comskip.ini file included with the latest build of comskip is wacky. I'm gonna try a buddies comskip.ini to see if that helps.

[raoul]

Ok... I will see if I can figure out how to generate .edl files. However, shouldn't that be a correction on the order of seconds (or less) and not minutes?

In the readme of the latest comskip, it mentions that he set it up to be more agressive and to rollback to the .ini included with version 10#. The comskip website is currently down, so perhaps someone can post here a good working .ini file? That would be very helpful.

thanks

-R

[mickp]

I recall having problems like you describe with .txt files. Changing over to .edl fixed the problem for me. I'd guess that there's something not quite right with the support for .txt in sagemc.

Here's my current ini file for comskip. I don't think I ever got around to fiddling with the settings in this incarnation. It's been working okay for me - YMMV.

detect_method=43			;1=black frame, 2=logo, 4=scene change, 8=fuzzy logic, 16=closed captions, 32=aspect ration, 64=silence, 255=all
verbose=10				;show a lot of extra info, level 5 is also OK, set to 0 to disable
max_brightness=60      			;frame not black if any pixels checked are greater than this (scale 0 to 255)
test_brightness=40      		;frame not pure black if any pixels checked are greater than this, will check average brightness (scale 0 to 255)
max_avg_brightness=25			;maximum average brightness for a dim frame to be considered black (scale 0 to 255) 0 means autosetting
max_commercialbreak=600 		;maximum length in seconds to consider a segment a commercial break
min_commercialbreak=25			;minimum length in seconds to consider a segment a commercial break
max_commercial_size=120			;maximum time in seconds for a single commercial or multiple commercials if no breaks in between
min_commercial_size=4   		;mimimum time in seconds for a single commercial
min_show_segment_length=120 	; any segment longer than this will be scored towards show.
non_uniformity=500
max_volume=500				; any frame with sound volume larger than this will not be regarded as black frame
min_silence=20				; Any deep silence longer than this amount  of frames is a possible cutpoint
ticker_tape=0				; Amount of pixels from bottom to ignore in all processing 
punish=0					; Compare to average for sum of 1=brightness, 2=uniform 4=volume, 8=silence, 16=schange, set to 0 to disable
punish_threshold=1.3		; Multiply when amount is above average * punish_threshold
punish_modifier=2			; When above average * threshold multiply score by this value
intelligent_brightness=0 		; Set to 1 to use a USA specific algorithm to tune some of the settings, not adviced outside the USA
logo_percentile=0.92			; if more then this amount of logo is found then logo detection will be disabled
logo_threshold=0.80
aggressive_logo_rejection=0
connect_blocks_with_logo=1		; set to 1 if you want successive blocks with logo on the transition to be regarded as connected, set to 0 to disable
cut_on_ar_change=0			; set to 1 if you want to cut also on aspect ratio changes, set to 0 to disable
delete_show_after_last_commercial=1	; set to 1 if you want to delete the last block if its a show and after a commercial
delete_show_before_or_after_current=1	; set to 1 if you want to delete the previous and the next show in the recording, this can lead to the deletion of trailers of next show
shrink_logo=5				; Reduce the duration of the logo with this amount of seconds
after_logo=0		; set to number of seconds after logo disappears comskip should start to search for silence to insert an additional cutpoint
padding=0
ms_audio_delay=5
delete_logo_file=1			; set to 1 if you want comskip to tidy up after finishing
output_framearray=0			; create a big excel file for detailed analysis, set to 0 to disable
output_videoredo=0
output_womble=0
output_cuttermaran=0
output_mpeg2schnitt=0
output_mpgtx=0
output_zoomplayer_chapter=0
output_zoomplayer_cutlist=0
output_edl=1
output_txt=0
output_vcf=0
output_btv=0				; set to 1 if you want Beyond TV chapter cutlist output
output_projectx=0			; set to 1 if you want ProjectX cutlist output (Xcl)
output_demux=0				; set to 1 if you want comskip to demux the mpeg file while scanning
live_tv=0					; set to 1 if you use parallelprocessing and need the output while recording
output_debugwindow=0
cuttermaran_options="cut=\"true\" unattended=\"true\" muxResult=\"false\" snapToCutPoints=\"true\" closeApp=\"true\""
mpeg2schnitt_options="mpeg2schnitt.exe /S /E /R25  /Z %2 %1"

Mick.

[JREkiwi]

Originally Posted by mickp

I recall having problems like you describe with .txt files. Changing over to .edl fixed the problem for me. I'd guess that there's something not quite right with the support for .txt in sagemc.

.edl files are more accurate, but if you are using .txt files you need to make sure you have the frame rate NTSC/PAL set correctly.

To get comkip to output .edl files just set
output_edl=1

in the comskip.ini

John

[raoul]

Just to follow up on this, I switched over to EDL and all of my problems disappeared and comskip became 100% accurate on the same recordings. Again, when people said EDL was more accurate, I was thinking on the level of a second or so... not minutes. It is probably due to the frame rate being slightly off as indicated. Anyway, it now works!

The only glitch I see is that sometimes when it autoskips the commercial block, I lose all audio. I have video, but no audio until I fast forward or rewind a step and then it comes back. Any guesses on that one? It only happens maybe 50% of the times... like it loses sync or something.

-R

[QueOnda]

Ok. I have sageMC stv install and set to comskip automatically but nothing happens. DO I need to install COMSKIP program separately install???

Having issues in trying to get different fwd/rwd (currently 10sec but changed to 30/7) but still does 10.

[MeInMaui]

Originally Posted by QueOnda

Ok. I have sageMC stv install and set to comskip automatically but nothing happens. DO I need to install COMSKIP program separately install???

Having issues in trying to get different fwd/rwd (currently 10sec but changed to 30/7) but still does 10.

Yes, you have to set up comskip (or Show Anayzer) to process the files separately. Once either of these programs has generated the .edl (this is the preferred format) file for a recording, then SageMC will use it automatically. You can use Dirmon2 or comskip monitor to set up automatic generation of the .edl files.

The only thing I can suggest about your other question is make sure you set the times on the correct set of ff/rw buttons. HTH

Aloha,
Mike

[QueOnda]

OK. installed comskip monitor 2.7. I have usb drives (g). Please look at my process log below. I seems to be trying the same files in a loop over and over, not progressing to the next files. Any idea? Or should I do a network drive format instead (even though same computer)?

1-23-2008 16:47 Processing file: G:\Amar sin Límites-2007-11-26-0.avi
1-23-2008 16:47 Processing file: G:\Destilando Amor-2007-11-29-0.avi
1-23-2008 16:47 Processing file: G:\Inside Area 51-2007-10-12-0.avi
1-23-2008 16:47 Processing file: G:\Most Shocking-2008-01-09-0.avi
1-23-2008 16:47 Processing file: G:\The Simpsons-(I Don't Wanna Know Why the Caged Bird Sings)-2007-10-14-0.avi
1-23-2008 16:47 Processing file: G:\Yo Amo a Juan Querendón-2007-11-28-0.avi
1-23-2008 16:58 Processing file: G:\Amar sin Límites-2007-11-26-0.avi
1-23-2008 16:58 Processing file: G:\Destilando Amor-2007-11-29-0.avi
1-23-2008 16:58 Processing file: G:\Inside Area 51-2007-10-12-0.avi
1-23-2008 16:58 Processing file: G:\Most Shocking-2008-01-09-0.avi
1-23-2008 16:58 Processing file: G:\The Simpsons-(I Don't Wanna Know Why the Caged Bird Sings)-2007-10-14-0.avi
1-23-2008 16:58 Processing file: G:\Yo Amo a Juan Querendón-2007-11-28-0.avi
1-23-2008 17:09 Processing file: G:\Amar sin Límites-2007-11-26-0.avi
1-23-2008 17:09 Processing file: G:\Destilando Amor-2007-11-29-0.avi
1-23-2008 17:09 Processing file: G:\Inside Area 51-2007-10-12-0.avi
1-23-2008 17:09 Processing file: G:\Most Shocking-2008-01-09-0.avi
1-23-2008 17:09 Processing file: G:\The Simpsons-(I Don't Wanna Know Why the Caged Bird Sings)-2007-10-14-0.avi
1-23-2008 17:09 Processing file: G:\Yo Amo a Juan Querendón-2007-11-28-0.avi
1-23-2008 17:20 Processing file: G:\Amar sin Límites-2007-11-26-0.avi
1-23-2008 17:20 Processing file: G:\Destilando Amor-2007-11-29-0.avi
1-23-2008 17:20 Processing file: G:\Inside Area 51-2007-10-12-0.avi
1-23-2008 17:20 Processing file: G:\Most Shocking-2008-01-09-0.avi
1-23-2008 17:20 Processing file: G:\The Simpsons-(I Don't Wanna Know Why the Caged Bird Sings)-2007-10-14-0.avi
1-23-2008 17:20 Processing file: G:\Yo Amo a Juan Querendón-2007-11-28-0.avi
1-23-2008 17:31 Processing file: G:\Amar sin Límites-2007-11-26-0.avi
1-23-2008 17:31 Processing file: G:\Destilando Amor-2007-11-29-0.avi
1-23-2008 17:31 Processing file: G:\Inside Area 51-2007-10-12-0.avi
1-23-2008 17:31 Processing file: G:\Most Shocking-2008-01-09-0.avi
1-23-2008 17:31 Processing file: G:\The Simpsons-(I Don't Wanna Know Why the Caged Bird Sings)-2007-10-14-0.avi
1-23-2008 17:31 Processing file: G:\Yo Amo a Juan Querendón-2007-11-28-0.avi

EDIT (MORE INFORMATION) : Ok, Seems all are AVI files. Does that mean Comskip cannot do AVIs?

[ke6guj]

Originally Posted by QueOnda

EDIT (MORE INFORMATION) : Ok, Seems all are AVI files. Does that mean Comskip cannot do AVIs?

Yup, comskip (and showanalyzer) only work on mpeg files. SO, if you want to convert to avi, make sure you have already marked the commercials before converting.

That is one of the upcoming concerns about the new HD capture device that hauppauge (and Sage) are talking about. They capture in mpeg4, so no comskip.

[QueOnda]

Thanks ke6guj for the clarification. I think for mpg4 they would have to convert to mpg2 to do the comskip and back to mpg4. That will be nasty.

I finally got this comskip to work. Well, it's bangin away on a folder now. I had to actually add the actual folder (instead of G\: I had to put g\:\\folder).

Now, If I have it enabled to do when a show is being recorded how does sagemce tell comskip that is has started a new file? Would I see it in the log in the data folder or the postprocessing log in the comskip79_76 folder?

[ke6guj]

Originally Posted by QueOnda

Now, If I have it enabled to do when a show is being recorded how does sagemce tell comskip that is has started a new file? Would I see it in the log in the data folder or the postprocessing log in the comskip79_76 folder?

You set5 up comskip monitor to monitor your recording folder and it will trigger comskip to start scanning the new file when it sees one. Sage itself has nothing to do that.

[QueOnda]

Originally Posted by ke6guj

You set5 up comskip monitor to monitor your recording folder and it will trigger comskip to start scanning the new file when it sees one. Sage itself has nothing to do that.

Then all sage does is read the .edl file if associated with the recording (if enabled) and in middle of recording (enabled in both SAGEMC and comskip)?

[ke6guj]

You got it, If you are using SageMC, or the Default UI with comskip plugin, and it sees an .edl file, it will use it to skip commercials. The .edl file will be periodically re-read to see if there is new comskip data in it. That is why you can even do comskipping on "delayed" LiveTV when comskip udates the .edl file as it finds new commercials.

[QueOnda]

Ok, the show which have a comskip edl and txt and log file, I try to watch on the client and don't see commercial markers. Is there also an icon (like the watched W icon) which tell you which shows have commercials?

Under the sagemc options I have Background commercial processing, while awake, while recording and commercial auto skip enabled on the client. Start and stop delay 0 and broadcast type 29.97, since Ntsc.




UPDATE: (see below)

Looking at the sage.properties on the server side is this:

background_commercial_processing=false
commercial_auto_skip=false

This may be the problem. I will shutdown sage and change the properties and startup when I get a chance. Why didn't the server sage properties change when I changed then on the client side? I guess I don't understand the client/server relationship.

[ke6guj]

Originally Posted by QueOnda

Ok, the show which have a comskip edl and txt and log file, I try to watch on the client and don't see commercial markers. Is there also an icon (like the watched W icon) which tell you which shows have commercials?

OK, if you try to watch on the server, you do see the commercial markers? And you've confirmed that the comskip files are being created. I'm not sure, but I'd probably set up comskip to create either an .edl or .txt file, not both.

yes, there should be a CS icon if there are comskip files.

The reason that the client isn't comskipping is that it probably doesn't have direct access to the .edl file. Sage is streaming the video file to the client, but doesn't stream the .edl file. For that, you need the client to be able to see the .edl file itself. Right now, the server says that the show was recorded on the server to E:\SageTVfiles\tvshow.mpg. The client looks for the .edl file in E:\SageTVfiles. If that path doesn't match on the client, comskip won't work on the cleint. There are two ways to fix that, one is to take the drive letter that the server records to and map it on the client with the same drive letter. The second way is to have the server record via UNC paths. In that case on the server, you'd share the SageTVfiles folder and then tell sage to record to \\Server\SageTVFiles. Now, for any NEW recordings, the file path that sage will report is \\Server\SageTVfiles\tvshow.mpg. With the .edl file in the same folder, the client won't have any problems comskipping.

Under the sagemc options I have Background commercial processing, while awake, while recording and commercial auto skip enabled on the client. Start and stop delay 0 and broadcast type 29.97, since Ntsc.

Disable "Background Commercial Processing" in SageMC. That is outdated code that you don't need to use since you are already using comskip monitor. Just leave "Commercial Auto Skip" enabled for SageMC to use the existing comskip files.

[QueOnda]

Thanks ke6guj for you help. Now got it working. I decided to map the drives to the client computer since they were G, H, I, J drive letter designation.