Starting to test Comskip with h.264 support

[erik]

Did you read the readme.txt file inside the zip file?
Did you copy the two DLL files to the same directory as the comskip executables?

[erik]

There seems to be an error in this release.
I removed it from the server

[Chaos~Theory]

Hey erik,

I'm still having problems..... sent you another detail \ed email.

thanks!

[erik]

Could you download and install from http://www.microsoft.com/downloads/d...displaylang=en
the vcredist_x86.exe package and test again?

[erik]

The copy of ff_libfaad2.dll I included in the comskip distribution fails on certain systems.
As a work around you can copy ff_libfaad2.dll from a ffdshow installation.
And I will upload a new release later today

[Chaos~Theory]

Originally Posted by erik

The copy of ff_libfaad2.dll I included in the comskip distribution fails on certain systems.
As a work around you can copy ff_libfaad2.dll from a ffdshow installation.
And I will upload a new release later today

All thanks to me! HA!
I'm just mess'n with ya..... I'm glad I could help!

You are doing great things for us. I'm sure I speak for a lot here when I say... Thank you for all you hard work!
And for FREE, I might add! For y'all that haven't donated to Erik for using comskip.... shame on you!

Keep up the fantastic work!

-Chris

[bialio]

Gave version 80-2 a whirl last night. It seems like it takes a whopping 4 hours to process a 1 hour show from my HD-PVR!

However, it seems to have detected commercials a little better than the most recent SA.

At this point a release of SA that's a few releases old has the best commercial detection on H264 source.

I set the two Sage flags in comskip.ini = 1 - something I've always done when running comskip with Sage. Do you think those hurt my results on H264?

btl.

[larryf]

Tried 80-1 (didn't crash for me) and 80-2 on a football game... Picked up the commercials well, but as noted above took a long time to complete. I didn't tweek anything, just tried it as released.

A side note, I am recording the same shows on SD (on a separate tuner) alongside my HD-PVR until I am satisfied the fan will prevent lockups. The EDL file on the SD recording looks almost identical to the HD recording. There is maybe a 2 second lag that I attribute to the HD STB. This leads me to believe I can simply rename the EDL file to work with the HD recording and change the padding a bit.

I have a quad core CPU so I will probably stick with Erik and Jere's latest releases Anyone with a single core or slow CPU may want to consider recording SD/HD in tandem, running comskip/SA on the SD and renaming the EDL file as a short term fix. The SD recording takes very little space and you can always delete it afterwards.

Or.... another thought... maybe use a TS to MPG converter to create the file you will run comskip on, but I have no idea how much CPU is required to do the conversion.

Larry

[Eckwell]

Larryf -

What process do you have in place to get the HD and the SD recordings simultaneously?

Just curious?

[larryf]

Originally Posted by Eckwell

Larryf -

What process do you have in place to get the HD and the SD recordings simultaneously?

Just curious?

I have 5 tuners including the HD-PVR with a HD STB... In this case, I taped the SD on local channel 2 and the HD on local channel 102 (basically forcing the recording through the guide). I only do this on recordings I MUST have, like a football game. Once the HD-PVR is recording without a chance of a lockup, I'll skip the SD 'backup'.

In my setup, the SD tuner cannot see the HD channels, and the HD tuner cannot select the SD channels.

There is a lag, but I think you can circumvent it with padding.

larry

[Sam]

Originally Posted by larryf

A side note, I am recording the same shows on SD (on a separate tuner) alongside my HD-PVR ...believe I can simply rename the EDL file to work with the HD recording and change the padding a bit.

Out of curiosity, if you right-click and select properties for both the *.ts and *.mpg files, are the create timestamps also apart by about 2 seconds? If so, that may be one way for an automated process to attempt to figure out how much padding needs to change in the EDL.

One limitation of the method you used here is it requires both the HD and SD versions of the show to be available on different channels. As an alternative, here is a method I've thought about researching, but I haven't had time to mess with Studio yet to see how easy it would be.

My idea was to remove one of my analog tuners from the sage video source config, so that sage does not try to use it. The run the s-vid output on the STB which feeds the HD-PVR, and have it feed the s-vid input on the orphaned analog card.

In Studio, the concept would be to watch for recordings which will be using the HD-PVR. As close as possible to the exact time the HD-PVR would begin recording, spawn a process using ffmpeg or equivalent which captures the s-vid signal and writes to a file with the same root but mg extension. It probably only needs to capture it in 320x240, but since there is no transcoding going on and the file can be temporary, a larger frame size should be fine too. Ideally to avoid disk fragmentation, you'd write this temp file to another drive. The process could also kick off SA or Comskip naming the source path to the .mpg and outputting the EDL to the drive with the *.ts file.

I think this would allow SA or Comskip (SD versions) to provide real-time commercial detection with very little CPU overhead on the server, even when recording a premium channel not available in both HD and SD concurrently.

I just haven't used Studio yet, so don't know how hard it would be to setup such a STVi.

Or.... another thought... maybe use a TS to MPG converter to create the file you will run comskip on, but I have no idea how much CPU is required to do the conversion.

Using DVRMSToolbox 1.2.1.4 against a *.ts recorded at 13.5Mbps, its predefined "Convert a file for commercial scanning" took about 18 minutes and 45 seconds on my quad-core. This is to produce a 320x240 output of about 280MB vs the 6GB input file.

Ironically, using DVRMSToolbox to create a 720x480 file in 16/9 aspect ratio at DVD quality 6.3Mbps took only slightly longer, at about 20 minutes.

In both cases, the CPU gets pegged at over 95%, because I have DVRMSToolbox set to use multiple threads and Comskip was not running at the same time. So far, I haven't figured out how to get DVRMSToolbox to perform the mpg conversion while the show is still airing. So this method means the conversion and commercial detection has to wait until the show is over. But then it doesn't take hours -- just 20 mins for the conversion then the normal SD detection time.

But I still think the really low CPU overhead way to do this in real-time is to concurrently record the show in SD (using the STB s-vid out instead of a separate airing recorded by Sage).

I suppose another way of doing this instead of Studio is just to use something like DirMon2 and watch for *.ts files to be created. It would then have to immediately fire off a s-vid capture to the same filename with a *.mpg extension. But using this method would probably lead to a nominal time lag which could be reduced using a STVi monitor.

[larryf]

Originally Posted by Sam

Out of curiosity, if you right-click and select properties for both the *.ts and *.mpg files, are the create timestamps also apart by about 2 seconds? If so, that may be one way for an automated process to attempt to figure out how much padding needs to change in the EDL.

One limitation of the method you used here is it requires both the HD and SD versions of the show to be available on different channels.

I have deleted the SD version of the game so I can't tell you the time it was created... But I just kicked off another SD/HD show in the next hour and will try to post back here later.

I do recall comparing the EDL files, and each entry seemed to be off by between 9 and 11. (ie, if file 1 was 123.5, file 2 was 131.6). I really think the lag is in the cable companies HD STB.

On my cable system, I can't think of a single instance where I could not record both HD and SD at the same time. Well, perhaps HBO, but no commercials there .

Larry

[larryf]

Originally Posted by larryf

... But I just kicked off another SD/HD show in the next hour and will try to post back here later.

I do recall comparing the EDL files, and each entry seemed to be off by between 9 and 11. (ie, if file 1 was 123.5, file 2 was 131.6). I really think the lag is in the cable companies HD STB.

The .ts and .mpg files are created within a few 100ths of a second of each other (and I couldn't see that in Vista X64, it does not show seconds in the timestamp! I had to load a 3rd party app to look at the timestamps)

I also found, while turning on 3 tvs in the house, QAM and HD are in sync, but both lag analog cable for several seconds.

Your idea of using an svideo cable out of the STB looks like an easier method... Have you tried it? My only other possibility is to set up one of the HDHR tuners to record QAM in low resolution (near SD).

I need to dig up an svideo cable in my junk box.

[thomaszoo]

Erik

This totally rocks!

I am running this on a single core AMD 4000+ and it takes about 4.5 hrs for a 1 hr show. I was expecting much worse performance. Accuracy is off just a little on the few shows that I have recorded so far. But for me this is usable at this time. I have no doubt that things will improve with time but at least this gets me 95% of the way there (accuracy wise).

Thanks again.

Wayne

[Sam]

Originally Posted by larryf

I also found, while turning on 3 tvs in the house, QAM and HD are in sync, but both lag analog cable for several seconds.

You can get the SD channels in clear QAM too? I can't with my local Comcast.

Originally Posted by larryf

Your idea of using an svideo cable out of the STB looks like an easier method... Have you tried it? My only other possibility is to set up one of the HDHR tuners to record QAM in low resolution (near SD).

Again, I don't think most of us have the option of getting the clear QAM of the same show. But the STB s-video will inherently always be in sync with the component output. So if the recordings start at very nearly the same time, the EDL should be good for both the *.mpg and *.ts versions.

Alas, no I haven't tried it. This is just theory. When I get time, a may attempt a little utility to try and automate this. Or else dive into the Studio docs and do it in a STVi.

[Sam]

Automating a concurrent capture of s-vid to a *.mpg when a HD-PVR recording start is not turning out to be as easy as I guessed.

I knew ffmpeg and mencoder could do video grabs and write to a file, so I thought I'd go that route while monitoring folders for new *.ts files. But it appears the video grab support in them is limited to Linux and doesn't work in Windows. So I looked for other ways to capture the video using .NET or whatever, but those APIs don't look like they give me the stream in mpeg-2 and I am trying to avoid another transcode here.

So I looked at the online API docs for Studio. I see how to enumerate the video source and manipulate properties. But I don't see how to just start or schedule a manual recording let alone how to get it to use the video source I want, and the filename I want.

So maybe your idea of recording a SD channel airing at the same time is more practical.

[sundansx]

I had the new comskip .80.002 run on a SCIFIHD recording of Eureka last night and it did indeed take a while (looks like 3 hours or so) on a Core2duo 6420.
What is strange is that it did not produce a .edl file. I have included a my comskip log for investigation.
In addition, it seemed to find lots of black frames and cut scenes, but in the end, it did not find much of a commercial list. I passed it my comskip.ini. I am using the one that came with comskip, but I changed max_volume to 0 as indicated in the release history.
I see in the log that output_edl=1, is there something I am missing in the .ini?
I will run this again tonight and turn on every "output" command with edl contained in it.
I am using this commandline from dirmon2:
program:

C:\Program Files\Comskip\comskip.exe

targets:

ts

command line options:

--ini="C:\Program Files\comskip\comskip.ini" --ts

which should cat together as:

C:\Program Files\Comskip\comskip.exe --ini="C:\Program Files\comskip\comskip.ini" --ts "filename"

[erik]

Could you set in comskip.ini
output_framearray=1
then drag and drop the file on ComskipGUI.exe to visually verify the video decoder is not crashing somewhere during the show?
Send me the generated .csv file so I can check if its just weird broadcasting style or something worse

[sundansx]

Erik,
Ok I started this off with output_framearray=1 and a drag and drop on the gui. It appears that it is not crashing right away and it is displaying decoded video frames in the gui with all sorts of graphs streaming as well. I can't sit and watch this thing for 3 hours...did you just want me to check if it crashes right away?
Also, do I need to specify the "--ts" or "-t" parameter for these files? It seems to run on them with the same results with or without this...
thanks.

[erik]

Comskip will automatically recognize ts recordings and select the main video.
SO if you see the video there is no need to specify the --ts or -t parameter

It would be nice if you could take a look at the video when its close (30 minutes ) to finishing. In that way you can verify there was no crash of the decoder.
In any case let is finish and send me the .csv file.