Starting to test Comskip with h.264 support

[clip]

Sometimes, it works, and some times I get a popup...

Runtime Error! Program: C:\comskip\comskip.exe

The application has requested the Runtime to terminate in an unusual way. Please contact the applications' support team for more information.

[erik]

Sometimes meaning?
Sometimes on the same recording
Sometimes on the same recording source
Sometimes when scanning all kind of h.264 files?
Sometimes on various systems?

[clip]

Originally Posted by erik

Sometimes meaning?
Sometimes on the same recording
Sometimes on the same recording source
Sometimes when scanning all kind of h.264 files?
Sometimes on various systems?

I'm chugging through 100+ recordings on 3 different machines. All of the recordings are recorded off the same HD-PVR (multiple channels/qualities). I haven't tried running the same file again. I'll see if I can reproduce with the same file on a separate system next time one fails.

UPDATE: Okay, I got one. Dora the Explorer. Same File, Two different machines.

[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]error while decoding MB 0 0, bytestream (td)
[h264 @ 003C29D0]mmco: unref short failurefps), 1.01 sec (8.91 fps), 1%
[h264 @ 003C29D0]mmco: unref short failure
[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]mmco: unref short failure
[h264 @ 003C29D0]mmco: unref short failure
[h264 @ 003C29D0]error while decoding MB 0 0, bytestream (td)15 fps), 1%
[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]warning: first frame is no keyframe
[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]error while decoding MB 0 0, bytestream (td)
[h264 @ 003C29D0]mmco: unref short failure
[h264 @ 003C29D0]mmco: unref short failure
[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]mmco: unref short failure
[h264 @ 003C29D0]mmco: unref short failure
[h264 @ 003C29D0]error while decoding MB 0 0, bytestream (td)00 fps), 1%
[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]Internal error, picture buffer overflow

[killervette]

Originally Posted by erik

On a quad core you should run 4 in parallel to fully load the CPU.

yeah, but i am also using it to stream tv and blu ray movies, so i dont want it fully slammed

[erik]

Originally Posted by clip

[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]error while decoding MB 0 0, bytestream (td)
[h264 @ 003C29D0]mmco: unref short failurefps), 1.01 sec (8.91 fps), 1%
[h264 @ 003C29D0]mmco: unref short failure
[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]mmco: unref short failure
[h264 @ 003C29D0]mmco: unref short failure
[h264 @ 003C29D0]error while decoding MB 0 0, bytestream (td)15 fps), 1%
[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]warning: first frame is no keyframe
[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]error while decoding MB 0 0, bytestream (td)
[h264 @ 003C29D0]mmco: unref short failure
[h264 @ 003C29D0]mmco: unref short failure
[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]mmco: unref short failure
[h264 @ 003C29D0]mmco: unref short failure
[h264 @ 003C29D0]error while decoding MB 0 0, bytestream (td)00 fps), 1%
[h264 @ 003C29D0]illegal short term buffer state detected
[h264 @ 003C29D0]Internal error, picture buffer overflow

These errors imply that the h264 decoder is reading corrupt data (or has a wrong implementation)
It looks like its at the start of the recording (correct?)

[rtengvad]

Originally Posted by erik

On a quad core you should run 4 in parallel to fully load the CPU.

And on a duo core you should run two in parallel to fully load?

Rasmus

[clip]

Originally Posted by erik

These errors imply that the h264 decoder is reading corrupt data (or has a wrong implementation)
It looks like its at the start of the recording (correct?)

Correct, it was at the beginning of processing that file. I think I get errors through out the file. Although, it all seems to play just fine on the STX-HD100

[erik]

Originally Posted by rtengvad

And on a duo core you should run two in parallel to fully load?

Yes, two or more

[rtengvad]

Thanks!

Rasmus

[Beefcake550]

I want to add in on how many processes of comskip to run at a time. I found that on my Quad core Q9450, one process took up about 45-55% of all 4 cores.

Given that this machine is a SageTV server and file server to the other PCs in the house, I wanted to leave at least 1 core for anything else. More importantly, disk bandwidth was going to become an issue.

So, using Dirmon2, I made two jobs. One fore comskip and mpgs. The other for comskip and TS files. I made the mpg related job only need 40 "resources" and the ts related job needed 60 "resources".

This will allow me to have only 1 ts file being processed while another mpg file is processed, or 2 mpg files could be processed.

I know my system could handle more as I've run 30 comskip threads at a time (all mpg files) and the disk system was the limiting factor. It looked like each mpg stream needed 10MB/s while the TS streams were less than half that (since they are CPU limited).

Anyhow, that's the scoop. I hope this helps anyone else and I also invite constructive critisism. I am after what everyone else is.....the "perfect" server.

-Brian

[killervette]

Originally Posted by Beefcake550

I want to add in on how many processes of comskip to run at a time. I found that on my Quad core Q9450, one process took up about 45-55% of all 4 cores.

Given that this machine is a SageTV server and file server to the other PCs in the house, I wanted to leave at least 1 core for anything else. More importantly, disk bandwidth was going to become an issue.

So, using Dirmon2, I made two jobs. One fore comskip and mpgs. The other for comskip and TS files. I made the mpg related job only need 40 "resources" and the ts related job needed 60 "resources".

This will allow me to have only 1 ts file being processed while another mpg file is processed, or 2 mpg files could be processed.

I know my system could handle more as I've run 30 comskip threads at a time (all mpg files) and the disk system was the limiting factor. It looked like each mpg stream needed 10MB/s while the TS streams were less than half that (since they are CPU limited).

Anyhow, that's the scoop. I hope this helps anyone else and I also invite constructive critisism. I am after what everyone else is.....the "perfect" server.

-Brian

i am running 3 processes on a quad core q6600 (all hd pvr) and watching live tv off the extender and the cpu is at 65-75%. i am surprised one stream is that high on yours.

[Beefcake550]

I paid closer attention to the current running HD-PVR comskip. It's moving between 37% and 48%. I'm also currentyl streaming a Scooby Doo DVD to the client, but that shouldn't take much at all.

We have very similar setups. I have a Q9450 running WHS with 4GB RAM.

I'm going to be checking out a few things this evening. More as it comes.

-Brian

[Eckwell]

Is everyone / Anyone also allowing Comskip to extract cc on mpgs? Just thinnking that two jobs would be good in that case were the Mpg.bat job could handle comskip with an ini set for cc, and the ts.bat could handle without cc...

question is... does the cc extraction process take enough bandwidth in the .ts senario to car about? if so, it should be turned off in the .ini as there is no cc data in the H.264 from the HD-PCR.

Are my assumptions correct?

[erik]

Compare to decoding h.264 there is nothing to care about
When decoding mpeg2 I have not found decoding of cc to have any speed impact.
When there is no cc data its even simpler. switching on or off will make no difference at all.

[Skiier__Dude]

Bravo Erik...got the new Comskip running using Comskip Monitor. My server is pretty modest -- an AMD 3200+. CPU runs at almost 100% when it's processing, but with Comskip monitor, I have scheduled it to not run during prime time. It scanned about 30-ish shows in about 2 days and works flawlessly out of the box.

I am now in TV-nirvana. Watching HD, commercial free.

[Beefcake550]

I just wanted to add taht I'm seeing the same type of errors that Clip is seeing and reported to Erik.

Not that it matters, but it looks like gettign comemricals out of children's shows here in the US is very difficult. The format of the show is much different and sometimes I can't tell what is the show and what is not. Not that it matters.....the kids' shows are not what "I" watch

[Eckwell]

Erik,

question... i have been using ccextractor for a while, and now I am trying to use your all-in-one comskip (802) and have it extract the .smi file. when it does, it puts an _1 at the end of the file. ccextractor had a command line option to stop that, is there a way to do that in comskip?

thanks...

UPDATE: I can't get comskip to finish without crashing when cc is enabled output_smi=1. I am still troubleshooting, but I have upgraded all the dlls, etc. and wondering if others have hadan issue like this related to ccextractor code in comskip?

[hemicuda]

yep, thought I either posted or PM'd about this a few weeks ago. had to turn off CC until a fix is released.

[Eckwell]

Originally Posted by hemicuda

yep, thought I either posted or PM'd about this a few weeks ago. had to turn off CC until a fix is released.

Thanks... Anyone else having CC problems? I really want to get this to work as ccextractor, when running, causes my network to hiccup so I get pauses when watching recordings through the HD extender. I can literally start and stop ccextrcator and hear my wife yelling at my down the stairs "It's doing that thing again!"... comskip doesn't cause that. I WAS able to run it against a family guy recording of of HDHR and it worked great. although I had to rename the file to get ride of teh _1

[erik]

I already found a robustness problem in the CC decoder. New build will be available soon. Hope I can also solve the _1 problem