Comskip for Sage Linux..... :)

[laurenglenn]

First, get a copy of comskip.exe (I used version 0.64) and put the files in /var/media/tv

emerge wine
cd /var/media/tv

----------------------------------------

Here is a copy of the bash prompt (keep in mind the exception error comes *after* it discovers the logo for commercial detection). Just run comskip on the file again and it will pick up the logo file.

STVMC-883 tv # wine comskip TheSimpsons-HOMR-110296-0.mpg
ComSkip 0.64, made using:
mpeg2dec-0.4.0 - by Michel Lespinasse <walken@zoy.org> and Aaron Holtzman
The commandline used was:
comskip TheSimpsons-HOMR-110296-0.mpg

Opening TheSimpsons-HOMR-110296-0.mpg
Using comskip.ini for initiation values.
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg0 sec (113.00 fps)
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
Z:\var\media\tv>comskip TheSimpsons-HOMR-110296-0.mpg
err:seh:setup_exception stack overflow 1324 bytes in thread 000b eip 7fbcc362 esp 7faa0ad4 stack 0x7faa1000-0x7fbb0000
STVMC-883 tv # wine comskip.exe TheSimpsons-HOMR-110296-0.mpg
ComSkip 0.64, made using:
mpeg2dec-0.4.0 - by Michel Lespinasse <walken@zoy.org> and Aaron Holtzman
The commandline used was:
comskip.exe TheSimpsons-HOMR-110296-0.mpg

Opening TheSimpsons-HOMR-110296-0.mpg
Using comskip.ini for initiation values.
Using TheSimpsons-HOMR-110296-0.logo.txt for logo data.
0:07:22 - 13281 frames in 95.66 sec (138.84 fps), 1.00 sec (150.00 fps)
start: 12619, end: 12930, len: 10s
0:07:52 - 14174 frames in 103.82 sec (136.52 fps), 2.12 sec (35.85 fps))
start: 12619, end: 12930, len: 10s
0:08:19 - 14973 frames in 109.94 sec (136.19 fps), 1.00 sec (135.00 fps)
start: 12619, end: 12930, len: 10s
0:08:38 - 15552 frames in 113.91 sec (136.53 fps), 1.00 sec (132.00 fps)
start: 12619, end: 12930, len: 10s
0:08:55 - 16060 frames in 117.89 sec (136.23 fps), 1.00 sec (132.00 fps)
start: 12619, end: 12930, len: 10s
0:09:10 - 16512 frames in 120.88 sec (136.60 fps), 1.00 sec (155.00 fps)
start: 12619, end: 12930, len: 10s
0:09:54 - 17849 frames in 130.92 sec (136.34 fps), 1.00 sec (133.00 fps)
start: 12619, end: 12930, len: 10s
0:10:20 - 18613 frames in 136.91 sec (135.95 fps), 1.00 sec (122.00 fps)
start: 12619, end: 12930, len: 10s
0:11:18 - 20357 frames in 150.92 sec (134.89 fps), 1.00 sec (130.00 fps)
start: 12619, end: 12930, len: 10s
0:11:32 - 20788 frames in 153.89 sec (135.08 fps), 1.00 sec (145.00 fps)
start: 12619, end: 12930, len: 10s
0:12:04 - 21732 frames in 160.94 sec (135.03 fps), 1.00 sec (140.00 fps)
start: 12619, end: 12930, len: 10s
0:12:23 - 22294 frames in 166.49 sec (133.91 fps), 1.56 sec (39.10 fps))
22294 frames decoded in 166.50 seconds (133.90 fps)
Commercials were found.

STVMC-883 tv # dir *.txt
NYPDBlue-OntheFence-78271-0.txt SportsCenter-80864-0.txt TheSimpsons-HOMR-110296-0.txt
STVMC-883 tv # dir -l *.txt
-rw-r--r-- 1 root root 13 Jan 22 15:36 NYPDBlue-OntheFence-78271-0.txt
-rw-r--r-- 1 root root 47 Jan 22 15:29 SportsCenter-80864-0.txt
-rw-r--r-- 1 root root 73 Jan 22 15:40 TheSimpsons-HOMR-110296-0.txt
STVMC-883 tv # cat TheSimpsons-HOMR-110296-0.txt
FILE PROCESSING COMPLETE
-------------------
14279 16533
18164 22071

[laurenglenn]

Here´s a batch file to run comskip at a lower priority so it doesn´t interfere with recording:

# nano comskip
#---
nice wine comskip.exe -n $1
------------------------------------------

Oh.... I almost forgot... Go to http://www.sage-community.org to download Comskip 0.64

You can also get the Comskip import from http://forums.sagetv.com/forums/show...comskip+import . Just make sure that you create a copy of the STV file and load it before importing the STVi.


Lauren

[laurenglenn]

Now.... if someone could write an automation program for KDE to run this automatically....

But until then, you can use Wine and DirMon to run comskip.exe

Lauren

[sleonard]

Comskip was originally written for Linux, for use in MythTV wasn't it? I also recall reading somewhere that some of the subsequent improvements made in the Windows port were put back into the original Linux version. Couldn't you just use the native Linux version of comskip instead of running the Windows version with the somewhat unstable Wine?

[laurenglenn]

Originally Posted by sleonard

Comskip was originally written for Linux, for use in MythTV wasn't it? I also recall reading somewhere that some of the subsequent improvements made in the Windows port were put back into the original Linux version. Couldn't you just use the native Linux version of comskip instead of running the Windows version with the somewhat unstable Wine?

I guess I could. Do you have a copy of this program? I couldn´t find a Linux version.

Lauren

[sleonard]

Unfortunately, I don't. The only reason I even know about it is that I kept up w/ everything comskip from when it was ported to Windows to when Jere came out w/ SA.

[churth]

Here is a perl script that I wrote to process all the files in a set of directories against comskip (using wine).

It will skip files that already have a txt file.

Change the $inifile to the location of the comskip.ini file (in the script, I use /usr/local/etc/comskip.ini) and change @directories to list the directories that contain the mpg files.


----

#!/usr/bin/perl

### CHANGE THESE VALUES FOR YOUR SYSTEM
# You have to use 4 \'s per one /, since perl changes \\\\ to \\
# and shells change \\ to \
$inifile = "Z:\\\\usr\\\\local\\\\etc\\\\comskip.ini";
@directories = qw (/video1 /video2 /video3 /video4 /video5 /var/media/tv);



foreach(@directories) {
$dir=$_;
opendir(DIR, "$_");
@files = grep (/\.mpg$/,readdir(DIR));
closedir(DIR);
foreach (@files) {
if ($_ =~ /mpg/) {
$filename = $_;
$filename2 = $filename;
$filename2 =~ s/\.mpg/\.txt/;
if (-e "$filename2") {
print "Comskip file $filename2 already exists\n";
} else {
system "(cd $dir ; nice wine /usr/local/bin/comskip --ini=$inifile $filename )\n";

system "(cd /$dir ; nice wine /usr/local/bin/comskip --ini=$inifile $filename)\n";
}
}
}
}

[bcjenkins]

Originally Posted by laurenglenn

Now.... if someone could write an automation program for KDE to run this automatically....

But until then, you can use Wine and DirMon to run comskip.exe

Lauren

Why not just add it to a cron job? On gentoo emerge cron should do it. Then add the script to your crontab

B

[laurenglenn]

Originally Posted by churth

Here is a perl script that I wrote to process all the files in a set of directories against comskip (using wine).

Very nice and simple. It does the job well. Is there any way to put a pause in there in between files? This way, if it processes all of them, it doesn't go into some kind of endless loop. Thx, Lauren

[Arawak]

Hi All....

I was encouraged by Lauren's successes to take a chance on the Linux version, and got it successfully installed last night. Very pleased to report that it installed quite well.

Of course, immediately I wanted to get commercial detection and transcoding working, so I have been googling away and found this topic.

So, i've emerged wine, and have it running comskip nicely from a shell. My next step is to use the EDL to run mencoder on it and produce a mpeg4 minus commercials. Now, my question is how to make the new file visible to Sage. Do I just overwrite the old mpeg2 with the new mpeg4, complete with .mpg extension? Or is there some other way I should let Sage know about this file?

Anyone know?

Eventually I think I'll write a simple daemon that monitors the system load and the output dir, and launch this process from there.

[churth]

Originally Posted by laurenglenn

Very nice and simple. It does the job well. Is there any way to put a pause in there in between files? This way, if it processes all of them, it doesn't go into some kind of endless loop. Thx, Lauren

It shouldn't get into an endless loop, as it checks for a comskip text file before processing it. If you still want the pause, change where it says sleep 1; to sleep (some number representing the number of seconds).

#!/usr/bin/perl

### CHANGE THESE VALUES FOR YOUR SYSTEM
# You have to use 4 \'s per one /, since perl changes \\\\ to \\
# and shells change \\ to \
$inifile = "Z:\\\\usr\\\\local\\\\etc\\\\comskip.ini";
@directories = qw (/video1 /video2 /video3 /video4 /video5 /var/media/tv);



foreach(@directories) {
$dir=$_;
opendir(DIR, "$_");
@files = grep (/\.mpg$/,readdir(DIR));
closedir(DIR);
foreach (@files) {
if ($_ =~ /mpg/) {
$filename = $_;
$filename2 = $filename;
$filename2 =~ s/\.mpg/\.txt/;
if (-e "$filename2") {
print "Comskip file $filename2 already exists\n";
} else {
system "(cd $dir ; nice wine /usr/local/bin/comskip --ini=$inifile $filename )\n";

system "(cd /$dir ; nice wine /usr/local/bin/comskip --ini=$inifile $filename)\n";

sleep 1; # Change this here
}
}
}
}

[Dekard]

Lol, emerging Wine is taking forever. I'm at 2 hours so far and still cooking. Of course, the linux box is fairly low powered. This is my first time installing Wine, just didn't expect it to take this long.

ps.. emerge is so cool

[drewg]

Originally Posted by churth

Here is a perl script that I wrote to process all the files in a set of directories against comskip (using wine).

Thanks, it was a great help as I'm pretty weak in perl.

I modified your script somewhat to garbage collect .txt and .edl files which seem to linger. I think this happens when you change channels in live tv after you've already started comskipping a file.

Drew

#!/usr/bin/perl

### CHANGE THESE VALUES FOR YOUR SYSTEM
# You have to use 4 \'s per one /, since perl changes \\\\ to \\
# and shells change \\ to \
#$inifile = "Z:\\\\usr\\\\local\\\\etc\\\\comskip.ini";
@directories = qw (/var/media/tv);
#@directories = qw (/tmp/tv);

foreach(@directories) {
  $dir=$_;
  opendir(DIR, "$_");
  @files = grep (/\.mpg$/,readdir(DIR));
  closedir(DIR);
  foreach (@files) {
    if ($_ =~ /mpg/) {
      $filename = $_;
      $filename2 = $filename;
      $filename2 =~ s/\.mpg/\.txt/;
      $pathname = $dir . "/" . $filename2;
      if (! -e "$pathname") {
#       print "Will comskib $filename ($pathname missing)\n";
        system "(cd $dir ; wine comskip $filename )\n";
      }
    }
  }

# look for comskip .txt or .edl files left over after .mpg file has
# been deleted.  These seem to mainly come from live tv.

  opendir(DIR, "$_");
  @files = grep (/\.edl$|\.txt$/,readdir(DIR));
  closedir(DIR);
  foreach (@files) {
      $filename = $_;
      $filename2 = $filename;
       $filename2 =~ s/\.[et][xd][tl]/\.mpg/;
      $pathname_mpg = $dir . "/" . $filename2;
      if (! -e "$pathname_mpg") {
        $pathname = $dir . "/" . $filename;
#       print "rm $pathname\n";
        unlink $pathname;
      }
  }
}

[erik]

Originally Posted by sleonard

Comskip was originally written for Linux, for use in MythTV wasn't it? I also recall reading somewhere that some of the subsequent improvements made in the Windows port were put back into the original Linux version. Couldn't you just use the native Linux version of comskip instead of running the Windows version with the somewhat unstable Wine?

There is a native Mac port of Comskip on google code.
Get the source and you should be able to compile it on Linux

[erik]

Originally Posted by Dekard

Lol, emerging Wine is taking forever. I'm at 2 hours so far and still cooking. Of course, the linux box is fairly low powered. This is my first time installing Wine, just didn't expect it to take this long.

ps.. emerge is so cool

Comskip runs under wine at the same speed as native on windows.
The speed depends on your system

[drewg]

Originally Posted by erik

Comskip runs under wine at the same speed as native on windows.
The speed depends on your system

Comskip under Wine runs about 2x to 3x faster on my system than the native Linux MythTV commercial skipping did.

Drew

[ntk2]

Besides running comskip on the files, what else is required? Comskip detects commercials and generates a text file with the locations. Do you have to run another program to actually create an .mpg file with the commercials removed?

Does comskip work equally well on SD, DT, or HD files? Some of my files are recorded in standard definition using a PVR-150. Others are recorded using a digital tuner (HD-5500).

[bcjenkins]

You need the comskip plugin loaded to use the marking file.

B

[chip33az]

Originally Posted by churth

It shouldn't get into an endless loop, as it checks for a comskip text file before processing it. If you still want the pause, change where it says sleep 1; to sleep (some number representing the number of seconds).

#!/usr/bin/perl

### CHANGE THESE VALUES FOR YOUR SYSTEM
# You have to use 4 \'s per one /, since perl changes \\\\ to \\
# and shells change \\ to \
$inifile = "Z:\\\\usr\\\\local\\\\etc\\\\comskip.ini";
@directories = qw (/video1 /video2 /video3 /video4 /video5 /var/media/tv);



foreach(@directories) {
$dir=$_;
opendir(DIR, "$_");
@files = grep (/\.mpg$/,readdir(DIR));
closedir(DIR);
foreach (@files) {
if ($_ =~ /mpg/) {
$filename = $_;
$filename2 = $filename;
$filename2 =~ s/\.mpg/\.txt/;
if (-e "$filename2") {
print "Comskip file $filename2 already exists\n";
} else {
system "(cd $dir ; nice wine /usr/local/bin/comskip --ini=$inifile $filename )\n";

system "(cd /$dir ; nice wine /usr/local/bin/comskip --ini=$inifile $filename)\n";

sleep 1; # Change this here
}
}
}
}

I'm a little lost on this one. I have Wine set up, and I modified the directories in this script to what I have. The program runs, but it seems to run on each video file, even on ones that have already been processed.

Any suggestions?

[bcjenkins]

I wrote this script for the ComChecker utility I put together for our Mac friends. It worked in my limited testing on my Mac, but it should work in linux with no issue.

http://forums.sagetv.com/forums/down...do=file&id=338