STV: Cayars17 Ultimate STV (Release 17 - Jan 18)

[Deadbolt]

I have VideoRedo.

[zoundz]

Originally Posted by davephan

Maybe parts of the comskip.exe need to be written in a more efficient coding (such as assembly language, if that is possible?) to reduce the CPU load. Maybe the only easy alternative is to offload the process to another computer.

I'm currently in the process of reworking the existing Comskip to improve some of the efficiency aspects. Unfortunately, most of the CPU time that Comskip uses is devoted to the mpeg decoding process. AFAICT, libmpeg2 is about as efficient at that process as any of the available software decoders, so not much improvement can be expected in the overall speed. It is my plan to make Comskip optionally work from a script file, so that once it is told what to do, it will just do it, rather than having to be restarted for each .mpg file. The problem with moving Comskip off the main SageTV encoding machine is that it will create a lot of network traffic reading the files to process them. That may not be a bad thing. Hopefully, within the next couple of weeks I'll be able to make a new one available that will be somewhat easier to deal with. Just as a matter of information, the changes and additions that Jere Jones has made over the past few months are incredibly functional are not going away. Most of my work is concentrated on improving the code efficiency and simplifying some of the processes involved.

If anyone wants to make suggestions about changes to Comskip, now's the time.

[Deadbolt]

I use comskip from a networked SageClient machine....it works fine even though it is over the network.

[rsagetv99]

Originally Posted by Cayars

But, I'll try and keep everything modular and scriptable so you can set it up the way YOU WANT it to run.

Yeah modular and scriptable is always better. Depending on where your burner is installed (mine's on my SAGE PC) two complete reads over the network might not be any worse since you would have to copy the file twice, once to the processing PC, then process it (two complete reads), and then copy it back to the SAGE PC - of course the second file copy would be a lot faster since the file would probably not have any commercials and might even be compressed - but you get the idea.

[rsagetv99]

Originally Posted by Cayars

I.E.
Check channel or show information to see if it's on a list of channels or shows to process OR check to make sure it's not on a list of Do Not touch shows/channels comskip,
Then Run ComSkip on the file. With results of ComSkip cut commercials from show.
Then transcode show to DivX using AutoGK or DivX.
Put an "Upload File" in a directory for the STV to processes. Also copy show back to video folder.

Then the STV could look for any new "Upload Files" and if it find them update show information accordingly (new size, new extension and size, etc).

Just some ideas...

This reminded me of a feature that I had thought of before, but always forget to request .

It would be a nice feature if you could automatically compress shows without having to select it for each show. What I am thinking is maybe put a control option in the "Favorites" screen that you could set once and then forget about it. I have a few favorites that I always compress, so if this option was available it would make things a little easier.

Maybe another Favorite option could be Commercial Skip High/Low/Disabled, if you have some shows that you would rather have processed before other shows, or maybe some shows that shouldn't be processed at all. (I know you can disable CS by channel, but by show could be more accurate).

Also, maybe you could have an option to point to a different comercialskip.ini file for certain Favorites - like maybe some settings work better for Football/sports since the commercials tend to be more random and different lengths than for most programs.

Having the options at the favorite makes it a little easier to get exactly what you want

[Cayars]

Originally Posted by rsagetv99

This reminded me of a feature that I had thought of before, but always forget to request .

It would be a nice feature if you could automatically compress shows without having to select it for each show. What I am thinking is maybe put a control option in the "Favorites" screen that you could set once and then forget about it. I have a few favorites that I always compress, so if this option was available it would make things a little easier.

Maybe another Favorite option could be Commercial Skip High/Low/Disabled, if you have some shows that you would rather have processed before other shows, or maybe some shows that shouldn't be processed at all. (I know you can disable CS by channel, but by show could be more accurate).

Agreed, sort of like the new options I added for Comskip in Detailed Setup? Has anyone even seen these yet? This is especially important on machines that have IR turned on. The show could easily be removed before it's even done processing!

Also, maybe you could have an option to point to a different comercialskip.ini file for certain Favorites - like maybe some settings work better for Football/sports since the commercials tend to be more random and different lengths than for most programs.

Having the options at the favorite makes it a little easier to get exactly what you want

Yep, I've already been playing a little with this. Some of the % settings are pretty easy to adjust based on length of show. For example 30 minutes versus 1 hour versus over one hour shows tend to have a different percentage of commercials in the US. Some channels don't have commercials at all (ie. HBO). Some channels have logos always displayed and other have logos that are displayed even during commercials to throw off detection. We could tweak comskip.ini for each of these types of things. More trial and error then anything else to figure this part out but a good external process with some scripting ability would make this a lot easier to do.

Carlo

[rsagetv99]

Originally Posted by Cayars

Agreed, sort of like the new options I added for Comskip in Detailed Setup? Has anyone even seen these yet? This is especially important on machines that have IR turned on. The show could easily be removed before it's even done processing!

I just took a look at the settings, pretty cool . Won't help me much because I don't use IR but still cool. That's the general idea, but it would be nice if you could turn compression/CS on and off by modifying the favorite.

Originally Posted by Cayars

Yep, I've already been playing a little with this. Some of the % settings are pretty easy to adjust based on length of show. For example 30 minutes versus 1 hour versus over one hour shows tend to have a different percentage of commercials in the US. Some channels don't have commercials at all (ie. HBO). Some channels have logos always displayed and other have logos that are displayed even during commercials to throw off detection. We could tweak comskip.ini for each of these types of things. More trial and error then anything else to figure this part out but a good external process with some scripting ability would make this a lot easier to do.

Again, maybe something at the favorite level. Wouldn't even need to be that difficult, just a setting in the favorite that says something like "comskip.ini file = comskipfootball.ini" where you could select a different ini file for shows that comskip has problems with. If nothing is selected it could default to the usual comskip.ini file.

EDIT: That way you would not have to have to make one comskip.ini file fit all of the shows, you have separate settings depending on the show's tendencies.

[Cayars]

DVD Burner status.

I've been making lots of little changes here and there to try and overcome a few obstacles that keep popping up. I'm half tempted to scrap the approach and just write things myself and be done with it!

Venting! My original intention was to write a wrapper between SageTV and DVDAuthor. However, I quickly learned this has some quirks.

In order to make a DVD we have to DeMux the MPG files into two files (audio and video). Then you need to Mux these back together with DVD packets included. The open source DVDAuthor has command line utils to do this and I was using them originally. These files are mpgtx.exe and mplex. The problem I hit is that mpgtx.exe only works with local drives. It will not touch a UNC drive at all.

So I found another util called bbdmux.exe that will work with UNC drives. It however needs to be run twice on the source MPG. Once for audio and once for video. This seemed to work. I coded the app to use mpgtx.exe if the files are local and bbdmux.exe if the file is a UNC.

However, as I further tested things on my main machine I kept getting crashes (works fine on my dev machine). I ended up finding out that bbdmux files have some issue in the audio file generated that keeps things from MUXing again correctly.

To throw another monkey wrench into things I added support to be able to cut commercials out using VideoReDo based on ComSkip code. One of the nice features with VideoReDo is that you can have it generate elemental files (seperate audio and video) files. I also just discovered how to use VideoReDo to do the DeMux without cutting. VideoReDo DOES work with UNC files just fine. It's also faster then mpgtx.exe and bbdmux.exe but not quite as clean programming wise.

So what I'm going to do is use VideoReDo if you have it on your system as the first choice. BBDMUX.EXE will not be used at all. If you don't have VideoReDo installed then I'll just copy the UNC file over to the local drive (even if it's the same drive) and use mpgtx.exe on it in local mode.

I need to make a few adjustments and an STV change or two and then make a couple of test burns.

I was also working on a couple of different methods to actually write the DVD itself. For now I'm bagging this and will use a "safe" but somewhat slower method as it needs an extra step. I'll come back to this after we know all the other steps are working correctly!

Carlo

[Cayars]

Originally Posted by rsagetv99

<snip>
Again, maybe something at the favorite level. Wouldn't even need to be that difficult, just a setting in the favorite that says something like "comskip.ini file = comskipfootball.ini" where you could select a different ini file for shows that comskip has problems with. If nothing is selected it could default to the usual comskip.ini file.

EDIT: That way you would not have to have to make one comskip.ini file fit all of the shows, you have separate settings depending on the show's tendencies.

Maybe I missed something but would using "comskipfootball.ini" be a different ini file?

[rsagetv99]

Originally Posted by Cayars

Maybe I missed something but would using "comskipfootball.ini" be a different ini file?

Yes, what I am saying here is to have different ini's for different favorites. Maybe one default ini that is tweaked for most shows, but the option to point particular favorites to a different ini file with different settings that work better with that particular favorite (like football or sports).

[Cayars]

Or maybe pass all command on the command line based off a grid type thing so it could dynamically build the proper commands based on things like duration of show, channel number, CC data, etc...

[JasonJoel]

I've been doing a bunch of testing on different length shows (like you have I think), and agree that different settings for different shows/lengths will help.

Personally I like the idea you just mentioned. Having a way to automatically change the settings based on length.

Some people may want different settings per favorite, but so far from my testing it is more length of show dependant than anything. Although I'll concede that sports may be an expection to that as it is very non-standard.

A way to dynamically make the INI based on preferences is the way to go.

Jason Bottjen

Originally Posted by Cayars

Or maybe pass all command on the command line based off a grid type thing so it could dynamically build the proper commands based on things like duration of show, channel number, CC data, etc...

[rsagetv99]

Originally Posted by zoundz

If anyone wants to make suggestions about changes to Comskip, now's the time.

1. I think I read in another post that comskip is based on the black screens and/or CC info (which may or may not be present depending on the show or hardware encoder being used), I think it was mentioned that comskip does not use audio for detection - I think the Replay DVR uses black screens and audio - so if it would help comskip be more accurate maybe that can be added.

2. Also, since we are talking about individualizing the comskip settings for different shows, the ability to call comskip with command-line options would be very helpful. Basically, if a command line parameter is used when calling comskip, then it would override that parameter in the comskip.ini file. The comskip.ini file could be used as the default setting for parameter that is not specified in the command-line. Of course this means that the majority of the comskip parameters would need to be available via the command-line.

EDIT: Carlo, since you are the one integrating this into the STV, will request number two work for you? If not please put in your 2 cents.

[Brent94Z]

Hello!

First off, let me just say you guys are incredible and I want to thank all of you who work on Sage to make it better for everyone!

With that, I'll say that I have not even yet installed Sage. I bought it some time ago and have been building my HTPC for like a month now (too many other projects ). After reading some things about 2.1/2.2 I was going to stick with just v2.0. However, after looking at all that is going on with this STV, I'm most likely going to give the newest version a shot

I'm the type who likes to somewhat understand what he's doing before just diving in (although sometimes I do have to just dive in!). This is where I'm a bit confused about installing this STV. Keep in mind though I don't even have Sage up and running yet... just spending a lot of time reading and trying to figure things out But, in reading/skimming many of the 80 pages here I'm stumped about just the pure basics. LOL!

For example, in the original post here as part of the install it indicates...

This should be all that is needed to upgrade from any release "5" or greater setup already installed.

Ok... so once I get Sage up and running, since I didn't get in on this from the beginning, how would I go about installing this right now with no previous release already installed?

I've got some time off from my job in the next few days and expect to have my HTPC up and running (finger crossed!) and would like to jump right in with this very cool STV. Would it be possible for one of you smart guys to step me through with a step by step "guide" on just how to install this on a new install of Sage? It would be much appreciated and since I don't like to think of myself as "stupid" I'm thinking some others might be wondering the same thing and it might help them too! LOL! Thanks!!!

[krutaw]

Cayars ~

I was just trying to download the latest of your STV, and noticed that 2 of the 3 required files are no longer available. Any idea where I can get these files?

The files I'm talking about are from the first post in this thread...

"DOWNLOAD FILES
You will need to download 3 files:
Extract JavaR1.zip into your Java library extensions directory (i.e. C:\Program Files\Java\j2re1.4.2_05\lib\ext)

Extract CoreFilesR1.zip into your main SageTV directory. Please make sure nested folders get extracted correctly.

Extract Cayars15.zip into your main SageTV directory. Please make sure nested folders get extracted correctly. This file normally only contains the menu and STV files. This should be all that is needed to upgrade from any release "5" or greater setup already installed."

[Cayars]

Try it again now.

[rsagetv99]

Originally Posted by Brent94Z

Would it be possible for one of you smart guys to step me through with a step by step "guide" on just how to install this on a new install of Sage? It would be much appreciated and since I don't like to think of myself as "stupid" I'm thinking some others might be wondering the same thing and it might help them too! LOL! Thanks!!!

Not sure how much more step by step Carlo could have been. The only thing that I can add to his steps would be at the beginning:

1.Install SAGE software.
2.Make sure you have Java Runtime Environment installed.
3.Backup your SAGE directory (i.e. c:\program files\frey technologies\)
4.Extract JavaR1.zip into your Java library extensions directory (i.e. C:\Program Files\Java\j2re1.4.2_05\lib\ext).
Etc...

[Cayars]

Or maybe the fact that after installing the files you have to change the STV file being used in Detailed Setup->Advanced.

[rsagetv99]

Originally Posted by Cayars

Or maybe the fact that after installing the files you have to change the STV file being used in Detailed Setup->Advanced.

Yeah, actually if you've never changed your STV before that is a BIG one!

[zoundz]

Originally Posted by rsagetv99

1. I think I read in another post that comskip is based on the black screens and/or CC info (which may or may not be present depending on the show or hardware encoder being used), I think it was mentioned that comskip does not use audio for detection - I think the Replay DVR uses black screens and audio - so if it would help comskip be more accurate maybe that can be added.

2. Also, since we are talking about individualizing the comskip settings for different shows, the ability to call comskip with command-line options would be very helpful. Basically, if a command line parameter is used when calling comskip, then it would override that parameter in the comskip.ini file. The comskip.ini file could be used as the default setting for parameter that is not specified in the command-line. Of course this means that the majority of the comskip parameters would need to be available via the command-line.

Currently Comskip tries to use a number of different criteria for determining commercials. Black frames delimit blocks. Comskip looks at the presence of station logos, CC's if present, dominant aspect ratio, scene change rates, and block lengths. Currently it does not use audio information because it is not available using libmpeg2 as a decoder. Maybe in the future. You are correct that the audio information could be helpful, in particular for detecting that a small block of program is actually part of the program and not a commercial. The sound has a tendency to be continuous across that sort of division.

As far as using different parameters from the command line, I think that once scripting is available, that need might better be addressed as script input rather than command line. The ability to use different .ini files for different programs can easily be done from scripts and it's certainly conceivable to allow .ini file input directly from the script to override the .ini file itself. We'll see as that progresses.

Thanks for the input.