Automated in-place recompression

[ptaylor]

Ok.. I've spent a good portion of the day working on my automated method of in-place recompression of SageTV...

I just got the first file that ran through (in a test directory)... AutoGK compressed it from about 1 GB to about 168 MB (set it to 20%)... I played it in Window Media Player and it honestly doesn't look as good as past files that I've selected the filesize, so perhaps that's something AutoGk is better at... I have an idea about how to fix that...

The original file was in "F:\InDirTest" and the .AVI file was in "F:\mpegWorkDir". My program ran again, saw the .AVI file was done, deleted the original file, then copied the new one to the "F:\InDirTest" location with the original filename (.mpg), and performed a "touch" to set the timestamp properly.

Since Sage doesn't know anything about that directory, I copied that file to the F:\Sage directory overwriting the REAL original file (that I had made a copy of)... I've checked and the copy preserved the "touched" timestamps... When I try to play it in Sage, I hear audio and have a blank screen. Almost immediately, I get a popup box indicating that Sage has crashed.... The progress bar at the bottom shows 30 minutes, the correct length of the show... Until I acknowledge the pop-up, I can still hear the audio playing...

So, I try some things and it seems that the nVidia codec is causing the crash... Using SageTV's built-in video codec, it works.... So does the one from PowerDVD.

Anyone know how to make the nVidia codec work also?

I also got a message about comskip crashing... That was after I deleted all the various files related to this video and restarted Sage.... I'm guessing that comskip tried to run against that file and it didn't like the fact that it wasn't an MPG file.

Paul

[ptaylor]

Well, I have a program that I'm testing that looks pretty good so far...

It's written in PHP and uses a file based database (SQLite) to keep track of everything. You configure it via the command line to tell it which shows to record, etc... Basically, you have to install PHP 5.04 on your machine (unzip it to a directory and add that to your path), have an installation of AutoGK, and make about three directories, and unzip my package to one of them, configure it and schedule it to run each night..

After it's all configured, it does this (It does more, but this is the basics):
1. Scans the directories that you specified for your native Sage recordings.
2. If it finds a .mpg file that is for a show you've specified to be recompressed, it checks its history table to see if its already compressed it.
3. If the file is specified as a series you want compressed and it hasn't yet been compressed, it's added to the AutoGK queue and to the "inProgress" table. It repeats this for any other shows it needs to compress, then starts AutoGK's batch job.
4. Next time SageRecompress runs, it checks to see if any of the queued up processing is complete. (and has been complete for 1 hour.. Did this because it was possible to have the 1st pass .avi file mistaken for the finished product.) If it is done, it will backup the original .MPG file (optional), and copy the compressed .AVI file over the top of the .MPG file (including extension), updating the timestamp on the file to match the original .MPG file. It also cleans up most of the files left behind by AutoGK... (It can't clean them all up because it's possible that AutoGK is still processing other files, and touching some of the temp files could interrupt that process.) It also adds the file to a history table. If a file is in either the history table or the inProgress table, it won't be queued up in AutoGK again...

So, the plan is to have this run automatically every night at about 2AM... This will kick off recompression for the shows that have been specified that were recently recorded...

Problems:
The biggest problem that I've found is that the nVidia codec version 1.00.67 that I have doesn't handle reading .avi files that are renamed to .mpg very well... (I think I've read that 1.00.68 is out? If so, point me there and I'll download it and try, but I could only find 1.00.67 on nVidia's site, I believe). When I try to play one of these recompressed files with the nVidia codec in Sage, it crashes Sage. Using the codec that comes with Sage or the one that comes with PowerDVD, it works fine. I also own a copy of Sonic's codec, but haven't tried it. It's on my wife's PC, so I'll test it out later.

In addition to this, I have seen occasional issues with adding files to AutoGK... It's not a real frequent thing, but it is an issue that has popped up occasionally on my machine.. Not sure if it's because I'm remote controling the PC when I'm testing (if VNC is somehow causing an issue with the mouse movement?)...

Now, the natural thing to do would be to write up a web interface into my SageRecompress program... I'm not that much on web page design, but I would like to have an interface to make updates and changes to the way it works easier than via a command line interface... I can do back-end programming and all, if I can get someone who is talented enough to come up with a decent template for me to use.

I am not quite ready to release this into the wild just yet, but I wouldn't mind getting an alpha tester or two. If you are interested, let me know and I can zip up a pack and send it to you within a few days.

Paul

[alon24]

Hi ptaylor,

I can test your prog, its something I think is missing in sage for a long time.

Also, do u only support Autogk? I am trying to see what mencoder can do right now, and I see that people say its better quility, and faster.

But I will be happy to test anyways, as this is something close to my heart.

[ptaylor]

Ok - I'll put something together within the next couple of days to let you know how to install and use it.

Paul

[ptaylor]

Just a few lines to let everyone know that I'm not working on this at this time... I ran into an issue or two with the program and have had trouble getting time to test possible solutions.

My largest issue was that my recordings in Sage stopped having sound after adding my new ATI tuner... Oddly enough, the sound problem seemed restricted to my wife's soap, on especially vital days to the plot, apparently... So naturally, I had to find a solution immediately. No one was able to offer me a fix in the forum, so I decided to look back at BTV.. I'm not having sound issues with it, and I am pretty happy with the functionality of the latest version, so I have switched back to it for now... The integrated ShowSqueeze seems to do what I was attempting to do with this project, though the resulting files are a bit larger than XVID at similar visual quality.