Overnight Conversion MPG to XviD

[rickgillyon]

I've been using various DOS batch scripts for the last few months, to automate overnight transcodes from MPG to XviD. Pretty amateurish I'm sure, but I've been improving them and they work pretty well for me. Quality is generally excellent (of the ouput, not the scripts). Figure I'll post them here and some people might find them useful, and might be able to improve them for me.

Anyway, the scripts are here. Each script contains comments and references, and urls.txt shows where to find a number of the utilities used. Could be adapted easily to produce DivX if that's what you prefer.

Summaries of each script follows! As background, I prefer to use Mencoder to transcode my XviD, but have found that the DVB data from some channels in the UK makes Mencoder crash (five) or Mencoder picks the wrong audio stream (C4). Therefore I use a combination of Mencoder and AutoGK, and decide which by using WGET to query nielm's excellent webserver to see what channel the recording is from. Whew...

I also have a stack of scripts to do odd bits and pieces like one-off transcodes, films etc.

MPG2XVID.BAT
This is my main script, and runs three further scripts. I schedule this to run overnight (2:30), but you could just as easily use another utility to run this for one file using DirMon or similar. Kicks off at low priority for the most part, so TV is unaffected.

Here is where you'll need to set up where MPG, XviD and Temp folders are. Firstly it looks in your MPG folder and calls VRD.BAT, which takes each file which doesn't already have an XviD, and uses VideoReDo to fix it up and copy it to the Temp folder. If you don't want to use VideoReDo, you can just copy the file here. Then it calls both MEN.BAT and AGK.BAT, each checking the channel for the recording to see whether it's the right one to use.

VRD.BAT
Checks if the file exists in the XviD folder and if not, uses VideoReDo to run a quickstream fix and output to the Temp folder (this fixes a lot of sync problems etc.). If not using VideoReDo, just change to a copy.
This script ignores films, which I encode using FILM.BAT, and you may need to change "Film" to "Movie" if you're from the wrong side of the pond...

MEN.BAT
Takes the files in the Temp folder, and uses WGET to check what channel the recording is from, by grabbing the data from nielm's webserver. Doesn't transcode from any of three problem channels otherwise uses 2-pass Mencoder transcoding to produce the XviD.

AGK.BAT
See above, if it's a problem channel this script uses AutoGK for the transcode. If you prefer AutoGK you can just not call men.bat, drop the WGET and use AutoGK exclusively.
You can choose one-pass or two-pass transcoding depending on whether you use a percentage or a calculated filesize (see comments). Then uses AutoGKAdd.EXE to add the file to the AutoGK queue, which is kicked off at the end of the whole process using AutoGKStart.EXE (in MPG2XVID). Simple huh?

AutoGKStart.exe and AutoGKAdd.exe are produced using AutoIt v3, which is a free automation tool. If you want to check them out you can reverse-compile the EXEs using AutoIT.

FILM.BAT
I use this one to transcode Films. Uses VideoReDo, AutoGK and 2-pass transcoding to produce films at 1.4GB (3 per DVD). Tweak as neccessary! Associated in WinXP with MPG files, so appears on right-click as "Transcode Film". If you make the size 1400 like in this file, AutoGK will also create two 700MB files for CD Archiving; if you make it 1401 no split files will be created (a bit qucker).

AUTOGK.BAT
Simple right-click transcode, kicks off VideoReDo then AutoGK with a fixed quality of 62%. I use this where I want a quick encode without waiting overnight. Associated in WinXP with MPG files, so appears on right-click as "Transcode with AutoGK".

DELXVID.BAT
Simple script to compare XviD folder with MPG folder; looks for XviDs with Sage-style names, and if no equivalent MPG exists asks you if you want to delete the XviD. Just used to clean the XviD folder when you've watched the video in Sage.

EMPTYVID.BAT
Use WinXP to associate with MPG, passing %1 as parameter, so it appears on right-click as "Exclude from XviD"; just creates a small fake XviD .avi so the file won't be processed overnight.

Questions, corrections, improvements, insults - bring 'em on!

Oh, and use at your own risk - if these scripts crash your PC, burn down your house or cop off with your girlfriend/boyfriend/wife/cat/dog, then I take no responsibility!

Rick

[jlindborg]

Hey Rick.

Thanks for the effort on this, by the way... Wish I'd know about that nifty AutoIT scripting app - wasted a bunch of time grabing focus and shoving keystrokes into the AutoGK app and such - that's a much slicker mechanism.

I'm working on a little graphical "wizard-y" type interface for me to use for standing replacements (I guess I can make it optional for standing vs. keeping both the mpg and AVI around). I'm working on a way to just have a configuration GUI for determining aspect ration preferences, compression values, which converter to use etc... and then be able to call it via a simple command line using DirMon2 - Jere's already got all the nifty processing rules/folder iteration, web server checks and stuff in there so no need to reinvent the wheel on that front. If it doesn't completely stink up the joint I may post an install for it out on my page. We'll see how it goes.

Anyway, thanks for slogging through the details on some of this.

[rickgillyon]

Woot! A reply!

The effort was really all for my own benefit...it does the job for me, but I expect most people would need to customise the scripts or hack them into other products. Using AutoIt to build a simple front-end should be easy enough. Good luck...

Interesting comment about Dirmon2, I tried it, but had some problems, and the simple scheduled task does the job for me. Saw no need to have another product running all the time, but it is a little more, uh, sophisticated than my solution...

As you've no doubt seen, setting aspect ratio with AutoGK can be done via the advanced settings but more simply by copying a .aspect file to the process folder. I find nearly everything is 16:9 now anyway in my setup, and if you let AutoGK auto-detect aspect you can get problems if you start recordings early, and the preceding program is 4:3. Ouch. I've amended my script to always set 16:9 on my PVR.

[Enormous]

Great work here. I've recently been playing with some scripts to cut the commercials and transcode to XviD as well... I'm going to check yours out, and probably steal some ideas! My ultimate goal is to be able to do like we used to with Cayers STV. But that's probably a long way off for me to accomplish.

[rickgillyon]

Originally Posted by Enormous

I've recently been playing with some scripts to cut the commercials and transcode to XviD as well...

If you use comskip (I don't so I have no idea how it works!), you can use the output file with either mencoder or VideoReDo to produce the fixed mpeg file, apparently. So you should find most of what you need for that in these scripts...

[jlindborg]

Just in case anyone was wondering - ran into all kinds of trouble trying to avoid shutting down Sage and restarting it - viewed on the server directly the Sage interface works dandy, but the MVPs are all jacked up (presumably the logic to transcode is confused by the .mpg extension on a XDiv compressed AVI) - so much for that plan.

I did go ahead and just shut down the sage service at 2am, copy all the AVIs that had been crunched on another box over, "touch" their time and dates to match and then delete the MPGs and start the sage service again - all seems to have gone well. This is a rather brutal process at best and I haven't figured out how to tell automatically when there's going to be a window of time that will let me shut the service down safely without losing any record jobs. Automatically finding and stacking up compression jobs and such is not hard now thanks to the bat files to work with for the process flow and the nify AutoIt app to generate remote events in the AutoGK GUI app (why they don't have a proper CLI on that thing I don't understand). But changing file names and requiring restarts of the service is proving to be a bit of a mess.

Continuing to wish there were an easy way to punch into the Sage show DB and change the filename (or location-think off box archiving here) of a recording on the fly... you wouldn't think something like that would be overly tough but I have no clue what that Wiz.bin file is actually structured like...

Sigh... time to go watch the Mariners beat up on Boston

[rickgillyon]

Just a quick note that these scripts still work with AutoGK 2.35, the latest release. I've not seen any problems anyway!

I'm currently working mainly with 1-pass XviD at 50% quality, looks great on my 20" LCD...

Rick

[Malfunction]

I can't figure out how to get filenames into the batch file. I can get full filenames in, but making the output .AVI filename has me stumped. Example:

What I want to do:

touch -r show1.mpg show1.avi

Buggy line, don't do it this way:

FOR %%Z IN (*.mpg) do touch -r %%Z.avi

Is this done with DIRMON or something?

[JREkiwi]

Originally Posted by Malfunction

I can't figure out how to get filenames into the batch file. I can get full filenames in, but making the output .AVI filename has me stumped. Example:

What I want to do:

touch -r show1.mpg show1.avi

Buggy line, don't do it this way:

FOR %%Z IN (*.mpg) do touch -r %%Z.avi

Is this done with DIRMON or something?

Using this doc (my lifesaver), so something like this

touch -r %1 %~dpn1.avi

[Malfunction]

Thanks, that helped a lot! It turns out I needed to create 2 files:

(Anyname.bat)
@echo off
FOR %%Z IN (*.mpg) do call fixdate2.bat %%Z

(fixdate2.bat)
@echo off
touch -r %1 %~n1.avi

Don't do this in your main Sage show directory or it will drop 0 byte .avi files, which could cause you lots of grief. I do my compressing on another computer and copy the finished files over to the Sage computer over my network.

[davephan]

I am new to video compression. Will Xvid playback with a Hauppauge MVP media extender? Are there advantages using Xvid verses DivX or AVIs? I currently record programs with MPEG2 using the 'standard DVD' quality level, about 3 gigs per hour.

I would like to try using your scripts using a second computer with mapped drives. I assume that the trancoding process is very CPU intensive. I would then be able to offload the heavy CPU job.

I have VideoReDo. I don't understand why you are processing the files with VideoReDo. Does it improve the quality quite a bit?

Dave

[rickgillyon]

Originally Posted by davephan

Will Xvid playback with a Hauppauge MVP media extender?

No, if you have XviDs as imports then Sage transcodes them back to MPGs to play on the MVP. If you want your vids playable on MVP, best to leave as they are! I transcode so that I can grab the XviDs onto a removeable hard drive and take away with me to watch when I'm working out-of-town.

Originally Posted by davephan

Are there advantages using Xvid verses DivX or AVIs?

Both XviD and DivX are AVI. I find XviD gives better quality in a smaller file, but others would no doubt disagree.

Originally Posted by davephan

I would like to try using your scripts using a second computer with mapped drives. I assume that the trancoding process is very CPU intensive. I would then be able to offload the heavy CPU job.

It is, and that's exactly what I do. All my transcoding now is over the network on my main, non-TV PC.

Originally Posted by davephan

I have VideoReDo. I don't understand why you are processing the files with VideoReDo. Does it improve the quality quite a bit?

Not really, and if your recordings are analogue it probably isn't neccessary. But my digital recordings often have sync errors, and this just cleans those errors before proper transcoding starts. I need to do the same for the recordings from my camcorder.

[davephan]

Originally Posted by davephan
Will Xvid playback with a Hauppauge MVP media extender?

[QUOTE=rickgillyon]No, if you have XviDs as imports then Sage transcodes them back to MPGs to play on the MVP. If you want your vids playable on MVP, best to leave as they are!

I was thinking about compressing the video files so that I could fit more files on my SageTV computer without having to increase the disk capacity (I already have over 1 TB). I was thinking that the MVP could playback the compressed videos with the SageTV computer decompressing the compressed video files on the fly.

Does this sound like a practical solution, or does it sound like it would not work?

Dave

[rickgillyon]

It will work, but there will be more load on the server as it transcodes for MVP. Whether this will harm live tv or not depends on your setup. Personally I'd just leave them as MPG and buy another disk, it's easier...

Ask Santa for a 750GB drive.