Cmd Line MPEG4 Encoder - ffmpeg

[Alex0230]

I'll take a crack at it if no one else wants to, so let me get this straight:

-The UI will basically create an XML dump file that gets places in to a folder (configurable location through sage.props?).
-You want a scheduled task that will just pick up all the xml files (configurable location) and compress each mpeg2 to mpeg4 that are found in the various xml files (deletes them after processing correct?).
-The .my file comes from the scheduled task which gets it data from the xml...the format for the .my file can be found in your other thread right?

It doesn't sound too bad, and since the weekend it coming up...perfect timeing. I will most likely write this in java, should be fun.

One question.....this UI is based on show level right, meaning user selects the show/recording to compress (not the file)? How will a show be handled that happened be broken up in to multiple mpeg2 files? Will there be multiple sourcepaths and multiple destpaths or does ffmpeg have the ability to splice as well?

[mlbdude]

Originally posted by Alex0230
I'll take a crack at it if no one else wants to, so let me get this straight:

-The UI will basically create an XML dump file that gets places in to a folder (configurable location through sage.props?).
-You want a scheduled task that will just pick up all the xml files (configurable location) and compress each mpeg2 to mpeg4 that are found in the various xml files (deletes them after processing correct?).
-The .my file comes from the scheduled task which gets it data from the xml...the format for the .my file can be found in your other thread right?

It doesn't sound too bad, and since the weekend it coming up...perfect timeing. I will most likely write this in java, should be fun.

One question.....this UI is based on show level right, meaning user selects the show/recording to compress (not the file)? How will a show be handled that happened be broken up in to multiple mpeg2 files? Will there be multiple sourcepaths and multiple destpaths or does ffmpeg have the ability to splice as well?

Here is the XML format:

<?xml version="1.0" encoding="ISO-8859-1" ?>
<Airing>
<Title>Stephen King's Kingdom Hospital</Title>
<Episode>Seizure Day</Episode>
<Description>After having seizures, two patients begin to have visions of the living dead haunting the hospital.</Description>
<ActualStartTime>1088730000000</ActualStartTime>
<StartDate>Jul 1, 2004</StartDate>
<StartTime>9:00:00 PM</StartTime>
<EndTime>10:01:00 PM</EndTime>
<File>W:\SageTV\StephenKingsKingdomHospitalSeizureDay-2037013-0.mpg</File>
<TargetRoot>m:\Compressed</TargetRoot>
<VideoBitrate>1800</VideoBitrate>
<AudioBitrate>192</AudioBitrate>
<DeleteWhenComplete>true</DeleteWHenComplete>
<FirstRun>true</FirstRun>
<Favorite>true</Favorite>
<Manual>false</Manual>
<Watched>false</Watched>
</Airing>

Here is the current .my format:

title=Stephen Kinds Kingdom Hospital::Seizure Day
overview=After having seizures, two patients begin to have visions of the living dead haunting the hospital.
released=Jul 1, 2004 9:00:00 PM

Here is a ffmpeg cmd line: (the video and audio bitrates match the XML above)

ffmpeg -i W:\SageTV\StephenKingsKingdomHospitalSeizureDay-2037013-0.mpg -deinterlace -y -f avi -vcodec mpeg4 -b 1800 -g 300 -bf 2 -acodec mp2 -ab 192 m:\compressed\Stephen Kings Kingdom Hospital\StephenKingsKingdomHospitalSeizureDay-2037013-0.avi

<DeleteWhenComplete>true</DeleteWHenComplete> will tell the compressor if it can delete the source when it is complete or not.

No solution for the multi-part files. I was thinking of generating an error dialog in Sage if a user tries to compress one of them. Generally it is hard to get one of those anyway.

It would be nice if the compressor allowed the user to select the priority of the ffmpeg jobs as well. By default though they should all be IDLE or Low just in case this is run on a SageTV machine.

It should also be assumed, that just like a network encoder, the machine doing the compressing needs to have drive letters matching what is on the SageTV server.

[Deadbolt]

One big benefit to DivX is the higer level of support right now...like in the MediaMVP, pocket video players, etc.

[mlbdude]

The nice thing about this method though is that if there is ever a DivX or Xvid command line tool or API a new compressor can be created for it and nothing needs to change in Sage.

Heck, someone could write one for WMV too if they did not use Sage for watching them.

I would assume any ISO MPEG4 files ffmepg genrates will be compatible with any DivX or Xvid player.

[Alex0230]

btw did you or has anyone gone through and try to get ffmpeg running? GOODNESS!! I'm trying to build an ffmpeg installer so no one has to go through what I'm going through... : ) If anyone has already done so....I'd appreciate them posting it. Basically you have to compile the source, but you gotta download like 4 different packages first. : )

[Jason]

Originally posted by Alex0230
btw did you or has anyone gone through and try to get ffmpeg running? GOODNESS!! I'm trying to build an ffmpeg installer so no one has to go through what I'm going through... : ) If anyone has already done so....I'd appreciate them posting it. Basically you have to compile the source, but you gotta download like 4 different packages first. : )

Try this link.

http://www.videohelp.com/tools?tool=263

There is a link to the pre-compiled windows version. It is based on the CVS release of 2-26 (ver 0.4.8), but it works!

-Jason

[Alex0230]

I just ran some ffmpeg conversion comparisons, here are some results:

30min Recording on Default Quality (1GB/Hr)

1800 - 192 -- saved ~50MB
1700 (input bitrate) - 192 -- saved ~ 71MB
1300 - 192 -- saved ~150MB

Each conversion takes about 20-30 mins and runs anywhere between 60-90% CPU on my 1.6GHz Centrino. Either in Java or in the scheduler, hopefully the priority can be dropped from normal to low...if in Java, I'll allow it to be configured.

[mlbdude]

If you can't drop the priority in Java, look into doing a a CMD line approach including:

start /wait /low ffmpeg.......

In my last testing I took a 15 minute 3.2GB/hr recording from 880MB to 248MB using 1800/192. Quality was almost the same.

[JUC]

This will be AWESOME!!! Thanks so much for the effort
JUC

[Alex0230]

Originally posted by mlbdude
If you can't drop the priority in Java, look into doing a a CMD line approach including:

start /wait /low ffmpeg.......

In my last testing I took a 15 minute 3.2GB/hr recording from 880MB to 248MB using 1800/192. Quality was almost the same.

I couldn't seem to use the start command when creating a process in java, nor in vb. I think this is because "start" is a part of the command line shell and not really an executable on your machine (atleast I couldn't find it). I have a windows api work around I'm attempting though..

I'm really closed to be finished guys...just gotta iron out some last minute things with MlbDude and I'll post RC1.

[edbmdave]

DIVX 5.2 and DrDivx 1.05 Released


http://www.divx.com

[mlbdude]

We are going into private beta testing with this feature using ffmpeg any moment. This feature will work similar to how I did the Weather menu in that other external encoders could be created if someone finds a better encoder or has the knowhow to write one directly for DivX or XVid.

We are having a couple of issues with ffmpeg though that may stall the entire project. This may be my last Sage project for awhile though so I want to do it as best as possible .

[Ryokurin]

Originally posted by edbmdave
DIVX 5.2 and DrDivx 1.05 Released


http://www.divx.com

Did you notice how in the readme they claim that you can now encode using the fastest setting in real time and it equals the same quality as standard? I kind of find it hard to believe. will be nice if so however.

[Ryokurin]

Originally posted by edbmdave
Does anyone know what the pros / cons of using MPEG4 vs. XVID?

Do we know if the project is actively moving and supported? It looks like the last release was 9/2003 (almost a year ago).

http://forum.doom9.org/showthread.php?threadid=79469 ffmpeg is still being worked on however the builds are similar to ffdshow in that they dont put them on sourceforge. In fact I think you have to build this on your own.

[mlbdude]

Howdy all,

Alex and I are ready to start to get this project tested out on a few other machines. We are seeing some issues that are hard to reproduce so we would like a larger test group.

Known possible issues:

• ffmpeg may not work well with large files. I notice audio cuts out after a few seconds. Might need some ffmpeg experts to help with this one.
  
  Spawned ffdshow sessions may not be started in Idle priority.
  
  Some Favorites are not being captured for auto-re-compress.
  
  Source being deleted even though I have not requested it to.
  
  Audio bitrate is not being set in STV (this is fixed for next release).

Part 1:

Here is how the STV works...

The user will specify a compression job dump folder. This folder will collect all XML job requests.

The user will specify a compression root folder. This folder should be under the users Library path somewhere. All compressed shows will go under this folder with another sub-folder matching its title. A .my file will be auto created for each job.

For a manual compress request, a user will simply hit the Compress button in the Recording Details menu. The user can also cancel this request if it has not started already.

A Favorite has a new Advanced setting. Under there you will see "Auto Compress". Set this to enabled to flag this Favorite for auto compression. When this is set, a new folder under the compression job folder that matched the description of the Favorite. Verify that this happens. At some point I will also store Favorite specific compression settings in this folder as well.

Whenever the RecordingScheduleChanged hook gets called Sage will look at your recent recordings and see if a matching folder exists under the Job folder. If it does, an XML compression job is created for that recording. Verify that this happens.

Compression options can be configured in Detailed Setup\Enhancements\Compress Options. They are limited right now but should get the job done.

STV:
http://download.sage.tv/~mlbdude/MlbDude2_Beta.zip

Part 2:

External compression program.

Alex has gone ahead and developed a nice program that will process these jobs using ffmpeg (the only command line tool we could find).

The install this program unzip it to the computer you want to handle the compression process. I use a client PC, but if the Idle issue is corrected you should be able to run it fine on the SageTV Server.
You can install this on multiple computers if you want all polling the same job folder for a more distributed setup.

To configure the compression program the following files must be updated for your setup.

ffmpeg_convert.properties - make sure the path to Java.exe is correct (or if it is in your path it should be fine)

There is one other property you need to modify that is in a file with the compressor. Modify the file properties.xml.

Configure the property:

<Pickup_Location>m:\compress_jobs</Pickup_Location>

P.S. The pickup job makes a log file in its directory called ffmpegPickup.log. Might be useful for debugging .

You can either run this manually at some time after some jobs are created, or you can do as I did and schedule it to run at some point overnight.

NOTE: The machine running this program must be able to see the SageTV MPEG files the same way Sage does. Just like SageRecorder, if on the Sage server a file is created at w:\sage\file.mpg this program will look for it at w:\sage\file.mpg.
So make sure you setup the machine accordingly.

http://download.sage.tv/~mlbdude/ffmpeg_pickup.zip

[edbmdave]

How does the FFMPEG job know where to look for files to process, does it check the Sage.Properties or the SageClient.Properties files?


I installed and can use the UI fine. The XML is created correctly in the Dump Job Folder. I can't tell that the Pickup job is running. I have manually invoked it and don't see anything. Java is in my path, and i also put the full path in the .properties file.

PS. Once you get this going, how about a folder for favorites that then breaks down into the episodes . .. IE. Simpons as a folder then all the Simpsons episode are under that?

[mlbdude]

Sorry, there is one other property you need to modify that is in a file with the compressor. Modify the file properties.xml.

Configure the property:

<Pickup_Location>m:\compress_jobs</Pickup_Location>

P.S. The pickup job makes a log file in its directory called ffmpegPickup.log. Might be useful for debugging .

[edbmdave]

With my limited Java skills (none) I can't reverse engineer enough of this to figure out why it isn't working. The STV portions appear to work fine. I can't get the ffmpeg_compress.exe to do anything. Again java is in my path,and I tried it by modifing the ffmpeg_convert.properties file (below)

C:\Program Files\Frey Technologies\SageTV\Convert>type ffmpeg_convert.properties

Java_Executable_Path=C:\Program Files\Java\j2re1.4.2_04\bin\java.exe
Class_path_to_use=.
Class_to_run=ffmpegPickup
Args_if_any=
Java_run_minimized_no_focus=true
Extern_run_minimized_no_focus=true


I have attached the .xml file created in the drop directory.
(in zip)

What am I not doing or doing wrong?

[mlbdude]

Your post is long so you may have been typing it when I posted above. If that still does not work, post your ffmpegPickup log file here.

Thanks for testing!

[edbmdave]

You were dead on. We cross posted. It looks like it is chugging away. Will report on the results.