(Old) MediaShrink - An HTPC focused encoder

[rmac321]

After hearing of the speed improvements in Handbrake 0.9.4, I figured I'd upgrade, so I figured I'd just upgrade to the newest mediashrink. My first file took way LONGER to encode than normal, so I thought I'd check in to see if anyone else has had similar results.

FWIW, I upgraded the mediashrink and dropped my old defaults file into the folder. I'll attach my defaults below in case I'm doing something that makes my setup slower than standard. I'm going to restart the computer and shrink a couple more files to see if this first run was anamolous.

my mediashrink.defaults.txt:

/defaultProfile autoEncode /findFileRegEx "avi|mpg|mkv|mp4|mpeg|VIDEO_TS|ts|ogm|divx|mpts|dvr-ms" /saveLog /cutComm /onlyWhenVprj

Edit:
see next post

[rmac321]

After shrinking a couple more files, the results are consistent. What used to be a 3.5 hour 2-pass shrink has now turned into a 5 hour shrink. The output file used to be around 500MB (std OTA 42 minute show) and is now around 900MB.

The difference must be in the auto encode profile. I don't understand the context of most of the file, but there is a new line that looks to test for HDTV content and set a higher quality.

 Encode CLI #6     =?>%%SNIP:HDTVCheck%%<:>/cliBitrate 2500 /twopass /highProfile<=>/onePass<?

What I don't completely understand is it appears to specify a bitrate rather than a quality level. I thought one of the new features of HB 0.9.4 was its improved ability to set a quality level and let HB vary the bitrate to meet the quality requested.

Just as an additional question to EP. IIRC, you've mentioned many times in the thread that you use one-pass encoding, yet the default setup for me has always run 2-pass. Should I be disabling 2-pass? The info on the HB site seems to indicate that 2-pass isn't necessary anymore. Any thoughts?

Thanks

[toricred]

I'm glad somebody else had the same speed issues. I noticed it being significantly slower and started altering my autoencode profile to simplify everything. In the end I decided to just buy more HDD space for Christmas.

[rmac321]

What I don't know yet, is whether the bigger files are worth it. I'll do some comparing. Frankly, the 1/2 GB mkv's are so much better than the avi's that I was happy with the old output. Maybe there will be some specific shows I might want to make a higher quality, but for most stuff, I'm just archiving.

FWIW, I dropped the HB 0.9.4 files into the old version of MediaShrink on my other server to do a comparison and it is running now.

Also, for those that have asked about the progress reporting. It appears to be the new version of MediaShrink that is no longer reporting, not the new version of HandBrake since I am getting status on this test. It looks like EP has changed the command calling the handbrake program.

Here is the way it is being called in the currently running MediaShrink ver 1:

- Executing command: start /D "C:\Program Files\mediashrink\mediaEngineBins\Handbrake 9.3" /BELOWNORMAL /WAIT /B HandBrakeCLI.exe -v -i "//server1/videos/Video_Work_Folders/4_Ready_To_Shrink/10x04 CSICrimeSceneInvestigation-CoupdeGrace.mpg" -o "//server1/videos/Video_Work_Folders/4_Ready_To_Shrink/10x04 CSICrimeSceneInvestigation-CoupdeGrace.workFolder/10x04 CSICrimeSceneInvestigation-CoupdeGrace.scratch.36.MAIN.mkv"  -e x264 -T -x ref=2:bframes=2:me=umh -b 1500 -B 160 -E ac3  -2  --detelecine --decomb --crop 12:12:15:17 -l 720

Here is the way it is being called in the currently running MediaShrink ver 2:

- Executing command: "C:\Program Files\MediaShrink\mediaEngineBins\HandBrake 9.4\HandBrakeCLI.exe" -v -i "//server1/videos/Video_Work_Folders/4_Ready_To_Shrink/103 Trauma-BadDayatWork.mpg" -o "//server1/videos/Video_Work_Folders/4_Ready_To_Shrink/103 Trauma-BadDayatWork.workFolder/103 Trauma-BadDayatWork.scratch.40.MAIN.mkv"   -e x264 -T -x b-pyramid:b-adapt=2:me=umh:subme=9:analyse=all:trellis=0 -b 2500 -B 160 -E ac3  -2  --detelecine --decomb --crop 12:12:15:17 -l 720   > "\\server1\videos\Video_Work_Folders\4_Ready_To_Shrink\103 Trauma-BadDayatWork.workFolder\103 Trauma-BadDayatWork.scratch.40.HandBrakeCLI.log" 2>&1

Is that a redirection command at the end of the second one that is responsible for the status being sent to the log file instead of to the console?

[rmac321]

Here is what the HandBrake site says:

----------------------- begin quote ----------------------
Constant quality encoding

No more looking for the perfect bitrate for a source--HandBrake is migrating to quality-based encoding. This means that instead of telling encoders to use a specific size and vary quality to meet it, we tell the encoder to vary size to meet a given quality level. Overall quality improves, since bits are spent only when they are needed, and are saved when they are not. While this means output size is somewhat unpredictable, the results in picture quality speak for themselves.

As part of this change, the quality slider has been made more prominent, and now works off the quality values used by the video encoders, instead of a confusing, custom, percentage scale.

Another result is that 2-pass encoding is not needed. A single pass at a constant quality provides just as much compression efficiency as two passes at an average bitrate.

-------------------- end quote ---------------------------------

They haven't updated the online documentation for the CLI version, so I don't know how the "slider bar" translates to a corresponding cli switch. Maybe EP can tell us more.

[rmac321]

EP,

Is this change to quality-measuring vs bitrate-measuring what you were contemplating when you proposed the vProfile and or /IVTC switches? Is this just a contemplation looking for input, or has this been implemented? I interpreted that post as a request for input.

My preference would be to have a way to specify quality. Maybe a few more levels of quality in the vProfiles would be useful. A step or two between Normal and HQ perhaps.

Thanks

Originally Posted by evilpenguin

I'm kicking around a much nicer set of CLI options that will allow for cleaner profiles. Let me know if anyone has any thoughts...

The biggest benefit is that I can prevent user supplied options from getting clobbered by the automatic ones (the reason /verticalScale wasn't working before).

Simple Usage: Drag and drop videos on MediaShrink.exe 
CLI Syntax  : mediaShrink.exe [options] <File|Folder> <File|Folder>

### Basic Options-----------------------------------------------------------

    /saveLog                Save log file when complete (default)
    /saveAll                Save all temp files when complete
    /batch                  Disable some features intended for running manually
                            that can get in the way of automated execution from
                            Dirmon2 or SJQ (default)

### Output Options-----------------------------------------------------------

    /inPlace                Replace the original video when complete
    /outputName <name>      Name for output file.  Default is same as original
                            with new extention.
    /outputDirectory <path> Directory to place encoded video.  Default is same
                            path as original.
                            
  /centralWorkFolder <path> By default all encoding work will be done in a 
                            directory next to the original video.  If you can
                            set this option to force mediaShrink to do all its 
                            work in a specified directory.  Useful if you have
                            recordings stored on a NAS.
                            
    /container <container>  A/V container to use
                              - mkv : Uses mkv container (default)
                              - mp4 : Uses mp4 container

### Video Options ------------------------------------------------------------

    /vcodec <encoder>      Set the video encoder
                    (Default) - x264   : Uses x264 encoder
                              - ffmpeg : Uses ffmpeg encoder, roughly divx

    /vbitrate <kbs>         Set video bitrate (Default 1500)             

    /vProfile <profile>     Set the video encoding parameters
                              - HQ      : High Quality, slow encode speed
                    (Default) - Normal  : Good Quality, resonable encode speed
                              - Apple   : Handbrake's Universal Apple video options
                              - iPhone  : Handbrake's iPhone video options
                              - iPod    : Handbrake's iPod classic video options 

    /onePass                Do a one-pass encode, faster
    /twoPass                Do a two-pass encode, slower/higher quality

    /xRes <###>             Scale video to ????x#### (Y Res is auto-calculated)
    /yRes <###>             Scale video to ####x???? (X Res is auto-calculated)

    /IVTC                   Uses Handbrakes variable frame rate encoding
    /noIVTC                 Blocks usage of IVTC by autoEncode
    
    /denoise                Uses Hanbrakes denoise filter
    /noDenoise              Blocks usage of denoise by autoEncode

    /decomb                 Uses Hanbrakes decomb filter (Selective Deinterlace)
    /noDecomb               Blocks usage of decomb by autoEncode

    /deinterlace            Uses Hanbrakes deinterlace filter
    /noDeinterlace          Blocks usage of denoise by autoEncode

    /noAutoCrop             Disable Automatic Cropping
    
    /noSubtitles            MediaShrink will automatically attempt to attach
                            all subtitle/CC tracks it finds since its a pretty
                            harmless operation.  Set this if you want to disable
                            
    /burnSubtitles          Burn subtitles straight right onto the video.

### Audio Options --------------------------------------------------------------

    /acodec <encoder>      Set the audio encoder
                              - faac   : AAC codec (Default Stereo/SDTV)
                              - lame   : MP3 codec
                              - vorbis : OGG codec
                              - ac3    : AC3 passthough (Default HDTV)
                              - dts    : DTS passthough

    /abitrate <kbs>         Set audio bitrate in kbs (Default 160)          
    


### DVD Options ----------------------------------------------------------------

    /forceTitle <#,#,...>   By default mediaShrink will automatically attempt to 
                            detect which titles to encode, however, every once
                            and a while it gets it wrong.  For those cases you 
                            can specify just the DVD title numbers you want to 
                            encode sepeated by commas.
                            
### Commercial Cutting Options -------------------------------------------------
    REQUIRES VIDEOREDO TO BE INSTALLED
    
    /cutComm                Attempt to cut the commercials.  If no edl or vprj
                            file exists it will attempt to make one using comskip
    
    /onlyWhenVprj           Only cut if a vprj file already exists (default) 
    
### Defaults -------------------------------------------------------------------

    HDTV (1080i or 720p):
      /container mkv 
      /vCodec    x264 
      /vBitrate  2500 
      /vProfile  HQ
      /xRes      720 
      /aCodec    ac3  
      /onePass 
    
    SDTV (480p) or SDTV in HDTV (Black Bars on the side:
      /container mkv 
      /vCodec    x264 
      /vBitrate  1500 
      /vProfile  Normal
      /xRes      540 
      /aCodec    faac 
      /aBitrate  160  
      /onePass

[perfessor101]

Hello EP ...
Thanks very much for this great encoding tool ...

I was up to about 7 of 9 GB before starting to use this ;-)
(unRaid 4.5)

I was curious if there was a way to set the horizontal cropping more conservatively it seems to be cropping the same amount on the top and the sides for some of my videos. (I seem to get one zealously over cropped video a day)

The vertical cropping most times seems to be just about perfect ...

Possibly a configurable max vertical and max horizontal crop width ... most of 'my' content seems to fall into the same range, either 4:3 broadcast or 16:9 fit into 4:3 (letterboxed?) ...

(and of course now I'm looking to go to HD content ... so that might be changing soon as I can figure out the options in Vancouver, BC ... either HD-PVR or r5000 ... for OTA I'm on the downside of the slope from most local channels)

Thanks for your time,
Bobby

[ohpleaseno]

I was discussing this over in the SJQ thread, but you were telling me about the 'No audio/video" issue being a problem with the source material. I used Mediainfo to look at one of the .mpg files that Mediashrink wouldn't convert and it showed that there wasn't an audio stream in the file, yet when I play the file it obviously has audio.

Why would it report no audio stream?

[razrsharpe]

Originally Posted by ohpleaseno

Why would it report no audio stream?

probably because the .mpg header is messed up either from the broadcast or from when sage wrote it. I have had instances where the recording info shows 2 audio streams but only one stream is present in the stream. This only happens on the local fox OTA affliate. The stream that actually contains audio is usually not the default stream... this causes mediashrink to use the stream with no audio when it processes the file.

[ohpleaseno]

Originally Posted by razrsharpe

probably because the .mpg header is messed up either from the broadcast or from when sage wrote it. I have had instances where the recording info shows 2 audio streams but only one stream is present in the stream. This only happens on the local fox OTA affliate. The stream that actually contains audio is usually not the default stream... this causes mediashrink to use the stream with no audio when it processes the file.

It just manages to kill the whole process for me and not do anything with it.

Is there any way to resurrect the header?

[evilpenguin]

Originally Posted by ohpleaseno

I was discussing this over in the SJQ thread, but you were telling me about the 'No audio/video" issue being a problem with the source material. I used Mediainfo to look at one of the .mpg files that Mediashrink wouldn't convert and it showed that there wasn't an audio stream in the file, yet when I play the file it obviously has audio.

Why would it report no audio stream?

You can run it through VideoRedo QuickStream Fix to solve that problem, I run into it occasionally as well.

[nyplayer]

EP,

This is a sample of a bad audio.

U:\testfolder>s:\SJQSAGE\tools\mediainfo\mediainfo.exe U:\testfolder\TheEllenDeG
eneresShow-5515388-0.mpg
General
Complete name                    : U:\testfolder\TheEllenDeGeneresShow-5515388-0
.mpg
Format                           : MPEG-PS
File size                        : 337 MiB
Duration                         : 13mn 22s
Overall bit rate                 : 3 523 Kbps

Video
ID                               : 224 (0xE0)
Format                           : MPEG Video
Format version                   : Version 2
Format profile                   : Main@Main
Format settings, Matrix          : Default
Duration                         : 13mn 22s
Bit rate mode                    : Variable
Nominal bit rate                 : 15.0 Mbps
Width                            : 528 pixels
Height                           : 480 pixels
Display aspect ratio             : 4/3
Frame rate                       : 29.970 fps
Colorimetry                      : 4:2:0
Scan type                        : Interlaced
Scan order                       : Top Field First
Bits/(Pixel*Frame)               : 1.975

Audio #1
ID                               : 0 (0x0)
Format                           : AC-3
Format/Info                      : Audio Coding 3
Format profile                   : TrueHD
Duration                         : 13mn 22s
Bit rate mode                    : Variable
Channel(s)                       : 2 channels
Channel positions                : L R
Sampling rate                    : 48.0 KHz
Video delay                      : 13mn 21s

Audio #2
ID                               : 129 (0x81)
Format                           : AC-3
Format/Info                      : Audio Coding 3
Duration                         : 13mn 21s
Bit rate mode                    : Constant
Bit rate                         : 192 Kbps
Channel(s)                       : 2 channels
Channel positions                : L R
Sampling rate                    : 48.0 KHz
Video delay                      : -578ms
Stream size                      : 18.4 MiB (5%)

Audio #3
ID                               : 130 (0x82)
Format                           : AC-3
Format/Info                      : Audio Coding 3
Duration                         : 13mn 21s
Bit rate mode                    : Constant
Bit rate                         : 128 Kbps
Channel(s)                       : 2 channels
Channel positions                : L R
Sampling rate                    : 48.0 KHz
Video delay                      : -623ms
Stream size                      : 12.2 MiB (4%)

The latest Comskip gives you the right PID in the VPRJ.

<Version>2
<Filename>TheEllenDeGeneresShow-5515388-0.mpg
<InputPIDList>
<VideoStreamPID>224</VideoStreamPID>
<AudioStreamPID>129</AudioStreamPID>
</InputPIDList>

what I do is that I check the vprj for the AudioStreamPID. I then execute a QFIX with my own script VPAC3.VBS with the correct PID.

EX. partial script below.

' Note: Use FileOpenBatch is you might be QSFing this file. Otherwise use, FileOpen
'
'openFlag = VideoReDo.FileOpen( sourceFile )
'openFlag = VideoReDo.FileOpenBatch( sourceFile )
openFlag = VideoReDo.FileOpenBatchPIDS( sourceFile, 224, 129 )

Is there a way you can do this. on the fly executing a quickfix?

PS I ran this file through mediashrink without a QFIX and it picked up SAP information audio.

You are using Comskip 0.79.144 can I just replace it with the current version?

[ChaOConnor]

EP,

I'm using the MediaEngine call you suggest for SJQ3, to alleviate the problem when nothing runs. Now I'm getting the attached. The job just fails out. Any ideas what i'm doing wrong? My client ruleset is:

MEDIASHRINKHQ {
:CPU "LOW"
:OFFHOUR "0-1, 16-23"
:MAX 1
:RESOURCES 51
#"C:/sjqc/MediaShrink/mediaShrink.exe \"%c%\" /inPlace /batch /highProfile /cliBitrate 2500"
"C:/sjqc/MediaShrink/mediaEngine.exe \"%c%\" /inPlace /mediaShrink /batch /saveLog /defaultProfile autoEncode /findFileRegEx \"avi|mpg|mkv|mp4|mpeg|VIDEO_TS|ts|ogm|divx|mpts|dvr-ms\""
}

[razrsharpe]

Originally Posted by ChaOConnor

EP,

I'm using the MediaEngine call you suggest for SJQ3, to alleviate the problem when nothing runs. Now I'm getting the attached. The job just fails out. Any ideas what i'm doing wrong? My client ruleset is:

MEDIASHRINKHQ {
:CPU "LOW"
:OFFHOUR "0-1, 16-23"
:MAX 1
:RESOURCES 51
#"C:/sjqc/MediaShrink/mediaShrink.exe \"%c%\" /inPlace /batch /highProfile /cliBitrate 2500"
"C:/sjqc/MediaShrink/mediaEngine.exe \"%c%\" /inPlace /mediaShrink /batch /saveLog /defaultProfile autoEncode /findFileRegEx \"avi|mpg|mkv|mp4|mpeg|VIDEO_TS|ts|ogm|divx|mpts|dvr-ms\""
}

can it find \\localhost? try using the UNC path instead of localhost... it seems to be complaining about no input file so i'd start looking there...

[ChaOConnor]

Originally Posted by razrsharpe

can it find \\localhost? try using the UNC path instead of localhost... it seems to be complaining about no input file so i'd start looking there...

localhost is what the %c% variable is returning. How do I force it to use UNC?

EDIT: would it be this:
"C:/sjqc/MediaShrink/mediaEngine.exe \"\\\\REBEL-HQ\\Recorded TV\\%f%\" /inPlace /mediaShrink /batch /saveLog /defaultProfile autoEncode /findFileRegEx \"avi|mpg|mkv|mp4|mpeg|VIDEO_TS|ts|ogm|divx|mpts|dvr-ms\""

When I put the above in, it now says "! couldn't understand (\\REBEL-HQ\Recorded TV\SidtheScienceKid-SpecialMomDayMeal-695310-0.mpg), throwing it away" but if I copy the exact path list into my run box, it loads the MPG fine. I'm really getting confused. Does MediaShrink not like UNC paths?

[ChaOConnor]

It doesn't show it in the failed log, but if I look at the log while it's running, it says it can't find the profile. It finds everything else, why not the profile? I've check, it's there.

[miegs3]

Hi guys. I'm new to this process so be gentle ;-)

I have run SageTV for a few years now after getting annoyed with the default app that came with my tuner, occasionly I would record but left as original. This year I have recorded a large ammount of video from different sources and looked into converting content for archiving. I found mediaShrink and it worked well with the defaults for my first video but then the next I tried handbrake appears to get stuck at a certain percentage (log just keeps outputting the same line after 5 hours or so).

I thought I read a post about using VideoRedo to quickfix if handbrake fails so I investigated and got that program but quick fix did not correct it. Is there some special settings I need to review or can I get a better idea of why handbrake is getting stuck?

Thanks for any help.

btw. I appreciate all the effort everyone goes to set these plugins etc up, they make life for everyone like me easy.

[ohpleaseno]

Originally Posted by ChaOConnor

localhost is what the %c% variable is returning. How do I force it to use UNC?

EDIT: would it be this:
"C:/sjqc/MediaShrink/mediaEngine.exe \"\\\\REBEL-HQ\\Recorded TV\\%f%\" /inPlace /mediaShrink /batch /saveLog /defaultProfile autoEncode /findFileRegEx \"avi|mpg|mkv|mp4|mpeg|VIDEO_TS|ts|ogm|divx|mpts|dvr-ms\""

When I put the above in, it now says "! couldn't understand (\\REBEL-HQ\Recorded TV\SidtheScienceKid-SpecialMomDayMeal-695310-0.mpg), throwing it away" but if I copy the exact path list into my run box, it loads the MPG fine. I'm really getting confused. Does MediaShrink not like UNC paths?

Don't change it in the SJQ command, change your recording directory in sage to the unc path. instead of the \\localhost\recorded tv have it be \\REBEL-HQ\recorded tv. that way it will pass the UNC to mediashrink

[sscumming]

I have been using MediaShrink successfully but now I have an issue where certain DVD's do not include audio. The major movie's provided by Netflix or Disney I have no problem, I get audio. When I process DVD's from my Camcorder or low budget production from my daughters dance company I get No audio, just video. Does anyone have any thoughts or ideas?

[evilpenguin]

Originally Posted by sscumming

I have been using MediaShrink successfully but now I have an issue where certain DVD's do not include audio. The major movie's provided by Netflix or Disney I have no problem, I get audio. When I process DVD's from my Camcorder or low budget production from my daughters dance company I get No audio, just video. Does anyone have any thoughts or ideas?

I'd need to see a log. See my sig for details on how to get one.