SageMC Video Tools 3.5 (Development Thread)

[evilpenguin]

Don't worry though, I haven't abandoned this project as both will tools serve a purpose in my greater plan

Big picture wise I'm trying to figure out what tools i'll need to solve a few closely related problems...

1. I'm ripping/encoding all of my TV/Movie DVD's into Sage. What's the easiest way to do it and how to I keep them organized?
2. If I have some episode of a show in Sage Recordings and some episodes as imported videos how do I get them to show up together?
3. How can I improve the experience when I'm navigating my imported videos so that it feels more like I'm navigating Sage Recordings?

So while Video Encoding is important, I still need a few extra pieces to bring it all together and metadata scraping was one of them. Work wise mediaScraper is a pretty small tool with a very manageable todo list so i'm hoping to be mostly done with it this month.

[Brent]

I'm looking forward to it. Time is a precious commodity so I certainly understand. The big picture ideas you're talking about sounds pretty interesting...

[lakeman]

I like big picture views...

Now if i could only figure out how to fit that new 52" HDTV into the budget!

[Jetracers]

Hey EP,
Any chance you can help me modify a profile to shoot for a target file size like in handbrake. I would like to shoot for 175MB for hr long SD programing and 30 Min HD shows and 350 MB for most other HD programming. Thanks in advance.
Jet

[evilpenguin]

Sure, all you'll need to do is add this line to Auto Encode.profile...

Encode CLI #9     =?>%%SNIP:HDTVCheck%%<:>/cliBitrate<&>800<=>/cliBitrate<&>400<?<&>/highProfile
Encoder #9        =setOptions

Those bitrates are pretty low so I also threw in /highProfile so the encodes don't look too terrible.

[toricred]

Now any news on the import for this version?

[evilpenguin]

I'll be honest, i'm pretty much done with my perl2dvd codebase. It just has way too much legacy code so working with it is starting to be a chore and pulling it out would take forever. My new plan is just to shift my focus to my metadata scraping tool.

Fortunately, if you've checked out mediaScraper, you'd have noticed that the profiles I use for scraping look suspiciously like my encoding profiles. That's because the entire tool is basically a fresh rewrite of the profile reading/executing engine from this tool. All it needs is a few more tweaks and I'll be able to start using it to encode videos using the exact same profiles I use now. It'll take a little longer but it will be *much* better for me.

[lakeman]

EP, given the new direction, can you comment on the usability of the DVD burning ability in Video Tools 3.0? Last I saw it was iffy. Correct? I am hoping to increase the WAF and easy click DVD burning is one sure way to get me some points. Any comments or direction will be most appreciated!

Thanks!

[toricred]

EP,

I completely understand the change of direction and I know that sometimes it's the only way to get a second chance. I hope you get a chance to set up the encoding again soon. I'm using the built-in compression right now and I'm not too happy with it. I've thought about going back to version 3.0, but I had hope 3.5 would be out really soon and I was going to wait. Now I'm really torn on what to do.

[Brent]

[Jetracers]

Originally Posted by evilpenguin

Sure, all you'll need to do is add this line to Auto Encode.profile...

Encode CLI #9     =?>%%SNIP:HDTVCheck%%<:>/cliBitrate<&>800<=>/cliBitrate<&>400<?<&>/highProfile
Encoder #9        =setOptions

Those bitrates are pretty low so I also threw in /highProfile so the encodes don't look too terrible.

I added this code to my autoencode.bat but it didn't do what was intended could you take a look at what i have and let me know what i did wrong

[razrsharpe]

I have been following the 3.5 development for awhile now and am excited to start using it from within SageMC. Does anyone know if it is possible to use the front end (STVi) from 3.0 with the encoding code from 3.5 (using dirmon2 to schedule the encodes)? I haven't yet had a chance to configure it and try it out so figured I would ask and see if anyone else has tried it. I would think it should work because the front end just passes some arguments to dirmon2 to run tv2pc.exe (maybe a custom or modified bat is needed....).

[razrsharpe]

EP,
Quick question... I started using this version to start doing my Sage encodes from outside SageTV. I wrote 2 .bat files to semi automate the process. The first one makes a file that Dirmon2 can monitor. The second one starts tv2pc.exe using the first file as an input.

Now to my question: I have the /deinterlace and /reversetelecine switches set in the .profile file. My understanding is the /reversetelecine switch should reencode the video to 24fps if it is IVTC (hdtv) content. However, when I encode video i end up with a mkv that is the same fps as the original but progressive (if it was interlaced). When mencoder (previous version) was doing the encodes i ended up with video that was 24fps. Am i missing something? Or where can i add the appropiate switches to make the fps the original 24 when the /reversetelecine switch is active?

Thanks for the help

[evilpenguin]

Originally Posted by razrsharpe

My understanding is the /reversetelecine switch should reencode the video to 24fps if it is IVTC (hdtv) content. However, when I encode video i end up with a mkv that is the same fps as the original but progressive (if it was interlaced). When mencoder (previous version) was doing the encodes i ended up with video that was 24fps. Am i missing something? Or where can i add the appropiate switches to make the fps the original 24 when the /reversetelecine switch is active?

Thanks for the help

That's my bad, I was using detelecine incorrectly in Handbrake. I've got a fix for that but the problem is it only works for 1080i and not 720p.

Now, I could either handle this two ways:

1) Use mencoder for 720p encodes
2) Not detelecine 720p encodes

I'll prolly use #1 if you explicitly set /detelecine but for AutoEncode I'll probably go with #2 because IMHO it has fewer problems and looks great.

[lakeman]

I read somewhere... That this code / tool was being integrated into the mediaScraper. How is that work progressing?

Thanks


Love the work!

[razrsharpe]

Originally Posted by evilpenguin

That's my bad, I was using detelecine incorrectly in Handbrake. I've got a fix for that but the problem is it only works for 1080i and not 720p.

Now, I could either handle this two ways:

1) Use mencoder for 720p encodes
2) Not detelecine 720p encodes

I'll prolly use #1 if you explicitly set /detelecine but for AutoEncode I'll probably go with #2 because IMHO it has fewer problems and looks great.

I've actually started forcing Mencoder with a customized AutoEncode profile because it does the detelecine correctly, seems to be faster than handbrake, looks great and doesn't have any of the earlier problems with the audio and/or green bars when the /copyAudio switch is on (and I don't seem to have any lipsync issues (at least not yet).

I tried to figure out the CLI for handbrake (used a combination of the one reported from the log and the handbrake CLI guide) and thought i was setting the options to detelecine correctly, but i could not get it to output a 24fps progressive picture (they were all the original frame rater, all 1080i content).

I think I found where you set the detelecine option for handbrake and the bug. In handbrake options.snip:

# Snippit: HandBrake Video Filters
#
# HandBrake's video filters and misc. options
#

Profile =Handbrake Video Filters
Encode CLI #1 =?>deinterlace<:>--decomb<? ?>!onePass<:>-2<? ?>deNoise<:>-8<? ?>reverseTelecine||isDVD<:>-vfr ?>!deinterlace<:>--decomb<?<? %%SNIP:Handbrake AutoCrop%%
Encoder #1 =HandBrakeCLI.exe

I think the -vfr should be --detelecine. However, this has no effect on the output (eventhough i believe it should). I think the problem is in Handbrake not doing the detelecine correctly.

Anyway, lots of rambling and I have a workaround, but wanted to share my research. Thanks for the great tool

[evilpenguin]

Originally Posted by razrsharpe

I've actually started forcing Mencoder with a customized AutoEncode profile because it does the detelecine correctly, seems to be faster than handbrake, looks great and doesn't have any of the earlier problems with the audio and/or green bars when the /copyAudio switch is on (and I don't seem to have any lipsync issues (at least not yet).

Are using a newer version of mencoder than the one in my download? Cause I always get very small, but annoying, lipsync issues with mencoder/detelecine/copyAudio.

Originally Posted by razrsharpe

I think the -vfr should be --detelecine. However, this has no effect on the output (eventhough i believe it should). I think the problem is in Handbrake not doing the detelecine correctly.

Yeah, that's the mistake i'm talking about, I haven't checked in the fix yet, but you can just change it in your own code.

So here's the deal with variable frame rate detelcine in handbrake.

1) It only works on 1080i due to the way the pullup filter works.
2) Due to the frame rate being variable, normal frame rate detection doesn't work and almost every tool will just report the video as being 29.97 fps. But if you playback the video in Quicktime (try it with an mp4 encode) which can report the instantaneous frame rate, you'll see that the actual frame rate is indeed constantly changing and will vary anywhere from 20-30 fps depending on the type of content being played back. (It took me a while to figure this one out too )

[razrsharpe]

Originally Posted by evilpenguin

Are using a newer version of mencoder than the one in my download? Cause I always get very small, but annoying, lipsync issues with mencoder/detelecine/copyAudio.

I am using the version from your download dated 11/29/2008. To be honest, I haven't watched something compressed yet start to finish (it currently is working on a backlog of old shows). I just skipped through the first couple encodes and watched for a minute here and there (and the end of the file) to make sure it was working as expected. Everything seemed ok. When i get a chance (hopefully tonight) I will watch something start to finish and let you know the results.

I actually went looking for a new mencoder download and couldn't figure it out. I was too tired to follow all the instructions on the mencoder website. And there was no simple download this and run it file... that I could find anyway... I am sure I'm missing something silly.

I updated my handbrake options.snip with --detelecine. Thanks for confirming that was the bug.

So here's the deal with variable frame rate detelcine in handbrake.

1) It only works on 1080i due to the way the pullup filter works.
2) Due to the frame rate being variable, normal frame rate detection doesn't work and almost every tool will just report the video as being 29.97 fps. But if you playback the video in Quicktime (try it with an mp4 encode) which can report the instantaneous frame rate, you'll see that the actual frame rate is indeed constantly changing and will vary anywhere from 20-30 fps depending on the type of content being played back. (It took me a while to figure this one out too )

AHHHHH.... I See. Thanks. That makes a lot more sense now. So my question is what does the sagetv hd-extender (or for that matter any playback device) actually output. I would think it is only capable of outputting a single video type (resolution, framerate combination)? How does it handle the constantly changing fps of the original stream? Maybe it all doesn't matter but I think it would.

[evilpenguin]

Originally Posted by razrsharpe

I am using the version from your download dated 11/29/2008. To be honest, I haven't watched something compressed yet start to finish (it currently is working on a backlog of old shows). I just skipped through the first couple encodes and watched for a minute here and there (and the end of the file) to make sure it was working as expected. Everything seemed ok. When i get a chance (hopefully tonight) I will watch something start to finish and let you know the results.

Thanks, i'd appreciate the 2nd opinion. Sitting around and watching videos over and over looking for a/v sync issues is the *worst* part about writing this tool.

Originally Posted by razrsharpe

I actually went looking for a new mencoder download and couldn't figure it out.

Here's a new one i've been using

Originally Posted by razrsharpe

Maybe it all doesn't matter but I think it would.

Nope, it shouldn't matter, I haven't had any trouble playing back vfr mkv files on my HD200.

[razrsharpe]

So I finally got around to watching one of the encodes (recent episode of Numb3rs). The video was encoded with a customized autoencode profile (attached) using the mencoder from your download (from nov 08). I did not notice any significant AV sync issues. Occasionally I thought I saw something very minor, but I it wasn't any worse/different than any other video. I am usually *VERY* anal about sync and the show was not distracting/annoying to watch at all. All around I say the encode looked fantastic and played great. I do not do any commercial cutting (my comskip setup isnt rocksolid enough and I haven't spent the time to make it 100%). I can definitely see commercial cutting introducing AV sync problems.

Thanks for the link to the updated mencoder. I will try it out.