What format to archive recordings in?

[wildgoose]

I am beginning to accumulate TV recordings that I want to keep rather than watch and delete. I am wondering if there’s a better format to keep the recordings in to save HD space, and preserve the information about the recording SageTV displays in the UI too.

The current recordings are mpg files from an HDHomeRun tuner. They are about 4-5G for an hour of 1080i recording.

So do you post process the recordings if you want to keep them around? Or just buy a bigger HD?

[bkirk]

I use VideoReDo to remove commercials and correct any stream errors followed by HandBrake to convert to H.264. The size of the final output will depend on what quality you're willing to live with. I can take a ~1.5 hour Formula One race down from 12000MB to about 600-700MB by downscaling to 700x400 and setting the average bitrate to 750kbps. If you leave the resolution at 1080 you'll probably want a bitrate over 1000kbps, and even at 1500kbps you're looking at around 1GB per hour of video.

[SWKerr]

Do you intend to play them on any other device or just have a copy to play back in SageTV at a later date?

mp4 is probably the most commonly supported across all devices but many can not handle certain bit rates etc. I have never found anything that is perfect. If you have anything apple I would use mp4 but most other vendors play nice with other formats.

I would not cut out commercial myself as I often have errors in the comskip marking. It is probably 90% correct but that 10% may be important.

There are a number of easy to use options that are built into SageTV compression as well. You can test them and see if the quality is to your liking just make sure you check the keep the original recording until you have decided on a standard of your liking.

MediaShrink does a very good job. (search the forum) Just drag the file onto the executable and it runs. Lots of options but no GUI. I have never found something I think is perfect.

[wildgoose]

Thanks for the suggestions. If I use handbrake to reprocess the videos, will it preserve the meta-data Sage keeps in the file? (things such as episode highlight, air date info, etc..), what about closed captioning?

When I re-installed sage on a clean new machine, it found the recordings and the info are there, so I assume it's encoded in the file somewhere. Preserving these info will be ideal.

I may need to playback the files on other devices, though right now I only have Sage HD300. Can you elaborate on how I can do the compressing in Sage itself?

Will try Handbreak..

Thanks!

[firenice]

Originally Posted by wildgoose

Thanks for the suggestions. If I use handbrake to reprocess the videos, will it preserve the meta-data Sage keeps in the file? (things such as episode highlight, air date info, etc..), what about closed captioning?

When I re-installed sage on a clean new machine, it found the recordings and the info are there, so I assume it's encoded in the file somewhere. Preserving these info will be ideal.

I may need to playback the files on other devices, though right now I only have Sage HD300. Can you elaborate on how I can do the compressing in Sage itself?

Will try Handbreak..

Thanks!

+1 on VRD (VideoRedo)
I archived shows from last season that averaged 3.5 to 5 GB each to less than a GB using VRD and a 3rd party program called VAP. Using a MKV profile at 1280x720 with an average bit rate of 2500 and max of 4000. Good thing about these mkv's is that I can transcode to mp4 or any other format when required. VAP will work with HandBrake also.

Good Luck
Mike

[bkirk]

VideoReDo claims to preserve TiVo and BeyondTV meta-data, but doesn't say anything about SageTV meta-data. The top-of-the-line version of VRD can also do the conversion to H.264, so if it can handle the meta-data, that might be your all-in-one solution.

I'd be very surprised if HandBrake passed the meta-data on to the newly encoded H.264 file. In that case, your only hope is if SageTV updated the meta-data in the file from the wiz.bin database. Someone with a lot more SageTV experience than me will have to tell you if it will do that. I still haven't figured out how to see what meta-data is in the recording file in the first place.

[stanger89]

Originally Posted by wildgoose

Thanks for the suggestions. If I use handbrake to reprocess the videos, will it preserve the meta-data Sage keeps in the file? (things such as episode highlight, air date info, etc..), what about closed captioning?

I'm not sure anything but an MPEG transport stream supports closed captioning.

[razrsharpe]

Originally Posted by bkirk

I still haven't figured out how to see what meta-data is in the recording file in the first place.

use a hex editor and look at the end of the file... it should be stored in plane text there.... good luck finding an editor that can open multi gigabyte files though...

[drewg]

Originally Posted by razrsharpe

use a hex editor and look at the end of the file... it should be stored in plane text there.... good luck finding an editor that can open multi gigabyte files though...

Standard linux hexump works fine, but its a bit awkward to figure out how many bytes to skip. Eg:

% hexdump -s 5023210550 -C /video/tv/PrimeSuspect-S01E09-AGorgeousMosaic-2724430-0.mpg | head
12b681c36  98 9f 11 ee 50 23 f8 06  f9 27 30 76 80 4c e1 23  |....P#...'0v.L.#|
12b681c46  a2 62 3b 0d fe 6d 57 51  d1 b1 3e e4 c3 32 13 4e  |.b;..mWQ..>..2.N|
12b681c56  a0 3f 19 c0 90 26 ff eb  82 68 02 12 c8 16 61 5c  |.?...&...h....a\|
12b681c66  13 3f f4 13 c0 10 96 40  bb 88 34 16 76 40 00 c1  |.?.....@..4.v@..|
12b681c76  a8 b3 83 4e 61 2c 41 83  51 80 0f 51 b0 26 00 1b  |...Na,A.Q..Q.&..|
12b681c86  40 1a 75 b8 1a 8c 00 00  01 be 02 c3 4d 45 54 41  |@.u.........META|
12b681c96  41 69 72 69 6e 67 44 75  72 61 74 69 6f 6e 3d 33  |AiringDuration=3|
12b681ca6  36 30 30 30 30 30 3b 41  69 72 69 6e 67 54 69 6d  |600000;AiringTim|
12b681cb6  65 3d 31 33 32 34 30 30  34 34 30 30 30 30 30 3b  |e=1324004400000;|
12b681cc6  48 44 54 56 3d 74 72 75  65 3b 43 43 3d 74 72 75  |HDTV=true;CC=tru|

[KryptoNyte]

Originally Posted by firenice

+1 on VRD (VideoRedo)
I archived shows from last season that averaged 3.5 to 5 GB each to less than a GB using VRD and a 3rd party program called VAP. Using a MKV profile at 1280x720 with an average bit rate of 2500 and max of 4000. Good thing about these mkv's is that I can transcode to mp4 or any other format when required. VAP will work with HandBrake also.

Good Luck
Mike

Same here. I use MKV for all archiving, mainly due to it being the only h.264 container that preserves DTS audio.

Takes me about 30 to 60 seconds per show to ensure commercials are cut properly.