SageTV Community  

Go Back   SageTV Community > General Discussion > General Discussion

Notices

General Discussion General discussion about SageTV and related companies, products, and technologies.

View Poll Results: How do you archive?
I delete stuff to make space, or buy more hard disks. 33 80.49%
I use mediashrink. 3 7.32%
I use handbrake. 5 12.20%
I use videoredo. 3 7.32%
Other -- please elaborate. 3 7.32%
Multiple Choice Poll. Voters: 41. You may not vote on this poll

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 05-22-2012, 09:17 AM
Dargason Dargason is offline
Sage Expert
 
Join Date: Oct 2003
Posts: 516
How do you archive?

I'm sure there are lots of different ways that people use to shrink recordings that they want to keep around. Over the years I've played with Mediashrink and Handbrake, and I've never been very happy with either. But it wasn't really a big issue because I could always dial back the HD-PVR recording quality for programs where it didn't really matter to me.

Now that I have a cablecard tuner I don't have that option... I have to either start deleting stuff faster, buy another hard disk, or bite the bullet and figure out how to get this 20GB recording of starwars down to a more manageable size.

In my case, I have a few requirements (or preferences):

- I want to keep closed captions.
- I would like to be able to comskip, or even edit out the commercials to save space, if it isn't too time consuming.
- I'd like to keep the DD 5.1 audio.
- I'd like it to play on my extenders just like my TS streams do.... the m4v and mkv files that I've produced using handbrake don't seem to skip forward/backward correctly. Maybe I'm using it wrong.

I guess what I'm saying is that I'd like to have a ts file, just a smaller one. (If someone can tell me how to make an mkv file act like a ts file, that's ok too...)
Reply With Quote
  #2  
Old 05-22-2012, 11:25 AM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,373
Most of my stuff does come from an HD-PVR so I don't bother archiving that. But kids shows do get archived - I comskip them and then run them through handbrake to convert to mp4 and finally copy them to a folder structure based on the show name. This also makes it easy to use the files on iDevices. I have written an SJQ process to handle all of this. I haven't gotten around to actually cutting out the commercials but that shouldn't be too hard to do with VRD.

I am pretty sure that I have no problems skipping around in mp4s in exactly the same way that I do in .ts files. My files don't have 5.1 audio but that shouldn't be an issue in Handbrake. I don't know about CCs as I never use them.

But to be honest the best option might just be to buy another hard drive now that prices have come back down. And a 2TB or 3TB drive holds a lot.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #3  
Old 05-22-2012, 11:48 AM
Dargason Dargason is offline
Sage Expert
 
Join Date: Oct 2003
Posts: 516
So you run comskip on the .ts file and then convert it? Will the edl file still work for the mp4 file?
Reply With Quote
  #4  
Old 05-22-2012, 08:21 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,373
In my case these are generally initially MPG files as they are generally SD, but yes, I run comskip on that and then the edl file still works on an mp4 file.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #5  
Old 05-22-2012, 09:59 PM
Fuzzy's Avatar
Fuzzy Fuzzy is offline
SageTVaholic
 
Join Date: Sep 2005
Location: Jurupa Valley, CA
Posts: 9,957
Sage's UI is what interfaces with the .EDL file, not the decoder, so it will work with and media type.
__________________
Buy Fuzzy a beer! (Fuzzy likes beer)

unRAID Server: i7-6700, 32GB RAM, Dual 128GB SSD cache and 13TB pool, with SageTVv9, openDCT, Logitech Media Server and Plex Media Server each in Dockers.
Sources: HRHR Prime with Charter CableCard. HDHR-US for OTA.
Primary Client: HD-300 through XBoxOne in Living Room, Samsung HLT-6189S
Other Clients: Mi Box in Master Bedroom, HD-200 in kids room
Reply With Quote
  #6  
Old 05-23-2012, 08:04 AM
fidget's Avatar
fidget fidget is offline
Sage Icon
 
Join Date: Jun 2003
Location: Cedar Rapids, Iowa, USA
Posts: 1,185
I copy my files onto slower (5900 RPM) long-term storage drives (2 TB). For TV series, they get replaced by the DVDs (encoded with Handbrake) when they are released. The other files get deleted, eventually.
__________________
Server: i5-2405S (4 core @ 2.5 GHz), 8GB RAM, NORCO RPC-4220 4U case
Tuners: 2 SiliconDust HDHomeRun , 2 Hauppauge HD-PVR Connected to 1 Pace700X and 1 TiVo Series 4
DVD Storage: 24 TB
TV Storage: 11 TB (4x1.5TB for recording, 5TB for archive)
Clients: 3
SageTV Extenders:5
Reply With Quote
  #7  
Old 05-23-2012, 08:40 AM
Dargason Dargason is offline
Sage Expert
 
Join Date: Oct 2003
Posts: 516
Quote:
Originally Posted by Fuzzy View Post
Sage's UI is what interfaces with the .EDL file, not the decoder, so it will work with and media type.
Good to know. I didn't know what the numbers in the file mean... are they time markers, or frame numbers, or something else?

I played around with handbrake a bit more last night. Either it has improved or I've become a bit more knowledgeable, because it wasn't quite as cryptic as I remember. I figured out how to add the right audio and CC tracks, and I'm working on improving the quality. The presets seem too focused on compression and the resulting quality suffers.

I've been mainly using the "Normal" and the "High Profile" profiles as my starting point. I like the "high profile" better, but it takes much longer to process.

To improve video quality, it looks like I can use "constant quality" and modify the quality setting. I tried moving this from 20 down to 10 and got video glitches in my test file. So now I'm trying to set an average bitrate. The default is 1500, and I tried 3000 which seem OK on my test file. I increased it again to 6000 and I'm going to try that next.

What settings do you guys use?
Reply With Quote
  #8  
Old 05-23-2012, 08:58 AM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,373
I think that I am pretty much using the iPad default settings with the excpetoin of turning on deinterlacing. Note that after you figure out what works for you if you run this from a batch file you will use HandbrakeCLI.exe and you will likely have to manually specify all of the parameters as (and this isn't well documented) the CLI does not accept custom presets.

Here is some info on edl files from the mplayer FAQ (presumably it is the same for Sage):
Quote:
The current EDL file format is:

[begin second] [end second] [action]
Where the seconds are floating-point numbers and the action is either 0 for skip or 1 for mute. Example:

5.3 7.1 0
15 16.7 1
420 422 0

This will skip from second 5.3 to second 7.1 of the video, then mute at 15 seconds, unmute at 16.7 seconds and skip from second 420 to second 422 of the video. These actions will be performed when the playback timer reaches the times given in the file.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #9  
Old 05-23-2012, 09:03 AM
Fuzzy's Avatar
Fuzzy Fuzzy is offline
SageTVaholic
 
Join Date: Sep 2005
Location: Jurupa Valley, CA
Posts: 9,957
That's correct. Sage, however, does not utilize the 'mute' flag, just the start-skip and end-skip times.
__________________
Buy Fuzzy a beer! (Fuzzy likes beer)

unRAID Server: i7-6700, 32GB RAM, Dual 128GB SSD cache and 13TB pool, with SageTVv9, openDCT, Logitech Media Server and Plex Media Server each in Dockers.
Sources: HRHR Prime with Charter CableCard. HDHR-US for OTA.
Primary Client: HD-300 through XBoxOne in Living Room, Samsung HLT-6189S
Other Clients: Mi Box in Master Bedroom, HD-200 in kids room
Reply With Quote
  #10  
Old 05-23-2012, 09:06 AM
Dargason Dargason is offline
Sage Expert
 
Join Date: Oct 2003
Posts: 516
Thanks! Eventually I might want to do all this with scripts, but I'm not that far along yet... I need to understand it much better.

I just read a thread on avsforums where they seem to be recommending a constant quality (RF) setting of 16, so maybe I was too aggressive when I set mine to 10. I'll try again with 16 when I get home and see how that turns out.

There were also some comments about audio sync. I didn't run any issues with that, so I'm keeping my fingers crossed.

That's interesting information on the edl file. So even after comskip does its thing, if I want to adjust a marker a second forward or backward, I can just edit the text file.
Reply With Quote
  #11  
Old 05-23-2012, 10:01 AM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,373
Quote:
Originally Posted by Dargason View Post
That's interesting information on the edl file. So even after comskip does its thing, if I want to adjust a marker a second forward or backward, I can just edit the text file.
Yep, and you can also have comskip produce .vrpj which are used by VideoRedo so you could automate comskipping, actually cutting commercials from the file (which might be dangerouse if comskip is inaccurate) and then reencoding.

One thing to try - if you have VRD then I believe you can also use it to reencode from a ts file in MPEG2 encoding to a H.264 file type such as mp4. And VRD also let's you do the editing or cutting. And it can also be run from a batch file.

In fact, here is a thread where someone has posted their automated process:http://forums.sagetv.com/forums/show...highlight=vprj

I occasionally have minor synch issues on my Handrake reencoded files but this doesn't always happen.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #12  
Old 05-23-2012, 11:36 AM
Dargason Dargason is offline
Sage Expert
 
Join Date: Oct 2003
Posts: 516
I don't have VideoReDo, but I was thinking about getting it, which is one of the reasons I started this thread. Being able to automatically cut out commercials would be great.

But while comskip works great on some channels/recordings, it doesn't do nearly well enough to automate, especially on movies, which is where I'd want it most. But I think VRD does have an editor of its own that would let you edit the .vprj files that comskip produces and 'fix' the problems that show up before you remove the commercials. I'm not sure how easy that is, but it would be a manual process in any case. And VRD isn't cheap.
Reply With Quote
  #13  
Old 05-23-2012, 11:46 AM
graywolf's Avatar
graywolf graywolf is offline
Sage Icon
 
Join Date: Oct 2009
Location: NC
Posts: 1,366
If you end up getting VRD, be sure to check the forum there and look into VAP add-on (user written/supported).

It helps automate many of the steps and make it more batch like.
Have it run the comskip to generate the vprj files, then at some time you can go ahead to check/modify the cut points on a bunch of the files. Then you start processing again and it'll perform the cuts on all the files where you have verified the cutpoints.
Reply With Quote
  #14  
Old 05-23-2012, 07:04 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,373
VRD costs about $75 but it is a very good program. It is quite easy to make edits and resave the file which it can do quickly without reencoding if you are saving to the same file type. If you are changing to a new file type then it may take a while but it may work for you as your encoding program as well.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #15  
Old 05-24-2012, 04:05 AM
ranger ranger is offline
Sage Aficionado
 
Join Date: Nov 2003
Location: Virginia
Posts: 256
I use VRD to edit the files and then compress with Handbrake. Using VRD is easy, but it can take some time if you have a large number of files to process. I make the required changes to the .vprj file generated by comskip and then use the batch function in VRD to save new files.

For Handbrake, I use the Normal setting modified just a bit. First I drop the framerate down to 29.97. If I leave it at 60, some of my clients will stutter a bit during playback. I use a constant quality between 19 and 21 depending on whether or not it is a favorite show. Typically don't touch the audio settings.

I have been usng mkv format, but am considering switching to mp4. Anybody have an idea of which might be more practical. Main use of the files is for playback on TV connected devices so I have been using mkv. Just wondering mp4 would give me any more flexibility.
Reply With Quote
  #16  
Old 05-24-2012, 09:58 AM
Dargason Dargason is offline
Sage Expert
 
Join Date: Oct 2003
Posts: 516
I'm glad you say that, because that's the direction I'm thinking of going.

So it would go like this:
Record show -> Comskip -> VRD to fix and remove commercials -> compress with VRD or Handbrake.

I tried the High Profile with quality setting RF=16 last night and the image looked good, but I had a corrupted timeline. I'm trying it again. The file was about 2/3s of the original size, which I think is pretty good. 5.1 Audio and CC were intact.

Is there a way to reconstruct a corrupted timeline?

Edit: Just did a quick search, and it sounds like VRD has a feature to fix a corrupt timeline... +1 for that!

Last edited by Dargason; 05-24-2012 at 10:00 AM.
Reply With Quote
  #17  
Old 05-24-2012, 10:11 AM
stanger89's Avatar
stanger89 stanger89 is offline
SageTVaholic
 
Join Date: May 2003
Location: Marion, IA
Posts: 15,188
FWIW, to the poll question, I don't archive, my TV recordings I just let roll off after I watch them (ie they're deleted), for movies, I get more space.

I find anything worth "archiving" isn't of "archival quality" from any broadcast source. It's either cropped/pan & scanned, edited, has logos burned in, poor audio quality (dynamic range compression) etc. I find anything I have worth "archiving" is worth buying on Blu-ray and not having to deal with cutting commercials and all the above issues.

The other thing is with the advent of services like Netflix/Amazon/etc, it's even less of an issue. I used to "save" (turned off auto delete) Top Gear, but since Netflix and Amazon have every episode from seasons 2-17, there's really no point in me keeping my (inferior) copies anymore.
Reply With Quote
  #18  
Old 05-24-2012, 11:09 AM
fidget's Avatar
fidget fidget is offline
Sage Icon
 
Join Date: Jun 2003
Location: Cedar Rapids, Iowa, USA
Posts: 1,185
My archiving is shows that we are waiting to get on DVD and programs that we have yet to watch. Since it is so easy to record things, my wife's list of things yet to watch is growing and growing (probably about 1.5 TB right now, all HD Dish shows).
__________________
Server: i5-2405S (4 core @ 2.5 GHz), 8GB RAM, NORCO RPC-4220 4U case
Tuners: 2 SiliconDust HDHomeRun , 2 Hauppauge HD-PVR Connected to 1 Pace700X and 1 TiVo Series 4
DVD Storage: 24 TB
TV Storage: 11 TB (4x1.5TB for recording, 5TB for archive)
Clients: 3
SageTV Extenders:5
Reply With Quote
  #19  
Old 05-24-2012, 06:01 PM
ranger ranger is offline
Sage Aficionado
 
Join Date: Nov 2003
Location: Virginia
Posts: 256
I do find that I am saving fewer and fewer shows because the availability through other sources like Netflix, but I still save a few because I'm too cheap to buy the DVD sets until they get discounted.
Reply With Quote
  #20  
Old 05-25-2012, 01:19 AM
david1234 david1234 is offline
Sage Aficionado
 
Join Date: Nov 2007
Location: Beaverton, OR
Posts: 312
Quote:
Originally Posted by stanger89 View Post
I find anything worth "archiving" isn't of "archival quality" from any broadcast source. It's either cropped/pan & scanned, edited, has logos burned in, poor audio quality (dynamic range compression) etc. I find anything I have worth "archiving" is worth buying on Blu-ray and not having to deal with cutting commercials and all the above issues.
The floating logos and the pop-up commercials over the top of shows drives me nuts. The worst offender is the ratings square that's stuck right in the middle, since they put it over the picture for a 4x3 crop. The other day we were watching something and it was right over the head of the guy who was talking!

If it weren't for the pop-ups and logos, what we record OTA is almost always better than what I get from DVD or Netflix's streaming service. Plus, as much as I really like Netflix, the I can't rely on it to ever have a specific show when we want to watch it.

For the couple shows we do hold onto, I just buy more disks for unraid, and move things over to the NAS.
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Auto Archive Wakecrash SageTV Beta Test Software 0 07-17-2010 01:58 PM
Moving from Archive, but still in archive HeyGuy101 SageTV Software 2 09-09-2007 11:30 AM
How to archive files on DVD? KarylFStein General Discussion 5 07-21-2006 07:41 AM
What are ppl using to archive shows!? jimbobuk SageTV Software 0 05-17-2003 12:37 PM


All times are GMT -6. The time now is 07:17 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.