General SageTV Recording Archiving/Space saving/Compression question...

[SomeWhatLost]

so, I have been a Sage user for a short while now (since V6) and I have seen people speak of "archiving" & compressing their recordings etc over those few years... and I always mostly ignored them because it seemed difficult and HDD's were cheap...
well.. now I find myself with 6TB of recording space, scratch that, I now have <400GB of free space, but my HDD are both less than a year old, I don't want to upgrade yet again, and I am out of MB sata ports to add a 3rd drive... there has got to be a better way...

so I finally have to start looking at real options on how to either shrink the file size of some of the existing recordings that I don't care about per se, but also do not want to delete, or I need to look at moving some recordings off to my drive pool (unraid, same box that is running sage)

just looking for pros/cons of the various options...
what are people doing and how well do they like it?
what all is involved with doing it?
and are there any guides, list of do's/do not's, etc that I can follow?

[phelme]

I think a lot of people here will say storage is cheap, just buy more.

I myself use Sage Job Queue (SJQ) and MediaShink (always using the latest HandbrakeCLI.exe) and compress stuff overnight. Depending on your source recordings, and what level of compression you can live with, you can probably save 20-50% or more per file and still retain very good quality. I back up everything to a Synology box as well.

[EnterNoEscape]

Originally Posted by phelme

I think a lot of people here will say storage is cheap, just buy more.

I agree with this philosophy unless you're archiving; then it might be worth the resources. When I used BeyondTV I had everything being converted to H.264 overnight to save on disk space and what I realized was that I was likely paying more on my electric bill than the cost of just getting another hard drive and the video was a little softer on playback which was a little disappointing. You can also argue that hard drive use electricity too, but not as much as transcoding.

[wayner]

How much disk space you can save depends on the original recordings. If you have lots of MPEG-2 recordings then you will be able to save a lot of space.

[phelme]

Originally Posted by EnterNoEscape

You can also argue that hard drive use electricity too, but not as much as transcoding.

I haven't measure my server's electricity usage in a while. Frankly I'm afraid too.

[pjpjpjpj]

I've always used the default Sage convert function. Every favorite can be set to automatically convert to another format and delete the original (it's an option on the favorite settings screen). You can even set specific folders where the conversion is saved so you can spread out your recordings over various drives for organizational reasons (if you wish). In the 5 or 6 years I've been doing it, the failure rate of the conversions is probably 0.0001 or something. It's surprisingly stable, consistent, reliable. Maybe twice a year does a conversion get messed up somehow (audio glitch or something).

I typically make things I want to keep "decent" into high-quality mkv, and my kids' cartoons into "good quality" mkv since they don't know/care. My recordings are OTA mpg, so I don't convert stuff I want to look "best".

You can also batch convert existing recordings. Focus on a recording group (without clicking into it) and choose either options or info (can't recall which). "Convert all recordings" is an option. It runs in the background while Sage chugs on (though I tend to do it overnight just to be safe).

[stanger89]

Personally I pretty much just watch and delete (or let delete). Stuff that I really like, I buy on Blu-ray. Broadcast quality isn't high enough, not to mention all the tickers, logos and other BS overlaid.

[phelme]

Originally Posted by pjpjpjpj

I've always used the default Sage convert function. Every favorite can be set to automatically convert to another format and delete the original (it's an option on the favorite settings screen). You can even set specific folders where the conversion is saved so you can spread out your recordings over various drives for organizational reasons (if you wish). In the 5 or 6 years I've been doing it, the failure rate of the conversions is probably 0.0001 or something.

Since the conversions are done immediately after the recording is finished, at least that's what I remember when I tried it years ago, I found this to be detrimental to system load when also running comskip etc. Glitches in newly started recordings while the transcoding was happening would always ensue. Or it would cause slowdowns of the UI during prime time watching on extenders and clients.

Now, I haven't tried SageTV's Automatic Conversion in my current system, which has way more horsepower than what I started with in the mid 2000's. I'm really comfortable with the delayed SJQ/HandbrakeCLI model, but maybe I'll give it another shot.

[SomeWhatLost]

Originally Posted by stanger89

Personally I pretty much just watch and delete (or let delete). Stuff that I really like, I buy on Blu-ray. Broadcast quality isn't high enough, not to mention all the tickers, logos and other BS overlaid.

oh, I do the same thing, it is just I have fallen way, way, way behind on the watch & delete side of things... like many TB's behind on the W&D....

[SomeWhatLost]

Originally Posted by pjpjpjpj

You can also batch convert existing recordings. Focus on a recording group (without clicking into it) and choose either options or info (can't recall which). "Convert all recordings" is an option. It runs in the background while Sage chugs on (though I tend to do it overnight just to be safe).

so, I just tried this, didn't work...
it queued up all the episodes to convert, then as it went to convert them it immediately failed and went on to the next episode that also immediately failed... on the bright side it only took a couple of minutes to fail at converting ~400 episodes...

the Sage log file was no help... it showed the episode being all queued up and being sent off to convert, then a supposed wait for 180 seconds (there is no way it actually waited...just zipped right through failing everything), but no mention in the log of that it failed, or why it failed...

could it be an unraid/docker permission thing?
or does the unraid docker lack the conversion doohicky?
but still wouldn't either of those show up as a failed "something" in the log?