Using an SSD (or 2) as recording only drive - then moving recording?

[mgpaulus]

with the 3D SSD drives (like the Samsung 850/860) becoming reasonably affordable in the 1GB range, I was thinking about putting 1 or 2 in my SAGE setup and using them as recording drives. But, here's where I'm not sure exactly how to go about this:

Does anyone know of or have a script/utility that will move the just recorded show off of the SSD onto my magnetic storage, that will balance the drives, etc?

I was hoping that by recording to SSD, I could help relieve some IO stress when I've got 4 or 5 tuners going at the same time (which is happening right now during the Olympics).

Thanks

Mark

[KarylFStein]

Originally Posted by mgpaulus

with the 3D SSD drives (like the Samsung 850/860) becoming reasonably affordable in the 1GB range, I was thinking about putting 1 or 2 in my SAGE setup and using them as recording drives. But, here's where I'm not sure exactly how to go about this:

Does anyone know of or have a script/utility that will move the just recorded show off of the SSD onto my magnetic storage, that will balance the drives, etc?

I was hoping that by recording to SSD, I could help relieve some IO stress when I've got 4 or 5 tuners going at the same time (which is happening right now during the Olympics).

Thanks

Mark

You might want to look into switching to unRAID. It does this and is well-supported by the SageTV development community. It can be daunting to overhaul a server, but something to consider. There's an unRAID subforum on here.

Otherwise I think some people use SJQ and a Groovy script to move files after recording, which is what I'm thinking about doing (along with making the file names so Emby recognizes them as TV recordings).

[Zogg]

My recording drive is a 250GB SSD for all recordings, and at 3AM a DOS batch file runs and moves the recording and comskip files to my NAS. It's pretty simple but it works.

I should probably add another drive for load balancing but so far it hasn't been an issue. I also run comskip on the recordings but limit it to 1 session at a time to make sure the drive I/O doesn't get overloaded.

[Telecore]

I'm not sure if you really need an SSD for 5-6 simultaneous recordings, usually two HDD's optimized for bandwidth will work - but you could try Stablebit Drivepool with an SSD cache. I use this on my desktop PCs (one SSD + two HDDs) and it has the added benefit of software raid to duplicate/back-up files. (For SageTV, I just use two x 2.5 inch HDDs @4TB each)

[newyankee]

I record to a 2T SSD, and manually transfer to a NAS every two weeks. Being conservative, I stop the Sage service, backup the wiz.bin file, move the files to the NAS, and then restart Sage.

[matt91]

Sage moves my recordings nightly to NAS via cron job. I don't stop/start sage, it just finds the files in their new location automatically.

(And then the NAS keeps the drives balanced.)

[benjamintm]

During premier season in the US, I regularly recorded 6-8 shows at the same time to a standard HDD, so an SSD would work.

That said, as others mentioned, you might want to look at Unraid. I switched over to it last year and it's been running perfectly ever since without an SSD in the mix.

Ben

Originally Posted by mgpaulus

with the 3D SSD drives (like the Samsung 850/860) becoming reasonably affordable in the 1GB range, I was thinking about putting 1 or 2 in my SAGE setup and using them as recording drives. But, here's where I'm not sure exactly how to go about this:

Does anyone know of or have a script/utility that will move the just recorded show off of the SSD onto my magnetic storage, that will balance the drives, etc?

I was hoping that by recording to SSD, I could help relieve some IO stress when I've got 4 or 5 tuners going at the same time (which is happening right now during the Olympics).

Thanks

Mark

[AlphaCrew]

Take a look at MCEBuddy.

[Zogg]

Originally Posted by benjamintm

During premier season in the US, I regularly recorded 6-8 shows at the same time to a standard HDD, so an SSD would work.

That said, as others mentioned, you might want to look at Unraid. I switched over to it last year and it's been running perfectly ever since without an SSD in the mix.

Ben

In the past, I've had issues when recording 4-6 shows at a time while also running comskip and playing back a recording.

Now this was with BeyondTV, and the solution was to have 2 recording drives. In BTV you could assign recordings to a specific drive, so for busy recording times I would move some recordings to an alternate drive. This also helped to spread the comskip load.

But since you can't specify a recording folder in SageTV, I just got an SSD that is fast enough to handle the load and limited comskip to only 1 instance. So far so good.

[Tiki]

Originally Posted by Zogg

In the past, I've had issues when recording 4-6 shows at a time while also running comskip and playing back a recording.

Now this was with BeyondTV, and the solution was to have 2 recording drives. In BTV you could assign recordings to a specific drive, so for busy recording times I would move some recordings to an alternate drive. This also helped to spread the comskip load.

But since you can't specify a recording folder in SageTV, I just got an SSD that is fast enough to handle the load and limited comskip to only 1 instance. So far so good.

Sage doesn’t let you assign specific recordings to specific drives, but it does have an option to spread recordings across all available drives to optimize for bandwidth. You can choose to optimize for bandwidth or for space - if you choose space it records everything to the drive with the most free space; if you choose bandwidth and if there are multiple recordings at the same time, the first goes to the drive with the most space, the second to the drive with the next most space and so on.

Of course I’m not sure how well that works if you are recording to something like UnRaid or tRaid. In those cases, even if Sage spreads the recordings around to multiple hard drives, the parity disk will get hammered and become the bottleneck.

[benjamintm]

I record to a cache drive in unRaid. The parity doesn’t get involved until the recording are moved to the array from the cache drive.

Ben

Originally Posted by Tiki

Sage doesn’t let you assign specific recordings to specific drives, but it does have an option to spread recordings across all available drives to optimize for bandwidth. You can choose to optimize for bandwidth or for space - if you choose space it records everything to the drive with the most free space; if you choose bandwidth and if there are multiple recordings at the same time, the first goes to the drive with the most space, the second to the drive with the next most space and so on.

Of course I’m not sure how well that works if you are recording to something like UnRaid or tRaid. In those cases, even if Sage spreads the recordings around to multiple hard drives, the parity disk will get hammered and become the bottleneck.

[Telecore]

Originally Posted by Telecore

I'm not sure if you really need an SSD for 5-6 simultaneous recordings, usually two HDD's optimized for bandwidth will work - but you could try Stablebit Drivepool with an SSD cache. I use this on my desktop PCs (one SSD + two HDDs) and it has the added benefit of software raid to duplicate/back-up files. (For SageTV, I just use two x 2.5 inch HDDs @4TB each)

Anyone using 2.5 inch HDD's+Windows - a lot of these want to park every 8s or so, which can lead to premature wear. To prevent, load CrystalDiskInfo at boot time to modify APM settings of drives from default of 0x80 to 0xC0..0xFE.

[mgpaulus]

Thanks all for the good info here.

I'm currently exploring using DrivePool for my 30TB of storage. I happened to have an extra (2) 120GB SSD drives that I put in as caching SSD drives in the DP pool. When I get some more cash, I'll probably replace those with a 512GB or 1TB 3D SSD, more for longevity sake than anything else. Still working on seeding all my recordings into the DP pool, but things look good so far.

Does anyone know if Sagetv will choke if 1 directory (/Videos) will have 10s of thousands of files in it? That is my one concern with using DrivePool. What's going to happen when I have a single "drive" with 24TB worth of recordings on it?

Anyway, thanks again for the feedback.

[mgpaulus]

Originally Posted by AlphaCrew

Take a look at MCEBuddy.

I already have a Plex server running on my unRaid box that connects to my Sagetv box that my kid and I both share, so I'm not sure what mcebuddy would do for me, but thanks for the input. I'll look at it again sometime.

[Telecore]

Originally Posted by mgpaulus

Thanks all for the good info here.

I'm currently exploring using DrivePool for my 30TB of storage. I happened to have an extra (2) 120GB SSD drives that I put in as caching SSD drives in the DP pool. When I get some more cash, I'll probably replace those with a 512GB or 1TB 3D SSD, more for longevity sake than anything else. Still working on seeding all my recordings into the DP pool, but things look good so far.

Does anyone know if Sagetv will choke if 1 directory (/Videos) will have 10s of thousands of files in it? That is my one concern with using DrivePool. What's going to happen when I have a single "drive" with 24TB worth of recordings on it?

Anyway, thanks again for the feedback.

I don't think that's a problem - I have a Windows Home Server 2011 with 30+ TB of Drivepool storage with thousands of music files and a lot of videos as well.

[texneus]

Originally Posted by Telecore

Anyone using 2.5 inch HDD's+Windows - a lot of these want to park every 8s or so, which can lead to premature wear. To prevent, load CrystalDiskInfo at boot time to modify APM settings of drives from default of 0x80 to 0xC0..0xFE.

I highly doubt that. If they park their heads its because they are designed to do so...let it do its thing. It's no different than using the same HDD in an actual laptop. I've used 2.5" drives for recording for many years, no modifications, no problems.

The specific problem your referring to is the WD green drives when used in Linux. That problem was only on Linux because one/some/most (not sure on that) distros felt they had to poll the drive for SMART data every 30 seconds or so, as if it really changes that fast. And even then it seemed to be mostly FUD, because some vocal people are under the belief the drive automatically dies once some specific number of cycles were reached.

[Telecore]

Originally Posted by texneus

I highly doubt that. If they park their heads its because they are designed to do so...let it do its thing. It's no different than using the same HDD in an actual laptop. I've used 2.5" drives for recording for many years, no modifications, no problems.

The specific problem your referring to is the WD green drives when used in Linux. That problem was only on Linux because one/some/most (not sure on that) distros felt they had to poll the drive for SMART data every 30 seconds or so, as if it really changes that fast. And even then it seemed to be mostly FUD, because some vocal people are under the belief the drive automatically dies once some specific number of cycles were reached.

I have read a lot of things on the internet about this and have experienced failures myself in my SageTV PC with Seagate ST4000LM024 or ST5000LM000 (4TB and 5TB 2.5 HDDs). Recently, I have put many of these into PCs that I have built and have had 2 drives fail with less than 2 years of use. Right now, I have 2 drives indicating >600,000 and >800,000 load/unload cycles on them and they are not that old. The manufacturers data sheet rates them for a max of 600,000 lifetime. In any event, they are parking heads a crazy amount of times per day. You can investigate this yourself and yes, I have had a couple of failures with limited use time so I do believe it is a problem.

I just discovered this in the past couple of days and now have a potential solution. The appeal of these drives is that you can buy them in USB enclosures for $99-$129 and take them out and use them (it voids the warranty). Performance is fine with SageTV (5900 RPM) and on desktop systems, I run Drivepool+SSD cache. I am presently running two PCs at work with these and 3 at home (SageTV, WMC and my desktop) all have a very large load/unload cycle count for the length of time that they have been in use.

By the way, the potential solution I have is to run hdparm, a free disk utility, to change the drive APM setting via task manager at startup and when resuming from sleep. This appears to cut down the # of load/unload cycles a lot.

[texneus]

You're sure those drives failed due to excessive head off loading? This would be the first confirmed case I know of. The life time in a spec sheet is a waranty parameter. It's a minimum number the drive is guaraneed to meet. Not sure where you read it was a maximum and the drive must fail at this number. Off loading the heads is not a tramatic experience, it's something the drive is designed to do as a matter of routine. I would expect the average drive to run well beyond this figure.

I have green that by all a counts should be dead. Originally used as a SageTV recording drive for 2-3 years, then as a server drive, and finally as my son's main HDD for many years. Still worked just fine when we pulled it, SMART says the drive is completely healthy.

Meanwhile at WD, Blue is the new Green. The new Blue drives don't work with WDIDLE which is opening this whole old wound once again. WD appears to be digging it's heals in this time...if they're wrong, I guess we'll all know soon enough. Got one myself I'm getting ready to install.

600,000 cycles in 2 years is ~6 cycles per minute asuing 24/7 operation. Defiantly not a typical HDD user.

[Telecore]

Honestly, not 100% sure it was due to excessive head parking because those failed before I even heard about this - but I have Stablebit Scanner installed on a couple machines and a few days ago it e-mailed me about the excessive load/unload cycles and indicated that the drive could be about to fail, which started my investigation, at which point I discovered a 2nd drive had even more load/unload cycles on my desktop (but apparently, for that one, I clicked on something to ignore the warning). For now, I have decreased the cycles at the expense of keeping drives spinning longer, more power and heat, but it also makes all the PCs more responsive - less waiting on drive spin-up - I do believe the default drive parking was excessive, probably because these drives came from a portable USB enclosure. I am going to keep an eye on this and continue to use the drives in the meantime. Last time I had a failure, a few months ago, I lost 1/2 of my SageTV recordings, so I am a bit concerned about re-purposing these drives.

[texneus]

Sorry to hear of that. Failures are guaranteed. As they say...In God we trust, everything else gets a backup