Why are my SageTV recordings fragmented?

[dotheDVDeed]

ALL (repeat ALL) my SageTV recordings are fragmented on my 64k formatted HD.

And the recordings seem to made up of what's close to 1 meg fragments. ie 4.9 GB file has about 4,900 fragments and a 6 GB file has about 6,000 fragments.

For the most part these fragmented recordings are playable on my HD extenders, but ah, this can't be a good thing, can it?

I'm using the standard SDTV theme, etc. Version 6.3.10.2--700 GB drive w/ 417 GB free.

TIM

[GKusnick]

If you record more than one program at a time, they're going to be fragmented, just because of the way disk allocation works. Or if your disk is close to full and it has to delete old programs to make room for new ones, you're probably going to get fragmentation as a result.

Fragmentation isn't necessarily a bad thing. The point of using 64K clusters is so that fragmentation won't matter so much, because the disk has plenty of time to seek to the next fragment while the previous fragment is playing back. (With smaller clusters it would have to seek a lot more often, and there wouldn't be enough time.)

Some people have argued that Sage should preallocate files in order to avoid fragmentation. That might help with playback (then again it might not), but it would probably be worse for recording multiple shows at once, because the disk would be seeking all over the place while writing to different files. You really want to minimize seeking during recording, to avoid buffer overruns and damaged recordings.

[dotheDVDeed]

Thanks Greg for the explanation. Seems to me there should be a "mini-app" included with SageTV that copies, defrags and replaces these files at a user-designated time when SageTV is neither serving or recording.

I've read other threads here where users have automated Videoredo to perform a quickstream fix on their recordings during downtimes. I can't help but wonder whether just a defragging of these recordings is all that's necessary in most cases.

TIM

[mdnttoker]

What about what app's like uTorrent does, allows you to allocate a large chunk of disk space at once?

Sage knows how long the TV show is going to be (non-live TV that is), and should be able to guestimate a disk allocation amount.

Why not allocate a large chunk @ the beginning, then fill it up as you go, like the uTorrent app does?

[Thomas Wischgol]

Originally Posted by mdnttoker

What about what app's like uTorrent does, allows you to allocate a large chunk of disk space at once?

Sage knows how long the TV show is going to be (non-live TV that is), and should be able to guestimate a disk allocation amount.

Why not allocate a large chunk @ the beginning, then fill it up as you go, like the uTorrent app does?

You still will end up with fragmentation once your hard drive gets full and Sage starts deleting old recordings. I do not think fragmentation is that much of an issue if you use 64K cluster size and have your system set up so that it uses caching (that is write caching - preferably write-back - as well as cached reads). Keep in mind that defragmenting the hard drive - either by copying files around or using a defragger - also increases wear because the drive will seek constantly during that process.

Thomas

[unkyjoe]

Ok, so any benefit to setting up 64k clusters on the DVD or Music store disks?

Or does it only benefit the recording drive - why do I ask? Well you see I formatted my DVD storage drives with 4k clusters instead of 64k.

Thanks for the help.

[rfutscher]

Each device has a I/O buffer. When a record buffer becomes full you want to empty it using one cluster. Like wise for a play back buffer. When a play back buffer becomes empty you want to fill it using the data from one cluster. If you have three tuners and three playback devices the heads are constantly moving to service the six buffers. This is why a fragmented file doesn't matter. The heads are always in the wrong place from servicing other buffers.

I would guess that the play back buffer sizes are the same for off air video and DVD playback. Therefore you would want the cluster size to be the same so that one cluster fills the playback buffer.

Originally Posted by unkyjoe

Ok, so any benefit to setting up 64k clusters on the DVD or Music store disks?

Or does it only benefit the recording drive - why do I ask? Well you see I formatted my DVD storage drives with 4k clusters instead of 64k.

Thanks for the help.

[BobPhoenix]

Originally Posted by GKusnick

Some people have argued that Sage should preallocate files in order to avoid fragmentation. That might help with playback (then again it might not), but it would probably be worse for recording multiple shows at once, because the disk would be seeking all over the place while writing to different files. You really want to minimize seeking during recording, to avoid buffer overruns and damaged recordings.

Originally Posted by mdnttoker

What about what app's like uTorrent does, allows you to allocate a large chunk of disk space at once?

Sage knows how long the TV show is going to be (non-live TV that is), and should be able to guestimate a disk allocation amount.

Why not allocate a large chunk @ the beginning, then fill it up as you go, like the uTorrent app does?

The reason you wouldn't want this is Greg's comment above.

[gplasky]

Originally Posted by unkyjoe

Ok, so any benefit to setting up 64k clusters on the DVD or Music store disks?

Or does it only benefit the recording drive - why do I ask? Well you see I formatted my DVD storage drives with 4k clusters instead of 64k.

Thanks for the help.

I'm running WHS for the Sage server and have 300GB left over from the primary disk which by default is formatted automatically to 4k. DVDs loaded on that drive play fine with no issues. My opinion is it is more important for the huge recordings. Especially HD. (Which mine are stored on drives NOT managed by WHS and formatted for 64K.)

Gerry

[Opus4]

The small clusters should be OK for fixed-content if the files are not fragmented & you are not reading multiple media files simultaneously from the same drive. If there were 2 clients trying to read from the same drive, that would raise the chance of stuttering playback because the drive would have to keep seeking between the 2 files & seeking to read a small cluster, then seeking to the next small cluster at another location is what the drive can't keep up with at times.

- Andy