Should I defrag my drives?

[rdefino]

I have all my recordings on a raid 5 set of 3TB. Now since I'm recording and deleting shows alot, I guess I would have alot of fragmenting. Is it wise to defrag? Can it be done safely on a raid volume?

Thanks

[StephaneM]

Hello,

Originally Posted by rdefino

Is it wise to defrag? Can it be done safely on a raid volume?

You can defrag safely (though my RAID5 array crashed 3 weeks ago during defrag but fortunately no data was lost).

But I don't think it is worth it. It will make your drives suffer a lot : defragging Gigs of data is very intensive especially with a RAID5 array and if your block size is a big one well fragmentation is not so a problem.

So basically defragging your drives will reduce their life a lot.

However it's wise to defrag your drives right after Windows setup and application installation.

Regards,
Stéphane.

[blade]

Some people swear defragging improves performance for them. I've heard stories about sluggish menus and the spinning Sage icon going away after defragging the recording drives. Some people claim it fixed their stuttering recordings (when already using 64k clusters).

I've had Sage setup for nearly 2 years and haven't defragged once. Many others have been running even longer and never defrag. I have no response problems, stuttering, etc... IMO if you aren't having performance problems don't bother.

Typically someone will tell you how defragging reduces the wear on your drives. I would agree with this if you're storing recordings for long term use; however, for recordings you don't intend to keep you're creating as much if not more wear on the drive by defragging.

[nielm]

I defrag maybe once every 6 months or so... I occasionally run the report, and when I see that loads of files have several tens of thousands of fragments, I think that its time for a cleanup...

The problem is that Window's defragger is not very good with large files and lots of fragments, so I use a combination of it and SysInternals 'Contig'

My defrag batch file follows (my recording drives are e: and g

defrag g: -f -v
Contig.exe -s g:\*.*
defrag g: -f -v
Contig.exe -s g:\*.*
defrag g: -f -v

defrag e: -f -v
Contig.exe -s e:\*.*
defrag e: -f -v
Contig.exe -s e:\*.*
defrag e: -f -v

[Pumpkinhead]

The general rule is that you should never defrag a drive unless you have a full backup.

Having said that, I never defrag video drives, whether for Sage, Video editing, etc. I only defrag my system drives every several months. Using timed benchmarks, there is sometimes an ever so slight performance increase in launching an application by defragging the system drive.

[DIBU]

i use this:

http://www.iobit.com/SmartDefrag/download.html

and i have never a problem with this product (freeware) on my server

[Polypro]

Until recently I used Diskeeper 10's "Set it and forget it" to defrag all my drives. It did a good job and never interfered with anything. With Diskeeper 2007's "Intellisense", the CPU is constantly used for defragging...I didn't like it. I switched to Raxco Perfect Disk 8.x and like it better. With 64k clusters I guess you don't need to defrag, but I do anyway. I also use Sprinrite 6 every few months as preventative maintenance.

P

[Mark SS]

Originally Posted by StephaneM

So basically defragging your drives will reduce their life a lot.

Stephane that is absolute nonsense. A typical user who keeps some of his files for long periods, records and deletes other frequently, will reduce more than likely reduce the distance his drive heads travel over the long term.

That said, defragging or not is unlikely to have any measurable effect on the lifespan of a drive.

[blade]

Originally Posted by Mark SS

Stephane that is absolute nonsense. A typical user who keeps some of his files for long periods, records and deletes other frequently, will reduce more than likely reduce the distance his drive heads travel over the long term.

The argument about lifespan and defragging or not defragging has been going on for ages and I highly doubt it will be settled here.

A recording drive isn't anywhere near typically usage. For one you're using sectors that are 16 times larger than what is typical. Also often times recordings are only accessed once or twice before being deleted. You have to remember the heads are having to jump around during the defragging process to move gigabytes of data around compared to reading once and then deleting. Also when defragging you're reading and re-writing to the disk. So yes you may save a little wear on the heads, but you are without a doubt increasing the number of reads and writes to the disk.

That said, defragging or not is unlikely to have any measurable effect on the lifespan of a drive.

That is the age long debate. Personally I just don't see the need to rearrange gigabytes worth of data if there are no performance benefits and if as you say there isn't any measurable effect on lifespan either way why bother?

[StephaneM]

Originally Posted by Mark SS

Stephane that is absolute nonsense. A typical user who keeps some of his files for long periods, records and deletes other frequently, will reduce more than likely reduce the distance his drive heads travel over the long term.

Well, I have 32 computers here and believe me or not, the computers who drives are failing every two years are those where the drives are heavily used and the PC that do extensive usage on HDD here are my Media PCs and Backup PCs. All other PC are servers (Web Servers / Mail Servers / Database Server).

I'm not saying that defregmentation is reducing disk drive life. I'm saying that defragging hard drives that are part of a RAID5 array and containing files that are GB sized is reducing the hard drive life. After all a typical defrag for this kind of setup result in hard drive intense activity for about 2 hours... (this will be especially worse if the drivers are stacked and not cooled enough).

So you can really trust me on this : I do have drives on servers that are 10 years old or more. But drives on my media PCs will last two or three years (I was defragging drives weekly). Now I don't defrag anymore because I really do think that :

1) it does not help for video playback quality (or you would have to defrag after recording and before watching)
2) it spares the drive, after all I only watch my recording once and they get deleted and recordings that I want to keep I recompress them in Xvid and the resulting Xvid file is never fragmented. So regarding the overal head travel distance thing : not defragging for me really reduce the overall load on the drives.

But for any other configurations deffraging doesn't impact at all drive life (I defrag my servers).

In my opinion the best setup for HTPC should be :

* A RAID5 array for OS and recordings
* A RAID0 array for live recordings (and possibly windows page file : to verify i don't know what will happen if a drive fail then => instant windows crash ?)

The RAID5 array should defragged upon Windows / Software installation

After each recording : move the recording from the RAID0 array to the RAID5 array => this way the recording will not be fragmented on the RAID5 array.

This kind of setup is possible with Intel raid chipset and the benefits are :

1) Having a secure array for OS and your recordings
2) Having a fast write array for live recordings
3) Reduce the need of defragging large files

Regards,
Stéphane.

[Mark SS]

Originally Posted by blade

That is the age long debate. Personally I just don't see the need to rearrange gigabytes worth of data if there are no performance benefits and if as you say there isn't any measurable effect on lifespan either way why bother?

Following our previous 'debate' on the issue I'm leaving a drive to get into a right old fragmented mess so I can at least offer some stats to backup the performance aspect

[Goodspike]

I think defrag is mainly a performance thing. I believe the moving of the heads is all electro-magnetic (motor-like) and thus not likely to cause wear. If a drive goes out, it's more likely to be the main bearings supporting the platters, not the bearing that allows the arm to move back and forth.

[blade]

Originally Posted by Mark SS

Following our previous 'debate' on the issue I'm leaving a drive to get into a right old fragmented mess so I can at least offer some stats to backup the performance aspect

I'll be interested to see if you can find any performance difference between a fragmented and defragged recording drive.

I just checked mine again and as usual:

Total Fragmentation: 50%
File Fragmentation: 99%
Free Space Fragmentation: 2%

My most heavily fragmented files:
5.80GB with 5,541 fragments
11.92GB with 4,606 fragments
9.40GB with 3,667 fragments

January will be 2 years without defragging and no stuttering or other performance problems as of yet.

[GTwannabe]

Defragging an active recording drive is a lost cause. Because you're constantly writing to the drive, you're instantly hosing any contiguous free space the defrag utility is able to allocate.

As far as performance, my 99% fragmented 64k-cluster video drives have no trouble streaming to multiple clients. It's a meager server, but it handles SageTV and webserver duties superbly.

[Goodspike]

Video isn't a terribly demanding use of a hard drive--especially with 64k clusters. Apples and oranges, but the first SD Tivos were only 4500 RPM drives.

I'd not really thought about this until someone mentioned it early in the year, but one place you will notice a performance difference in defraging is doing a anti-virus scan of your complete drive (or presumably any other complete scan). Apparently all those little seeks add up. It would probably be even faster with a defragmentor that sorted the files alphabetically by directory, since that's how AV programs tend to scan.

[Mark SS]

Originally Posted by blade

I'll be interested to see if you can find any performance difference between a fragmented and defragged recording drive.

Exactly why I thought I'd try and check, its interesting

One of my profesional hats is SQL Server DBA so monitoring disk performance is common practice. I can pretty much guarentee it will be an issue at some point but its impossible to say where without grabbing some stats.

[Lucas]

I 'll throw in my 2 cents worth of info from my experience.

In March 2005 I installed a 300Gb SATA drive for SageTV recordings. I sometimes recorded up to 3 concurrent streams at once.

By July 2006 the drive was almost full at around 270Gb and I noticed that when recording more than 1 show at a time, the recorded shows would have stutters and missing frames. The drive was heavily fragmented and it seemed that there was no large enough space to record both concurrent shows in close proximity on the physical disk area. As such the drive couldn't respond quickly enough and one or both shows being recorded would contain lost frames.

After experimenting with defragmentation tools and failing to achieve anything the only solution was to copy everything off the drive and copy them back to it. The ultimate defragmetation!

This is one of the reasons I don't use my RAID 5 array for recording. I only copy files that I want to keep long term, one at a time to the RAID 5 array to reduce its fragmentation.

[doc]

I can see why people wonder about defragging, I just analysed one of my drives, bearing in mind its only 33% used (mostly continuous white space in the defrag window), and most movies are in 2 or 3 fragments, but then last nights recording of the butterfly effect, well, is somewhat different....

but it all plays perfectly well and I've never had a performance issue.

[jptaz]

I have been using Raxco perfect disk on my Raid 5 Array daily since I started recording 3 HD streams almost a year ago...initially I was having problems and thought that the issue was related to disk performance and fragmentation. I have never really reconsidered this until now....I am going to disable my defrags for a week or so and see how the performance is and if it is really worthwhile, but I think I may have gone down the wrong path and just started a daily defrag routine without a solid reason...time will tell.

John

[Mark SS]

Originally Posted by doc

I can see why people wonder about defragging, I just analysed one of my drives, bearing in mind its only 33% used (mostly continuous white space in the defrag window), and most movies are in 2 or 3 fragments, but then last nights recording of the butterfly effect, well, is somewhat different....

From that image it looks like only 1 recording is badly fragmented and with so much free space you'd be unlikely to see a problem for a long while. Every recording on my test drive is now 2-3000 fragments so its getting to the state where I should be able to induce an issue.