Tutorial: SageTV on WHS with 64K clusters in the drive pool...

[inlvnv]

I just hate to read a thread like this when I'm about to do another new WHS install and after promising myself I won't do any more experienting with the system. I'm running a test system now to check out couple of 1.5 TB drives, so I guess it won't hurt to reformat the drives to 64k clusters and see. Formating to different cluster size makes sense, and MS should have had this option built in.

One question. Typically, data will be written to one pooled drive until filled before moving on to next empty drive. Since you are recording to pooled drives, have you noticed any performance hit when SageTV is recording HD/SD, at the same time extenders and client PCs are also reading/writing from/to one drive? I have 11 devices accessing WHS, although not all at the same time, so, overall performance while constant recording to pooled drives by SageTV is a concern for me.

[wayner]

When do drives get rebalanced? I have a system with two drives at 97+%, the system drive at 60% and a new 750GB drive added a few days ago that is at 0%. Will it automatically start moving data off of the system drive?

[S_M_E]

Originally Posted by inlvnv

Since you are recording to pooled drives, have you noticed any performance hit when SageTV is recording HD/SD, at the same time extenders and client PCs are also reading/writing from/to one drive?

I have no HD nor extenders but I can record 2 SD shows at once while viewing an imported video on a client with no noticed performance hit.

Originally Posted by wayner

When do drives get rebalanced? I have a system with two drives at 97+%, the system drive at 60% and a new 750GB drive added a few days ago that is at 0%. Will it automatically start moving data off of the system drive?

It should move data off the D partition relatively quickly if there are other drives in the pool available. You might try enabling and or disabling duplication to see if that will help. A reboot might force it and there is a 3rd party WHS tool that is supposed to force data off the D partition but it shouldn't be needed since PP1 and the November update.

[inlvnv]

Originally Posted by wayner

When do drives get rebalanced? I have a system with two drives at 97+%, the system drive at 60% and a new 750GB drive added a few days ago that is at 0%. Will it automatically start moving data off of the system drive?

Your client pc backup files likely will not move off of your D drive. Other data should eventually move off of the D drive and onto your new pooled drive; however, one possible way to speed up moving data off the D drive is, you could turn off your file duplication, wait a while for those duplicated data to be deleted off and then turn the duplication back on. Files being duplicated should go to your new drive.

[S_M_E]

If the D partition gets too full, WHS will migrate the backups too. My backups all reside on D: though.

[rlvogel322]

Originally Posted by inlvnv

I
One question. Typically, data will be written to one pooled drive until filled before moving on to next empty drive. Since you are recording to pooled drives, have you noticed any performance hit when SageTV is recording HD/SD, at the same time extenders and client PCs are also reading/writing from/to one drive? I have 11 devices accessing WHS, although not all at the same time, so, overall performance while constant recording to pooled drives by SageTV is a concern for me.

I've recorded three HD shows and two SD shows while playing music on one extender one of the HD shows on another extender and a recorded HD show on the MVP (transcoding) with zero issues. I did this for around a half hour checking on things periodically. Comskip was also running on the shows being recorded. PP1 fixed a bunch of issues.

[inlvnv]

Originally Posted by S_M_E

If the D partition gets too full, WHS will migrate the backups too. My backups all reside on D: though.

Any reason why your backups are all going to D drive? Filling up the D drive wouldn't be a good thing, since that's the drive where tombstones reside. It would be interesting to see where the WHS will store the tombstones once D drive runs out of space.

[S_M_E]

Originally Posted by inlvnv

Any reason why your backups are all going to D drive? Filling up the D drive wouldn't be a good thing, since that's the drive where tombstones reside. It would be interesting to see where the WHS will store the tombstones once D drive runs out of space.

I believe that is default behavior. I think that's why it moves backups off the D partition, if D is too full, to make sure there's is room for tombstones. If D:\ runs out of space for tombstones I think you get file system full errors but I don't really know for sure. I don't plan on running out of room and that's just another reason why it is better to go "too large" for the system drive than too small.

[inlvnv]

Files and backups physically residing in your data partition of your system drive is not a normal WHS behavior, unless your pooled drives are full. Looking at Microsoft's technical Briefs for Windows Homeserver for backups, WHS stores backups at the cluster level and in 4k cluster size. Since this is the case, is it possible that the WHS is physically storing backups in your D drive because your pooled drives are formatted with 64k cluster and WHS is looking for a drive formatted in 4k cluster to store the backups?

[S_M_E]

Originally Posted by inlvnv

Files and backups physically residing in your data partition of your system drive is not a normal WHS behavior, unless your pooled drives are full. Looking at Microsoft's technical Briefs for Windows Homeserver for backups, WHS stores backups at the cluster level and in 4k cluster size. Since this is the case, is it possible that the WHS is physically storing backups in your D drive because your pooled drives are formatted with 64k cluster and WHS is looking for a drive formatted in 4k cluster to store the backups?

I don't know about your WHS but my backups are in D:\folders\{00008086-058D-4C89-AB57-A7F909A47AB4} (also the place where you delete the files if you have an unrepairable backup database consistency error (another of my WHS forum tutorials)) Comparing the size of the data there with the size of my backups reported in the console says that 100% of my backups are on the D partition. Mostly in 4G .dat files

Furthermore, the backups were 100% on the D partition even before I formatted the pool drives and neither the pool nor the system drive has ever been "full." Also the same pre- and post-PP1. I believe that the backups stay on D unless D gets too full. Compare the folder above with your console's reported backup size and let me know what you get.

[jaminben]

Hi,

I have a WHS setup which is used for storage and backups only and I'd like to move my entire Sage setup to WHS as well.
I have bought 2 new 1TB drives and have 4 500GB already in use on the WHS. What I'd like to do is transfer all my data from the 500GB drives to the 1TB drives and use one of the redundant 500GB drives as the WHS install disc and another 500GB drive as the recording drive.
I've read various forums relating to WHS disc management and they seem to recommend you use your largest disc (1TB in my case) as the install disc and add the smaller ones to the pool. They recommend this as it has something to do with the landing pad??

So my question is which is the best route to take, Install WHS on the 1TB drive or the 500GB drive?

Regards

Ben

[gplasky]

I can tell you what I did. Not necessarily best practice. On my original WHS server I had a 400 GB OS drive (20GB C and the rest was D: drive) I had 3 x 500 GB and 1 x 1 TB out of the pool. I built a new one (new MB and case-actually my previous dev box) with a brand new 500GB PATA drive for the OS and D: drive. (It had only 1 x IDE channel so DVD drive and OS/D: drive are on that-the rest is SATA.) And I still left my 3 x 500GB and 1 x 1 TB SATA drives out of the pool. For me it just didn't seem worth it to bring them into the pool. And I was able to leave the old one running until I set up the new one with the new drive. Then I just moved the drives over, renamed the new box to the old box name and made sure the IP was set the same. It all just worked and anything I missed off the old OS or DATA drive was a copy away with an external case for the drive. MY PCs all get backed up (to the D: drive from what I can see) and I have all the functionallity I need. The new mb gives me enough oomph to run the latest comskip on the h.264 recordings without sucking up all the CPU and doesn't interfere with any watching of the recordings. SME has all the tutorials to make the OS drive bigger or all the drives in the pool 64k. To each their own. This just happens to work great for me. All my DVDs and music I have elsewhere so I'm not worried about losing a drive. Recordings can be recorded again. I don't need an image of my C: drive-a full copy of the drive was good enough. Less that an hour to reload the OS.

Gerry

[S_M_E]

Originally Posted by jaminben

Hi,

I have a WHS setup which is used for storage and backups only and I'd like to move my entire Sage setup to WHS as well.
I have bought 2 new 1TB drives and have 4 500GB already in use on the WHS. What I'd like to do is transfer all my data from the 500GB drives to the 1TB drives and use one of the redundant 500GB drives as the WHS install disc and another 500GB drive as the recording drive.
I've read various forums relating to WHS disc management and they seem to recommend you use your largest disc (1TB in my case) as the install disc and add the smaller ones to the pool. They recommend this as it has something to do with the landing pad??

So my question is which is the best route to take, Install WHS on the 1TB drive or the 500GB drive?

Regards

Ben

You no longer need to install WHS on the largest drive, 500G will be more than enough. Additionally, the old reasons for keeping the recording drive separate are now obsolete, the landing zone is no longer used and 64K clusters are now available in the pool if you follow the tutorial.


You *can* keep a separate drive for recording but I wouldn't. As others have posted, using 64K clusters and keeping the recording folder (not duplicated) in the pool still performs well as long as you have HW that can handle your Sage requirements. I, personally, would resize the 500G system drive to use 100G for the C partition and leave the rest for the D partition but I like to install more than just Sage and WHS so YMMV...

[psklenar]

I did my install yesterday ... I'm puzzled. The "D:" portion of my OS drive appears to be part of the "managed pool". And while I can "remove" the TB drives, I can't remove the "OS" drive from the "pool". Heck, in Disk Management, while it does show the three TB drives, it interestingly shows the D: partition being 365.75GB (as expected since I gave C: 100GB out of the 500GB drive), but it shows nearly 3TB or 800% "free"!?

Is this to be expected?

pat----

[S_M_E]

Originally Posted by psklenar

I did my install yesterday ... I'm puzzled. The "D:" portion of my OS drive appears to be part of the "managed pool". And while I can "remove" the TB drives, I can't remove the "OS" drive from the "pool". Heck, in Disk Management, while it does show the three TB drives, it interestingly shows the D: partition being 365.75GB (as expected since I gave C: 100GB out of the 500GB drive), but it shows nearly 3TB or 800% "free"!?

Is this to be expected?

pat----

As I've said, the space on the D partition is part of the pool but it shouldn't use that space unless the other pool drives are too full. Your media shouldn't be on the system drive, that's why you don't change the cluster size on the system drive, just the others. As long as you keep plenty of free space (or add more space before the pool gets too full) you'll be fine with 4K clusters on the C and D partitions. Since D also holds all of the tiny tombstone files, it's best not to format that one to 64K clusters because a lot of space will be wasted on the small files. Same with the C partition, a lot of small files so it'd be a waste to use 64K clusters there too.

The free space reporting behavior is to be expected since the November update, it was intentional to show all of the free space.

EDIT:
I should mention that if you WANT to format the system drive to 64K clusters, despite the wasted space (space IS cheap now) that is possible too but it requires more work to do it. As I mentioned in the tutorial, I got the idea for formatting the pool to 64K from a post by Gerry Plasky, in the WHS forums, where he formatted the D partition only during the beta.

[psklenar]

Cool. Nope, I have the 500GB drive (both 100GB C & ~400GB D) as 4K clusters and the three 1TB drives as 64KB clusters. I just wasn't sure, since I;'d seen discussion refering to "D:" and "pool" as, seemingly, two separate things. What I was seeing, and you seem to be confirming, is that D: is the pool.

WHS is working. Two systems successfully backed up overnight. HDHR installed and both tuners configured and working (clearn QAM only right now). Next ... HD-PVR.

Thank you,
pat----
pat----

[inlvnv]

Originally Posted by S_M_E

D:\folders\{00008086-058D-4C89-AB57-A7F909A47AB4}

I thought you were referring to your backup files physically residing on the data partition of the system drive which, until the November update, used to display as D: drive.

[S_M_E]

D: is still labeled as the DATA partition of the system drive and that folder *is* where my backup files physically are...unless that folder is a symlink. I could disconnect all of my pooled drives and just boot up with the system drive and see if the backups got smaller but I'm too lazy to test that out because I have too many drives.

[inlvnv]

I guess what I meant to say was, before the November update, D drive status used to display size and available space for that data partition only.

Anyway, Disk Management add-in will give you an idea how much space is available in the system drive and data partition. Or, you can just go into C:/FS folder and go through each of the pooled drives to locate the \{00008086-058D-4C89-AB57-A7F909A47AB4} folder. If the folder is in one or more of your pooled drives with all the backup files, you are fine.

[S_M_E]

Originally Posted by inlvnv

I guess what I meant to say was, before the November update, D drive status used to display size and available space for that data partition only.

Anyway, Disk Management add-in will give you an idea how much space is available in the system drive and data partition. Or, you can just go into C:/FS folder and go through each of the pooled drives to locate the \{00008086-058D-4C89-AB57-A7F909A47AB4} folder. If the folder is in one or more of your pooled drives with all the backup files, you are fine.

Yes, the November update changed drive space and file size reporting. PP1 changed the landing zone and other issues.

The DM add-in shows total space of the system drive including C and D. Checking the mount points in the C:\FS\* dirs does, indeed, indicate that the folder on D is a symlink. My old tutorial: http://social.microsoft.com/Forums/e...-e6c047c4c431/ still works, even after the November update. I did it last month.