SageTV Community  

Go Back   SageTV Community > Hardware Support > Hardware Support

Notices

Hardware Support Discussions related to using various hardware setups with SageTV products. Anything relating to capture cards, remotes, infrared receivers/transmitters, system compatibility or other hardware related problems or suggestions should be posted here.

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 02-15-2009, 03:19 PM
Beefcake550 Beefcake550 is offline
Sage Expert
 
Join Date: Oct 2004
Posts: 706
Drive Balancing tool for WHS

Hi All,

I've finally gotten time to finish up this app I've been working on. It basically balances all the data on your drive pool equally. Let me know if you like it! I thought it could be helpful with Sage setups since I'd think balancing can get all out of whack with large amounts of recordings.

Here's the readme:

UPDATED NOTE:
Some people have been having trouble not getting any data to move when their D:\ has lots of data on it. Before doing anything else, please run Drive Balancer to clear the landing zone only. To do this, choose "n" to include d:, then choose "y" to clear landing zone, and the "5" to exit. After this initial run (which can take hours), rerun Drive Balancer the with the options you want. I believe this will help the problem of no data being moved on some people's systems.

Usage Notes:
This program was designed to "even out" the data on a drive pool for WHS. The main purpose was when a person adds a drive to the pool, it doesn't necessarily start getting used for a while. While Microsoft has made changes to the usage of the Landing Zone (as in it doesn't exist anymore), some people's systems might still be in a non "evened out" state as wish to correct this. On my own system, I routinely get within 2% of perfect balancing on all pooled drives.

For a walk through of the tool running, checkout the walk_through.txt file.
Testing:
I have tested this program on my own system which consists of:

System Disk:
Areca 1210 RAID5 set consisting of (4) 250GB Seagate Drives = 20GB C: and 678GB D:

Pooled Disks:
(3) WD 1TB Green HDs
(2) Seagate 1.5TB HDs


IMPORTANT NOTE:
With some help from user "S_M_E" on SageTV forums, I am able to "restart" the Drive Extender service which is responsible for doing the actual balancing. The program detects a restart is needed if <5MB/s of total disk throughput is monitored on all pooled drives. Monitoring disk activity would not have been possible within Sam Wood's help. At this point, it restarts the service and marches along. Feel free to restart the service on your own if you are watching the process and see no disk activity (using Disk Management plug-in).

You WILL see some critical WHS warnings reported all related to storage space. Do not worry, this is expected behavior while the tool is running and will be cleared up at the end of the run.

If you need to hit Ctrl-C out of the program even though it is automatic and will end on it own. If you feel the program is behaving badly, and you Ctrl-C out of it, You will have to manually delete the following DIRs:

C:\fs\*\fake\
D:\fake\

I have extensively tested this program on my own system with my own data. I have lost nothing and you should realize that this program does not delete any data (It only creates fake files). As such, I can't guarantee that any data on your own system. Don't be afraid, either do a backup first, or jsut trust me and run the tool anyway!

Acknowledgements:
Theo van Elsberg - author of LZReallocator
Brent from wegotserved WHS forums
S_M_E from SageTV forums
Sam Wood - author of Disk Management add-in

======================================================================================

Revision History:
v1.05
- Attempting to solve problem where no files are moved on the system when D:\is filled
prior to running Drive Balancer. Please read UPDATE NOTE for potential solution
- Fixed bug where calculation of free space on D: was not "worked around". Refer to v1.03
history notes below. This fix will help systems that either
a) don't include D:\ in balancing
b) tried to clear the landing zone
- Added programmable time interval for checking disk activity
- Added programmable disk activity threshold for restarting Drive Extender service

v1.04
- Fixed bug where D: was not always cleaned up
- Modified restarting of DE service to monitor disk activity. This will restart DE within
seconds instead of 15 minutes. Should improve run time on systems with large files.

v1.03
- Handled bug where WHS returns free space of D: as (C: + D
- Fine tuned "crunching" algorithm so balancing should be better on different sized HDDs
- Added exception handling around creation of fake directories and fake files

v1.02
- Added option to allow D: to be part of balancing.
- Fixed bug where restarting DE would not occur and forward progress would likely not occur.

v1.01
- Fixed bug with option #2 and a drive was large enough to contain an equal amount of data.
- Added option for entering in your own data
- Added "quit" option in case all you wanted to do was clear the landing zone
Attached Files
File Type: zip Drive_Balancer_v1.01.zip (8.6 KB, 125 views)
File Type: zip Drive_Balancer_v1.02.zip (10.7 KB, 127 views)
File Type: zip Drive_Balancer_v1.03.zip (11.7 KB, 146 views)
File Type: zip Drive_Balancer_v1.04.zip (12.6 KB, 223 views)
File Type: zip Drive_Balancer_v1.05.zip (13.2 KB, 742 views)

Last edited by Beefcake550; 10-10-2009 at 01:22 PM. Reason: Release Update
Reply With Quote
  #2  
Old 02-15-2009, 03:24 PM
S_M_E S_M_E is offline
Sage Fanatic
 
Join Date: Dec 2007
Posts: 908
Can I say yes to 1 and not pick 2 or 3?
Reply With Quote
  #3  
Old 02-15-2009, 03:29 PM
Beefcake550 Beefcake550 is offline
Sage Expert
 
Join Date: Oct 2004
Posts: 706
You can say yes to #1, and then choose either 2 or 3. It will make a few calculations and then ask if it seems reasonable. You can then hit 'n' and it will clean up D: and exit.

-Brian
Reply With Quote
  #4  
Old 02-15-2009, 03:36 PM
S_M_E S_M_E is offline
Sage Fanatic
 
Join Date: Dec 2007
Posts: 908
Does #3 try to equalize used space or free space? With different sized drives that could be a big problem if it's used space.
Reply With Quote
  #5  
Old 02-15-2009, 03:43 PM
Beefcake550 Beefcake550 is offline
Sage Expert
 
Join Date: Oct 2004
Posts: 706
#3 equalizes used data on each drive. So, say you have 2TB total data used on the pool and 5 200GB drives, there would be (2TB/5 drives = 400GB) on each drive.

It would be easy enough to add a equalize free space option. I could whip that up quickly...

-Brian
Reply With Quote
  #6  
Old 02-15-2009, 03:49 PM
S_M_E S_M_E is offline
Sage Fanatic
 
Join Date: Dec 2007
Posts: 908
What if you have 2 250G drives, 2 500G drives, 2 1T drives and 2 1.5T drives with 1T free?

EDIT:
Doing the math it seems it would try to put ~687G on each drive which is impossible for the 250G and 500G drives. How would your tool handle that?

With free space equalization it would make 125G free on each drive which is wildly different percentages but at least the tool isn't trying to put 687G on a 250G drive.

Last edited by S_M_E; 02-15-2009 at 04:07 PM.
Reply With Quote
  #7  
Old 02-15-2009, 04:05 PM
Beefcake550 Beefcake550 is offline
Sage Expert
 
Join Date: Oct 2004
Posts: 706
well, 1TB free / 8 drives = 250GB free each disk. So, it would seem like your 250G drives would be empty! I'm sure that's not what you want.
What are you looking for?
Reply With Quote
  #8  
Old 02-15-2009, 04:13 PM
Beefcake550 Beefcake550 is offline
Sage Expert
 
Join Date: Oct 2004
Posts: 706
I guess I could put in an option where the user can put in the used/free space for each drive as they wish, but that can get very problematic to find error conditions with.

Using the equalize percentage used on your situation above, you'd get:

Total space = 6500GB

Free space after balancing:
250GB HDs = 38.46GB
500GB HDs = 76.92GB
1T HDs = 153.86GB
1.5T HDs = 230.7GB
So, each disk would have 15.38% of it's capacity free.

-Brian
Reply With Quote
  #9  
Old 02-15-2009, 04:15 PM
Beefcake550 Beefcake550 is offline
Sage Expert
 
Join Date: Oct 2004
Posts: 706
Update to equalize the free space, but no checks to see if we're allocating too much free space to a drive, so be careful. I'll implement that check later.

-Brian
Attached Files
File Type: zip Drive_Balancer.zip (8.1 KB, 167 views)
Reply With Quote
  #10  
Old 02-15-2009, 04:17 PM
Beefcake550 Beefcake550 is offline
Sage Expert
 
Join Date: Oct 2004
Posts: 706
There is a check to make sure that too much data isn't getting put on a drive.
Reply With Quote
  #11  
Old 02-15-2009, 04:19 PM
S_M_E S_M_E is offline
Sage Fanatic
 
Join Date: Dec 2007
Posts: 908
Quote:
Originally Posted by Beefcake550 View Post
well, 1TB free / 8 drives = 250GB free each disk.
125G


Quote:
Originally Posted by Beefcake550 View Post
I guess I could put in an option where the user can put in the used/free space for each drive as they wish, but that can get very problematic to find error conditions with.

Using the equalize percentage used on your situation above, you'd get:

Total space = 6500GB

Free space after balancing:
250GB HDs = 38.46GB
500GB HDs = 76.92GB
1T HDs = 153.86GB
1.5T HDs = 230.7GB
So, each disk would have 15.38% of it's capacity free.

-Brian
Yeah, equalizing by percentage or free space wouldn't seem to cause problems but what does your tool do if you try to put 687G on a 250G drive?


Quote:
Originally Posted by Beefcake550 View Post
There is a check to make sure that too much data isn't getting put on a drive.
So, using the used space equalization method, the 250G, 500G and 1T get full -10G and the 1.5T drives have ~470G free each?

Last edited by S_M_E; 02-15-2009 at 04:31 PM.
Reply With Quote
  #12  
Old 02-15-2009, 05:55 PM
Beefcake550 Beefcake550 is offline
Sage Expert
 
Join Date: Oct 2004
Posts: 706
yeah.. hehe 125GB.

I'm gettign confused by what you are asking. It is safe to run the tool to the point where it will ask you if the calculations it made seem right. I've put in range checking except for that new option of equalizing free space. You can exit the program by answering "n" when it asks if "it seems reasonable".

-Brian
Reply With Quote
  #13  
Old 02-15-2009, 06:25 PM
S_M_E S_M_E is offline
Sage Fanatic
 
Join Date: Dec 2007
Posts: 908
The first time I ran the first version I entered Y for D and after D was done I entered 2 and it crashed. You may want to have it create a log file. Now I'm using #1...
Reply With Quote
  #14  
Old 02-16-2009, 09:38 AM
Beefcake550 Beefcake550 is offline
Sage Expert
 
Join Date: Oct 2004
Posts: 706
WeGotServed forums found the same bug. Sorry about that. I've since fixed it and added a few more options with checking that should make you happy.

-Brian
Reply With Quote
  #15  
Old 02-16-2009, 10:58 AM
S_M_E S_M_E is offline
Sage Fanatic
 
Join Date: Dec 2007
Posts: 908
When using the % option I had a wide range when finished, from 78% to 92%. It's still more balanced than it was though and it cleared off D well.
Reply With Quote
  #16  
Old 02-16-2009, 12:12 PM
Beefcake550 Beefcake550 is offline
Sage Expert
 
Join Date: Oct 2004
Posts: 706
Givent he smaller sized hard drives in your system, it doesn't surprise me. I built in some padding such that it wouldn't be too aggressive. Each should end up within about 20-25G of each other Let me know if that's not the case.

-B
Reply With Quote
  #17  
Old 02-16-2009, 03:50 PM
S_M_E S_M_E is offline
Sage Fanatic
 
Join Date: Dec 2007
Posts: 908
Quote:
Originally Posted by Beefcake550 View Post
Givent he smaller sized hard drives in your system, it doesn't surprise me. I built in some padding such that it wouldn't be too aggressive. Each should end up within about 20-25G of each other Let me know if that's not the case.

-B
The drives I used in that other example aren't exactly the drives I have in WHS but just comparing 3 500G drives out of all my drives the range is 78% to 90% which is about a 57G difference in free space. *shrug*
Reply With Quote
  #18  
Old 02-16-2009, 06:35 PM
crarbo1 crarbo1 is offline
Sage Aficionado
 
Join Date: Jul 2008
Posts: 449
I think that I will end up using this utility but I do have a question. I use my WHS as my SageTV server as you probably expect but I was wondering if I could use this tool while SageTV was recording to the pool? If not, then I could wait until no one is recording anything, watching a recording, or watching live TV to do it. Just don't want to screw anything up.

Also, I assume all I have to do is unzip the files to a directory in the WHS and run from that directory. Is that correct?

Thanks,
Chuck
Reply With Quote
  #19  
Old 02-16-2009, 07:21 PM
Beefcake550 Beefcake550 is offline
Sage Expert
 
Join Date: Oct 2004
Posts: 706
That's a tough one crarbo1. I would think it be safe, but I'd feel much safer if you just stopped the SageTV service in order to prevent any recordings or disk activity besides this tool.

-Brian
Reply With Quote
  #20  
Old 02-16-2009, 09:52 PM
crarbo1 crarbo1 is offline
Sage Aficionado
 
Join Date: Jul 2008
Posts: 449
Quote:
Originally Posted by Beefcake550 View Post
That's a tough one crarbo1. I would think it be safe, but I'd feel much safer if you just stopped the SageTV service in order to prevent any recordings or disk activity besides this tool.

-Brian
I agree but I thought I would ask anyway. I will just have to do it when I have a few hours that it is not in use. That may be hard though.

Thanks,
Chuck
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Hard drive size for WHS PLUCKYHD Hardware Support 36 11-17-2010 01:15 PM
Formatting a WHS drive to 64k dislecksea General Discussion 15 12-11-2008 12:05 AM
WHS - Setting up an unmanaged recording drive jm9843 SageTV Software 11 12-09-2008 02:53 PM
WHS---Unmanaged recording drive SAGEaustin Hardware Support 1 08-28-2008 06:07 AM
How to add/setup unmanaged drive to WHS jelentz@gmail.c SageTV Software 7 03-06-2008 03:23 PM


All times are GMT -6. The time now is 07:48 AM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2018, vBulletin Solutions, Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.