WHS and Sage storage

[Tokendude]

I'm new to both whs and Sagetv and have a quick question. My set-up is a dedicated server with a 640gig primary drive (20 system/620ish data), plus 2 x 1TB drives placed into the pool.

I noticed that when whs backed up my desktop machine it fairly equally distributed the data across the D: data drive (20gig), and the 2 pool drives (37gig/ea). This makes it look like the D: is included as part of the pool.

My question is, when I install Sagetv will it also place some of the recordings into the D:? My understanding is that it will not occur unless the pool drives become full, but because of the way the backup was distributed I'm now not so sure.

If it does include the D: in the pool, then I will have to go back and change it to 64k clusters also (1TB pool drives are already reformatted). I hadn't done this yet because I wasn't sure I had a need to do so.

Thanks,

Roy

[gplasky]

The storage on D: is the backup database. That is what it is used for. It is NOT part of the pool and would only be used if your pool ran out of space. You don't need to format D: for 64k.

Gerry

[bhyman1]

Personally, I don't have Sage recording to the storage pool. I have two drive I've left out of the pool and formatted with 64k clusters. Works fine.

Also, I never bothered with the whole 64k cluster conversion on the pool for my stored media like dvd's, hd-dvd rips, etc. That works great too, no issues.

[Tokendude]

Originally Posted by gplasky

The storage on D: is the backup database. That is what it is used for.

Gerry

My confusion was that most of the backup went into the pool. Is this normal as I thought the backups went to the data drive only.

Roy

[gplasky]

The Windows Home Server side of the solution is a database that is specifically designed to efficiently backup multiple home computers on a daily basis. The cluster data and metadata from each home computer is stored on the home server hard drives.

The backup database is stored entirely in the folder D:\folders\{00008086-058D-4C89-AB57-A7F909A47AB4}\. The content of this folder is migrated by Windows Home Server Drive Extender from the primary data partition to a secondary data partition if the home server has more than one hard drive

What makes you think most of the backup went to the pool? Do you have any folder duplication turned on?

Gerry

[Skirge01]

Originally Posted by gplasky

The backup database is stored entirely in the folder D:\folders\{00008086-058D-4C89-AB57-A7F909A47AB4}\. The content of this folder is migrated by Windows Home Server Drive Extender from the primary data partition to a secondary data partition if the home server has more than one hard drive

I've been trying to figure this out definitively, so perhaps you just answered it. Does this mean that all computer backups are also duplicated elsewhere in the drive pool for redundancy?

[gplasky]

Originally Posted by Skirge01

I've been trying to figure this out definitively, so perhaps you just answered it. Does this mean that all computer backups are also duplicated elsewhere in the drive pool for redundancy?

I believe the answer is no. Here is the way I understand the backup occurs:

The home computer backup solution in Windows Home Server has a single-instance store at the cluster level. Clusters are typically collections of data stored on the hard drive, 4 kilobytes (KB) in size. Every backup is a full backup, but the home server only stores each unique cluster once. This creates the restore-time convenience of full backups (you do not have to repeat history) with the backup time performance of incremental backups

The home computer backup occurs as follows:
• When a home computer is backed up to the home server, Windows Home Server software figures out what clusters have changed since the last backup.
• The home computer software then calculates a hash for each of these clusters and sends the hashes to the home server. A hash is a number that uniquely identifies a cluster based on its contents.
• The home server looks into its database of clusters to see if they are already stored on the home server.
• If they are not stored on the home server already, then the home server asks the home computer to send them.
• All file system information is preserved such that a hard disk volume (from any home computer) at any backup point (time) can be reconstituted from the database.

The copies of the clusters that are stored on the home server in the order they are received. Additionally, the home server stores metadata files which allow it to deduce which clusters belong to which home computer at which times. The metadata that is typically stored for each home computer usually takes less than 5 percent of the space of the clusters that are backed up.

Another point to keep in mind:

Deleting the metadata is relatively fast (typically a few minutes per home computer), but it does not recover much space because the metadata for each home computer backup is usually less than five percent of all of the space consumed by the backup database. Deleting copies of clusters rarely recovers much space because the backup database is a single instance store of the unique clusters stored on your home computers. If another home computer backup that references a given cluster still exists, then the cluster will continue to be stored in the backup database.

If you remove a computer from the Windows Home Server Console, all of the backups for that home computer will be deleted during the next Backup Cleanup. This is when you will see the largest decrease in the size of the backup database, especially if the home computer had a lot of unique data.

Because of this method the first backup always needs to send the entire contents of the disk to the home server. This is why the first PC you backup shold be a PC with the largest hard disk/data in it.

The backup database is a collection of files that can be broken into three categories:

• Global files - Files that track the history of all home computer backups and all of the other files in the backup database.

• Cluster files - Files that store the actual cluster data from the home computers. These are usually in 4096 byte increments. These files grow to 4 GB in size before a new file is created. These files are stored as Data.XXXX.Z.dat, where XXXX=cluster size in bytes, and Z=0,1,2,…

• Home Computer specific files - A boot record and data for each hard drive volume that gets backed up from each home computer.

Also:

To restore individual items from a previous backup stored on the home server, a virtual volume driver is loaded on the home computer. This allows a previous image of a hard drive to be mounted, as if it were a read-only volume, across the home network. This driver opens a connection to the home server, and it receives the relevant clusters from the home server to build a virtual view into what a given hard drive looked like on a given day. The read-only volume is displayed in Windows Explorer where a user can drag the files and folders that they want to restore to the Windows Desktop of the home computer they are using.

And finally:

Saving a Copy of the Backup Database
The Windows Home Server backup database is not duplicated by Windows Home Server Drive Extender. So if you lose a single hard drive on your home server, you could possibly lose all of the backups of your home computers.

You may want to periodically copy the entire backup database from your home server to an external hard disk that you attach to your home server. The external hard disk should not be added to the Server Storage on your Windows Home Server.

Important
The cluster data files stored in backup database can grow to 4 GB, so it is important that the external hard disk is formatted as NTFS to support copying these large files. Some file systems, such as FAT-32 allow a maximum file size of 2 GB.

The backup database is stored entirely in the folder D:\folders\{00008086-058D-4C89-AB57-A7F909A47AB4}.

This tells me the backup database in in one place. There is also a add-on that will allow you to backup the backup database elsewhere.

Gerry

[HuMan321]

Originally Posted by gplasky

The backup database is stored entirely in the folder D:\folders\{00008086-058D-4C89-AB57-A7F909A47AB4}.

This tells me the backup database in in one place. There is also a add-on that will allow you to backup the backup database elsewhere.

Gerry

What would happen if you ran out of room on the Data drive? Would it utilize some of the pool for backups?

[Tokendude]

Originally Posted by gplasky

What makes you think most of the backup went to the pool? Do you have any folder duplication turned on?

Gerry

This is a brand new install of whs. I installed Disk Management add-in by Sam Wood, and look at drive utilization stats. Before I set up a user (me/my primary desktop machine) it showed the 2 pool drives with 0% utilization and the primary disk with around 12gigs used. After I set up my user on whs and set up the automatic backup of my desktop machine's c:drive (only) I waited overnight to see what changed on the whs drives. Now looking at the stats from the Disk Management add-in it shows:

Primary drive (c:/d: combined) 20gig (3%), so the backup really consumed about 8gigs.

Pool1 drive: 37 gigs (3%)
Pool2 drive: 37 gigs (3%)

This leads me to believe the bulk of the backup was stored primarily in the pool, not on the D:data drive, and is probably a function of the Storage Balancing Algorithm. The desktop system drive backed up has about 145 gigs full btw. Also, note that I have not done anything else to the whs system yet as I am still running lines and getting things ready to install Sagetv and the tuners, etc. This is just a plain vanilla install of whs + the one add-in. Also, duplication is not turned on for backups.

I'm just trying to understand how everything will be stored as I go along...

Roy

[Skirge01]

Thanks, Gerry! Great explanation.