Public Service Announcement: WHS gives a false sense of security for Sage purposes

[btrcp2000]

http://forums.freytechnologies.com/f...ad.php?t=46715

Above is the link to the hell I went through in January. Tonight I get to do it all again.

In spite of:

RAID1 mirroring of the WHS system drive
Sage server on its own UPS
Whole-house surge protection from the power company at my cost

I apparently lost both raid1 drives during a power outage. Neither BIOS nor the raid config utility sees them at the moment. Any suggestions are appreciated. Somehow I think I am screwed. Again. No, I have not yet tried the reinstall. See the link above for my faith in that process.

I am simply posting this to warn off anyone from thinking that a single box sage WHS solution is the way to go. Most have had success, but I am an example of exactly how wrong it can go, and how much time/life can be pissed away fighting with it.

Not blaming sage for this, fully blaming microsoft for selling whs as a security solution. Yes, I know raid1 isn't officially supported, but that was the best I could do to prevent January from happening again. If it's so unimportant to backup the c drive, then why is that a primary Vail feature?

[PLUCKYHD]

First like you said you are griping about raid 1 failing when like you stated it isn't supported and you IMHO where being risky using it in the first place.

I have had a c:\ on my whs fail and thanks to my weekly copying my wiz.bin it took me all of 20 minutes to get everything up and running on the c drive again after reinstalling whs. I had no issue keeping my pooled drives and getting them back. C:\ backup was added in vail because it was highly requested, but to me there are to many caveats i don't like about vail to keep me on whs version I have.

Sorry you are having issues I know it can be tough but blaming MS for you using raid and feeling safe is not right. There is no 100% safety net no matter what way you go, I have had high end business "safety" solutions fail. So to if someone gives them self false security than that is no ones fault but theirs or the person who told them that.

Good luck getting back up an running it sounds like whs may not be tue best solution for you.

[btrcp2000]

I am using an unsupported solution because the supported solution of "server reinstall" failed me miserably in January, requiring the purchase of another hard drive to move 1.5tb off and then back on after I fresh installed whs. Here we are ten months later, and I'm looking at it again. If there is something I have done to cause this that is otherwise preventable, I'm all ears, as I spent a lot of time and money trying to prevent things like this since January.

Glad you have had a good experience. Myself, I have had two WHS system drive losses in one year (three if you count both RAID drives), while having taken all protective steps I can find (ups, surge protection, raid1). Extremely frustrating that not a bit of it helped.

One of the failed drives is now showing life and allowed a boot into whs. Hopefully I can get the raid utility to allow me to use a hdd that is not part of the pool, which right now it does not want to do. It sees the new replacement drive, but will not let me rebuild to it.

And not being argumentative because you know more than me about these things, but how is using raid1 any more risky than going alone with one hard drive? Supported or not, at least there is a chance of recovery due to the redundancy. Worst case scenario (like my current situation), you just end up reinstalling either way. Am I missing something?

[PLUCKYHD]

I would never trust raid in a whs that is just my choice. You do have a chance of getting something but you are having to do way to much extra effort for that chance. If I was going to run any sort of raid I wouldn't be using whs in the first place but in nas storage solution.

The bigger issue lies to me in why are you having constant hard drive failures? Hard drives last longer than you are getting that is for sure. Is it a heat issue? Movement issue? Motherboard issue? Wrong kind/brand of hard drive?

I would myself be more interested in that issue.

[medwynd]

Originally Posted by PLUCKYHD

I have had a c:\ on my whs fail and thanks to my weekly copying my wiz.bin it took me all of 20 minutes to get everything up and running on the c drive again after reinstalling whs. I had no issue keeping my pooled drives and getting them back.

That's actually interesting. So when the hard drive failed, some of the pooled storage was on that drive correct? So how were you able to get it back?

[Spectrum]

Since you guys are talking about raid1 being unsupported I'm assuming you are using some soft of software raid? If it's hardware raid the OS shouldn't care as long as there are drivers for the controller.

I'm also with Plucky on the drive failure problem. I went through a bad batch of Seagates about a year ago, 6 drives in 2 months. Since replacing the offending drives with Western Digitals I haven't had a problem. I'm not knocking Seagate in general, I still have some 10+ year old drives that work fine, I just think they had a bad batch and I was tired of RMAing them. It might be worth your while to figure out what is nuking your drives.

As far as backup goes if your data is mission critical, like wiz.bin, you should technically back it up locally using 2 different mediums AND store a copy off site. That's the policy I used when I was a network admin eons ago

[btrcp2000]

These were WD drives. I remember back when seagates were the way to go, now they're crap. Nest year it will be the opposite. Seems like anymore you will find out a year later that all the newegg reviews were meaningless, because you got "a bad batch".

I was able to boot once into whs with the remaining drive and that was it. It seems to function like all is well, no more grinding, but it won't boot. While it was booted, of course the intel matrix program would only allow me to choose my storage drives to rebuiild, not the perfectly fresh new drive I had installed. Thought I would reboot to fix that, never saw the desktop again as something hosed ntoskrnl.exe. Recovery console won't let me rewrite that file from my xp install disc: "access denied"

Server reinstall doesn't like my motherboard's sata drivers this time around (it didn't have a problem with them in january), keep getting bluescreen stop errors.

Like I said, whs has been a total failure for me, and I believe it put my data more at risk than sticking with xp. If hard drive quality is so hit-or-misschy nowadays, whs' finicky reinstall process is more risky than plan old xp and acronis.

Also, I have the sage directories and wiz.bin, that's the easy part. It's the freakin' OS that never wants to cooperate. I honestly cannot even look at the thing right now. Wife took the kids to the zoo on a gorgeous day today, I sat in the basement piddling with sage and accomplished nothing.

[PLUCKYHD]

Sorry for your bad experience, but you have to remember using whs for sagetv is not what WHS was meant for and is asking allot of it (although I haven't had the issues you have). WHS in itself is actually a great product and does what is is advertised in many many solutions. I even install them at small business as they are great client backup machines.

Again it isn't made or written for things like sagetv although I have 0 issues with it running sagetv nether do allot of others. Nothing is failproof.

[bastafidli]

Originally Posted by PLUCKYHD

Nothing is failproof.

But for most things you can make the chance of failing negligent with proper preparation: RAID1 for important data and immediate recovery + mirror to second computer for all data + offsite replication/backup

[Peter_h]

Originally Posted by btrcp2000

<Snip>...
intel matrix program... </snip

You said the magic words right there. Intel Matrix is not a true "RAID". It is a software based raid implemented at the bios level by intel and unfortunately, it is not very good.

Google "intel matrix problems" and you will see tons of posts spanning multiple os's from xp to vista to WHS.

I can speak first hand of running WHS and being bit hard by intel Matrix Raid. About 2 years ago when I setup my WHS, I researched and researched and was very aware of not being able to backup the OS partition. So, I installed WHS onto a RAID 1 array created using the Intel Matrix Technology. The server gave me nothing but problems. Random lockups with Event logs pointing to IO errors and the OS volume becoming unavailable. Within 1 year, the entire array failed and I was left doing a reinstall. I was able to boot into it once, such as you and the event log was flooded with OS volume unavailable errors. Both Drives ended up testing out fine using the WD utility. The problem was simply so much data corruption on the OS array that WHS would not boot anymore.

I knew that Intel Matrix was suspect since the install ititially was not clean. I had to slipstream the install disk with the RAID Drivers for the install to even proceed. I also googled the technology and proceeded anyway despite all the issues i had seen.

The true solution for me was to go with a hardware based raid solution meaning, a dedicated hardware RAID card for the OS partition. As Spectrum mentioned, a true Hardware RAID solution will be transparent to the OS. I went out and bought an areca card w/ a dedicated battery backup.

I was able to create a RAID 1 array w/ a Hot spare w/ email notification that is completely invisible to the OS and can be managed ie rebuilt outside of the OS. The live hotswap actually gives me up to 2 drive failures now and will keep going.

Again, sorry you have to go through this but speaking from experience, the problem is not with WHS but with using WHS with Intel Matrix.

[PLUCKYHD]

Originally Posted by bastafidli

But for most things you can make the chance of failing negligent with proper preparation: RAID1 for important data and immediate recovery + mirror to second computer for all data + offsite replication/backup

I realize that I was refering more to home use and I hope people aren't going to that extreme for home data. If they are more power to them I guess. I backup pertinent data (mostly pictures) once a month to blu ray disc and keep a copy outside the house that is about extreme as I am going to get for household usage.

[btrcp2000]

After a night of tossing and turning, I am considering two alternatives.

My main goal is speed of recovery, as I have the data side covered for the important things.

1. Move the sage server back to xp, which was more stable for other reasons as well. Replace my NAS with a WHS box, possibly even off-the-shelf for warranty purposes, and have it back up the sage server OS drive just like any other desktop (but continuing to back up wiz and sage folders via scripts)

Questions:
Any reasons this approach would not work? Hopefully it would have me up and running again in no more time than a WHS restore backup process

Would the sofware raid running under xp be any more stable? Since I would already have the whs backup, and the mobo can do it, any reason not to have an ongoing os drive backup. It it craps itself again, I have the whs backup.

2. Move to h/w raid, per peter h's suggestion
Questions:
How does the hot swap work, is it just a third drive running alongside the two? Would it be best to use hard drives by different mfrs this time around? These two WD's were the only things in the entire house to be affected, maybe matching pairs aren't the best approach?

[PLUCKYHD]

Originally Posted by btrcp2000

2. Move to h/w raid, per peter h's suggestion
Questions:
How does the hot swap work, is it just a third drive running alongside the two? Would it be best to use hard drives by different mfrs this time around? These two WD's were the only things in the entire house to be affected, maybe matching pairs aren't the best approach?

As per peter are you sure those hard drives are actually bad. I have a hard time believing 2 hard drives are bad after that short of usage. Have you tried booting from dos with a hard drive utility to see if they are readable?

[btrcp2000]

I agree, ten months is way too short for two to fail, but it's not like quality has been consistently fantastic from any of the mfrs for the last several years, so I guess it's not out of the question.

When I first noticed the problem, they were both grinding away and neither were recognized by a usb sata dock attached to another computer. One is still that way, the other has since stopped grinding and is recognized. That's the one that booted once, but upon a subsequent reboot complained about ntoskrnl.exe being corrupted. Google suggested using my xp disc to get into recovery console to reinstall that file, but I kept getting "access denied" errors at the c prompt when I went to copy over the fresh ntoskrnl.

At this point,it does look like the second hard drive is functioning, but it won't boot. I am splitting my time learning about hardware raid and fixing boot errors, any thoughts there are welcome.

Speaking of the physical drive failures, the server sits in a dedicated basement room, ambient temps about 77-80 deg F. It's even on a dedicated circuit, which was a step I took after January. This all started with power outage, after which my ups should have gracefully shut everything down as it has in the past, but I have no way to confirm if that happened.I assume windows got hosed either when the ups battery died and the server was not shut down, or by the initial surge when the power came back on, both of which should have been dealt with by the ups.

[Oats]

If uptime and quick recovery is your goal I think running the server on XP with a good backup/restore solution in place is probably best. WHS is not a quickly restored OS from everything I've read about it.

I'd also not bother with motherboard/software RAID1, I used it on a Sage Windows 2003 Server. A drive was reported bad so I shut down to pull it to run some tests. Server wouldn't boot anymore, tried each drive individually and together and nothing was working. I eventually got one of them to boot useing the Windows install CD. I ran tests on both drives and both passed just fine and went on to run fine to this day, just not in raid.

I have a hardware raid card now I use for raid5 storage, but I will soon be ditching that. Everything on it that is important is backed up on an external drive so it isn't necesarry and just makes things more complex than they need to be.

[Spectrum]

Originally Posted by btrcp2000

Would the sofware raid running under xp be any more stable?

If you are talking about Dynamic Disks then no. XP's built in software raid is crap and far far FAR worse than Intel's matrix controller. Also in XP pro it may only be capable of doing Raid0, it's been a long time since I looked at it. If you are thinking 3rd party solution, then IDK about reliability.

Another thought, since you mentioned ambients in high 70s/low 80s. Are your drives in the WHS box actively cooled? Heat is by far the biggest killer of HDDs. If you have several drives clustered together without fans on them the temps near them could easily be 20+ degrees higher than your case ambient, which is usually higher than your room ambient.

[btrcp2000]

Just ran WD's data lifeguard on the "functional" drive, didn't take long for the error codes to come up. Looks like it really is shot. That's three under warranty hard drives from different manufacturers in 10 months. Unbelievable.

Would the temps have mattered since everything was hunky dory untilthe power outage? Yes, there is a five inch case fan on the back of the case (cooler master elite)

Looks like its back to XP, with a separate whs box.

I still stand by the subject line of my thread, supported raid or not. If this came dopwn to a "bad batch" of WD drives that couldn't withstand a little power fluctuation like the other ten drives

[Spectrum]

Originally Posted by btrcp2000

Would the temps have mattered since everything was hunky dory untilthe power outage? Yes, there is a five inch case fan on the back of the case (cooler master elite)

I can't say definitively yes or no, but removing power and re-applying it does put stress on the components in the drive, even the passives like resistors and caps. This can be limited somewhat by using inductive chokes, but only to a point. If high temps caused excessive strain on a component, the surge of removing and re-applying power could have been the straw that broke the camel's back perse. That resistor that WD payed less than a fraction of a cent for could pop and the whole thing went belly up.

This is all speculative and without detailed diagnostics that neither you nor WD want to do (think autopsy one of the drives) you'll never know for sure.

I do know that a $2.50 120mm fan blowing across the drive cage gives me pretty good piece of mind. Having worked in a datacenter and a university lab, any server I have seen in the last 5+ years that is "enterprise class" has fans blowing directly on the hard drive bays. They're there for a reason!

[btrcp2000]

Newegg has a 2tb Acer Easystore whs box for $350 after coupon (saw it on techbargains) which I will use to replace my NAS. Sage goes back to XP, and will be backed up both via the automated scripts as well as with WHS like all other pcs in the house. Maybe someday I'll pick up a hardware raid card for it too.

I have to say the scripted backups were awesome. The most time consuming part of this was determining that WHS just takes too long to reinstall should you lose a system drive. Especially when you can't server reinstall, resulting in 3-4 days worth of moving TBs of data off and then back onto the fresh-installed server, as opposed to just plugging the drive into an XP server and telling sage where to look.

Granted this could happen too with the Acer, but if they are separate boxes at least we will still have a functional sage system and the whs restore should be a simpler process with no third party stuff. At least the odds are longer that I would lose both WHS and the sage box simultaneously. Although, with the year I have had, that should happen right around Christmas!

[Oats]

I can't believe I missed that Acer deal, I've been looking at them for a few weeks waiting to get a good deal.