Recommended NAS?

[Article22]

Thanks for that advice, I am looking for a new motherboard on eBay now.

I did manage to get my disk array back up and running but boy was it a faff!
Here is the list of what I needed to do in the hope that someone else doesn't have to go through the same pain I did in trying to get it up and running again...

The main point was that two of my hard disks had failed - which generally means death for all your data...but in my case I was lucky and I was only getting bad sectors reported from them both and not just clicking noises.

Unfortunately they were the main operating system drives which were in a raid 1 format!

What I did was ...

1. Low Level Disk Scan using the Samsumg tools to work out which drives had failed (sda and sdb)
2. Low level format on sdb then re-tested it to make sure it was ok (took two formats before it worked though)
3. Rebooted using a LiveCD and ran it in linux rescue mode (which disables the drive arrays)
4. Fdisked sdb to match sda (OS and Data partitions)
5. Disk copied all the OS data from sda1 to sdb1 using dd if=/dev/sda1 of=/dev/sdb1
6. Installed the MBR on sdb
7. Rebooted into the normal OS which reported that my OS Raid 1 and Data Raid 5 arrays were degraded.
8. Rebuilt the Data Raid 5 array by adding sdb2 into it (this takes a while)
[I couldn't add sdb1 back into the OS Raid 1 as sda had failed as well so mdadm failed to sync sda to sdb...]
9. Low level format on sda then re-tested it to make sure it was ok (took two formats before it worked though)
10. Reboot back into the linux rescue with no drives assigned
11. Fdisk sda to match sdb partitions
12. Disk copy the data on sdb1 back to sda1
13. Installed the MBR on sda
14. Build a new OS Raid 1 disk array using mdadm --create, this will generate a new UID
15. Edit /etc/mdadm.conf and update the UID for the OS Raid Array to the new UID
16. Reboot normally, and my system booted with the OS Raid 1 array!
17. Just added sda2 back into the Data Raid 5 array and left it to sync.

Phew!

Nigel.

[mtyme]

I've got a raid5 array setup with EVMS like what's mentioned here. If I need to reload the OS, or move my array to a different computer, I should be able to keep the array intact without loosing any data correct? How does that work?

[mikesm]

Originally Posted by mtyme

I've got a raid5 array setup with EVMS like what's mentioned here. If I need to reload the OS, or move my array to a different computer, I should be able to keep the array intact without loosing any data correct? How does that work?

yes. the new system will recognize the array if you tell it to assemble the array from the disk devices... Depending on the distro, it may automatically detect it, though obviously you'll need entries in the fstab to mount it, etc...

[mtyme]

Very cool. I'll be sticking with SuSE. I like it and Ubuntu but I'm not really partial to either one. I'm not a total newb with regards to linux, but I'm a set it and forget it kinda person. I've had an Ubuntu server running web/mail/ftp for a year or two now but it was pretty basic stuff.

The only thing I'm having an issue with EVMS at this point is how to get the evms_activate to run at boot time so my fstab can mount my array. I tried adding it to the /boot/grub/menu.lst (I think that's the file, not in front of me so I can't double check) but either I'm doing something wrong or that's the wrong place to put it.

[ToxMox]

Just thought I'd throw an alternative to using EVMS here. After reading this thread I was inspired to build an Ubuntu server on our network to replace several file servers. First step build a raid. So I tried to follow along with using EVMS from this thread but after messing about a little too much it borked my Ubuntu install and even though I wasn't booting from the raid. I was getting an error over and over again and it would never boot the system. Even if I tried all the different runlevels. So I formatted and reinstalled Ubuntu.
Anyways... onto my point
I installed webmin on my server and you can do the same stuff as evms (or at least the stuff mentioned in this thread)

I was able with just webmin to create the mdadm raid 5 and also create the LVM2 volume etc.

[mikesm]

Originally Posted by mtyme

Very cool. I'll be sticking with SuSE. I like it and Ubuntu but I'm not really partial to either one. I'm not a total newb with regards to linux, but I'm a set it and forget it kinda person. I've had an Ubuntu server running web/mail/ftp for a year or two now but it was pretty basic stuff.

The only thing I'm having an issue with EVMS at this point is how to get the evms_activate to run at boot time so my fstab can mount my array. I tried adding it to the /boot/grub/menu.lst (I think that's the file, not in front of me so I can't double check) but either I'm doing something wrong or that's the wrong place to put it.

You need to tell it to start evms in the system services control panel. That should do it. You don't want it trying to start lvm if you are using evms, else the device mapper devices will get grabbed by lvm and evms will fail to start.

[mikesm]

Originally Posted by ToxMox

Just thought I'd throw an alternative to using EVMS here. After reading this thread I was inspired to build an Ubuntu server on our network to replace several file servers. First step build a raid. So I tried to follow along with using EVMS from this thread but after messing about a little too much it borked my Ubuntu install and even though I wasn't booting from the raid. I was getting an error over and over again and it would never boot the system. Even if I tried all the different runlevels. So I formatted and reinstalled Ubuntu.
Anyways... onto my point
I installed webmin on my server and you can do the same stuff as evms (or at least the stuff mentioned in this thread)

I was able with just webmin to create the mdadm raid 5 and also create the LVM2 volume etc.

Yeah, webmin has come a long way. EVMS is probably overkill if you just want to use raid with lvm, and IBM has kind of stopped supporting it, but it still gives you capabilities you can't really find elsewhere, at least not in an integrated way. If you can get by with what webmin offers, it's probably a simpler way to go.

[mtyme]

Originally Posted by mikesm

You need to tell it to start evms in the system services control panel. That should do it. You don't want it trying to start lvm if you are using evms, else the device mapper devices will get grabbed by lvm and evms will fail to start.

OK that worked. However, the array isn't mounting. Here's the line I'm using in fstab...

/dev/evms/media_vol /media/vol1 xfs defaults 0 0

After it boots up, I can enter mount /dev/evms/media_vol /media/vol1 and it mounts fine. What am I doing wrong here?

edit: figured it out, I had to go into expert mode and set evms to start at the boot run level.

[johnm160]

I have been reading here about what a POS WHS is and how the transfers are so slow. So I downloaded freenas and set that up on the same box, transfered the same two files and the transfer speed was about the same....freenas speed was about .5 mb/s slower. They both ran between 6-7 mb/s transfering 2 multi gig files.

Now I know nothing about linux so I dont know if I am doing something wrong or not but I saw no diff in speed.

Box is a athlon xp 2600 running on a 100 mb lan.

Any ideas?

Edit to add

6.5 MB/s = 52 Mb/s Network utilization is 61 - 69 % during the transfer.

Can I really expect better out of a 100 Mb /s connection?

[mtyme]

Originally Posted by mikesm

yes. the new system will recognize the array if you tell it to assemble the array from the disk devices... Depending on the distro, it may automatically detect it, though obviously you'll need entries in the fstab to mount it, etc...

I tested this out last night. Reinstalled SuSe 10.3 from scratch, opened EVMS and there was my array. Mounted it and all my data was still intact. Not that I had my doubts, I just wanted to see it work first hand. I'm a hands on kinda person lol.

I also installed webmin just to play around with it. It's a pretty neat little tool.

[mikesm]

Originally Posted by johnm160

I have been reading here about what a POS WHS is and how the transfers are so slow. So I downloaded freenas and set that up on the same box, transfered the same two files and the transfer speed was about the same....freenas speed was about .5 mb/s slower. They both ran between 6-7 mb/s transfering 2 multi gig files.

Now I know nothing about linux so I dont know if I am doing something wrong or not but I saw no diff in speed.

Box is a athlon xp 2600 running on a 100 mb lan.

Any ideas?

Edit to add

6.5 MB/s = 52 Mb/s Network utilization is 61 - 69 % during the transfer.

Can I really expect better out of a 100 Mb /s connection?

Depends on the particular lan chipset you are using. You might be able to get 80 Mbps out of a 100 Mbps LAN on a good day, but you are seeing almost 70% utilization, so that's not that all that bad.

If the LAN is the bottleneck, even WHS can tie a good implementation for performance. :-) I would upgrade to Gigabit ethernet if you want better speeds.

[mikesm]

Originally Posted by mtyme

I tested this out last night. Reinstalled SuSe 10.3 from scratch, opened EVMS and there was my array. Mounted it and all my data was still intact. Not that I had my doubts, I just wanted to see it work first hand. I'm a hands on kinda person lol.

I also installed webmin just to play around with it. It's a pretty neat little tool.

Yeah, neat huh? Not only that, but you could have moved the disks to a brand new computer with completely different disk controller hardware and completely different buses, CPU, etc... and it still would have found the volumes and put them back into service. Try that with a hardware raid controller.

It's good to put what people say to the test. Trust but verify I think is what Reagan said. It's especially good to test things before you put a lot of data on the disks... :-)

[johnm160]

Originally Posted by mikesm

Depends on the particular lan chipset you are using. You might be able to get 80 Mbps out of a 100 Mbps LAN on a good day, but you are seeing almost 70% utilization, so that's not that all that bad.

If the LAN is the bottleneck, even WHS can tie a good implementation for performance. :-) I would upgrade to Gigabit ethernet if you want better speeds.

Mike,

That is what I needed to know, after I did the math I did not think I could do much better on the current connection. I will have to deal with this for now, but I will make sure the next borard I put in the server is gigabit. PCI slots are at a premium right now and I will have to replace some switches. I have way to many items on my network to do the entire thing in gigabit.

Thanks for the info,

John

[mtyme]

Originally Posted by mikesm

Yeah, neat huh? Not only that, but you could have moved the disks to a brand new computer with completely different disk controller hardware and completely different buses, CPU, etc... and it still would have found the volumes and put them back into service. Try that with a hardware raid controller.

It's good to put what people say to the test. Trust but verify I think is what Reagan said. It's especially good to test things before you put a lot of data on the disks... :-)

What is a good tool to use to test read/write speeds for the array on the local linux box and across the network from a WinXP box? Also, forgive my ignorance, but would using a pata drive as opposed to a sata drive for the OS drive affect array read/write performance in any way?

[mikesm]

Originally Posted by mtyme

What is a good tool to use to test read/write speeds for the array on the local linux box and across the network from a WinXP box? Also, forgive my ignorance, but would using a pata drive as opposed to a sata drive for the OS drive affect array read/write performance in any way?

For quick an dirty stuff, I just do a dd if=/devnull of=filename bs=512 count=whatever to write a file full of nulls, the converse will read it. For a more comprehensive view, I use bonnie++.

SATA doesn't give you that much better speed than PATA, but you have NCQ and other stuff that can be helpful depending on the config (you don't want NCQ if you are in a RAID5 or RAID0 config). Just make sure UDMA is set and you are running at 133 speed, and you should be fine. However, one thing, you do NOT want to put drives on master and slave of one ATA controller channel. That does awful things to reliability, and causes performance problems too. In that way, SATA is far better.

For testing SAMBA access from XP, I just copy a file. :-)

[mtyme]

Something's not right here....

goliath:/usr/bin # bonnie -d /media/vol1 -s 4024
Bonnie 1.4: File '/media/vol1/Bonnie.22895', size: 4219469824, volumes: 1
Writing with putc()... done: 43483 kB/s 66.3 %CPU
Rewriting... done: 26225 kB/s 10.9 %CPU
Writing intelligently... done: 46109 kB/s 5.7 %CPU
Reading with getc()... done: 51601 kB/s 93.9 %CPU
Reading intelligently... done: 94640 kB/s 28.3 %CPU

I'm running an AMD 4000+ (single core) 2gb ram asus a8n-sli deluxe mb.
Using the onboard SI sata ports with 3 seagate sataII drives.

These numbers are way off right? Shouldn't it be in the 160-200 MB/s range? I've got a PMP and pci express 3132 sata controller on the way, but I would have though the read/writes would be faster than this.

[mtyme]

This doesn't look right -

ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata7.00: ATA-7: ST3500630AS, 3.AAK, max UDMA/133
ata7.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 0/32)
ata7.00: configured for UDMA/100
ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata8.00: ATA-7: ST3500630AS, 3.AAK, max UDMA/133
ata8.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 0/32)
ata8.00: configured for UDMA/100
ata9: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata9.00: ATA-7: ST3500630AS, 3.AAK, max UDMA/133
ata9.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 0/32)
ata9.00: configured for UDMA/100


EDIT: After playing with a few things, I've discovered that the Silicon Image 3114 controller is only SATA I, even though Asus says it's SATAII. And I can't get SuSe to boot with the NVRaid sata ports so I guess I get to wait till the PMP comes in to do anything else.

[mikesm]

Originally Posted by mtyme

This doesn't look right -

ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata7.00: ATA-7: ST3500630AS, 3.AAK, max UDMA/133
ata7.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 0/32)
ata7.00: configured for UDMA/100
ata8: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata8.00: ATA-7: ST3500630AS, 3.AAK, max UDMA/133
ata8.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 0/32)
ata8.00: configured for UDMA/100
ata9: SATA link up 1.5 Gbps (SStatus 113 SControl 310)
ata9.00: ATA-7: ST3500630AS, 3.AAK, max UDMA/133
ata9.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 0/32)
ata9.00: configured for UDMA/100


EDIT: After playing with a few things, I've discovered that the Silicon Image 3114 controller is only SATA I, even though Asus says it's SATAII. And I can't get SuSe to boot with the NVRaid sata ports so I guess I get to wait till the PMP comes in to do anything else.

The PCI bus that the 3114 is hooked to will max out at around 100 MB/s. That will be a bottleneck before SATA I will be.

The nvidia sata ports should work fine (they are not connected to the PCI bus), but you need to set the bios to not use them in raid mode. Linux raid doesn't need the bios set to RAID to work, in fact it won't work that way. Set the ports to enhanced mode, non-raid.

Suse works fine booting on nvidia sata ports - My system disk is set that way.

thx
mike

[mtyme]

Originally Posted by mikesm

The PCI bus that the 3114 is hooked to will max out at around 100 MB/s. That will be a bottleneck before SATA I will be.

The nvidia sata ports should work fine (they are not connected to the PCI bus), but you need to set the bios to not use them in raid mode. Linux raid doesn't need the bios set to RAID to work, in fact it won't work that way. Set the ports to enhanced mode, non-raid.

Suse works fine booting on nvidia sata ports - My system disk is set that way.

thx
mike

Well with the 3114 I am maxing out around 50MB/s. So I'm not sure what the problem there is.

As for the nvidia ports, I had the raid disabled in bios but when I put a drive on one of those ports linux won't finish booting, and my boot drive isn't sata. This is just taking a single sataII drive and plugging it into one of the nvidia ports. It gets stuck where it's trying to set the sata link up and it just keeps retrying over and over.

I picked up a 2 port sataII 3132 pciE controller and hooked up a single WD 500gb sataII drive. I did hdparm -tT on it and the array. The single WD drive hit 89MB/s The array hit 60MB/s.

[mikesm]

Originally Posted by mtyme

Well with the 3114 I am maxing out around 50MB/s. So I'm not sure what the problem there is.

As for the nvidia ports, I had the raid disabled in bios but when I put a drive on one of those ports linux won't finish booting, and my boot drive isn't sata. This is just taking a single sataII drive and plugging it into one of the nvidia ports. It gets stuck where it's trying to set the sata link up and it just keeps retrying over and over.

I picked up a 2 port sataII 3132 pciE controller and hooked up a single WD 500gb sataII drive. I did hdparm -tT on it and the array. The single WD drive hit 89MB/s The array hit 60MB/s.

Did you format with 128kb or 256kb chunk size, and XFS for the filesystem? Did you try running my little diskopt.sh script on startup?