Recommended NAS?

[KJake]

So you just cracked all four of them open? I wonder what that does for the warranty of them? Very cool though. I may have to check my local best buy...

[mikesm]

Originally Posted by KJake

So you just cracked all four of them open? I wonder what that does for the warranty of them? Very cool though. I may have to check my local best buy...

There is an easy guide to extracting the drive here: http://www.johnle.com/guide/hitachi/

BTW, the warranty is on the disks too, not just the assembly. You can register the drive just fine - it's the same serial number as the chassis. Hitachi made it easy on us this time.

You can stick in an old drive in the case too - it works with any SATA drive.

Thanks,
Mike

[KJake]

Sweet, I found 3 of them at my closest BestBuy. I just realised that I only have two spots left in my case. I guess it is time to get a 5SA from Addonics. I knew I would have to. I need some more SATA cables too, so this will just have to wait for a while, but the $199 deal was awesome, thanks!

[mikesm]

Originally Posted by KJake

Sweet, I found 3 of them at my closest BestBuy. I just realised that I only have two spots left in my case. I guess it is time to get a 5SA from Addonics. I knew I would have to. I need some more SATA cables too, so this will just have to wait for a while, but the $199 deal was awesome, thanks!

The addonics 5sa comes with sata cables, so no need to buy more. If you have a fry's nearby, they have a ams 5in3 hotswap rack for $95 that is a clone of the addonics (and comes with cables too)...

thx
mike

[mikesm]

Warp from other thread...

Originally Posted by BitBass

Mikesm, no problem with taking the conversation over there.

Six, I'm with you on that. My preference is something Debian based such as Ubuntu, but I'm flexible. I certainly don't get into distro wars!

See you in the other thread!

I understand that they may have picked up the libata-tj patches in the latest ubuntu kernel, but I don't use it so I can't verify that. 2.6.24 should have the full PMP merge in it, but it's still on rc5 right now. I suspect when 2.6.24 goes production most of the distros will have update options for it. This is really only an issue for port multipliers of course, but most folks here will need the ports...

I also like the idea of having a document that covers all this stuff. I suspect now that extenders are making Sage popular that lots of folks will be looking for some NAS options...

Also, I have done some work on my config. It's now rackmounted in an Enlight EN-8990 4U chassis, which can hold 3 of the 5in3 rack modules, though with no room for an optical drive (ah, it's so nice being to install linux from flash anyway)... I still love the stacker for this application, but I had to neaten up the garage by moving everything into a rack. I gave my stacker to a friend building a NAS... :-)

I'm up to 8 500GB disks plus 4 1TB disks. It's so cool to hit info on the SageMC home screen and see the diskbar say 3 TB used but only about half the space colored green. :-)

Also, I have started to use the SADMS application (Samba as Active Directory Member Server), which makes it really easy to configure SAMBA to take authentication info from a windows 2003 ADS server. No more having to deal with user accts on the NAS.

Thanks,
Mike


Thanks,
Mike

[KJake]

That's cool, I'm still running the custom 2.6.22-rc6 that I built for my Ubuntu installation. Will be nice to be on the stock kernel path again. I still haven't mounted the three 1TB disks that I bought the same time you did. I've been way too busy. I get time off around christmas so I should be able to do it then. I still have about 1.8TB free, but those drives were such a good deal, I couldn't pass on it.

Question about that migration process though. You said you were going to retire some old drives. How did you go about migrating the data from them? Does EVMS take care of all of that, where all you had to do was add in the new drives normally and remove the old drives and it said "please wait" for 20 minutes while it migrated everything? (I could probably look this up...but...*effort*).


Side-note:
Even with power outages, my NAS has been running full time since I turned it on downstairs. The outages have been less than 30 minutes, which my UPS can handle.

root@orion:/boot# uname -a
Linux orion 2.6.22-rc6 #1 SMP Sun Jul 22 18:50:14 EDT 2007 x86_64 GNU/Linux
root@orion:/boot# uptime
 08:38:32 up 110 days, 14:05,  1 user,  load average: 0.00, 0.05, 0.02
root@orion:/boot# ifconfig eth1
eth1    snip
          RX bytes:6216355724974 (5.6 TiB)  TX bytes:14758593649381 (13.4 TiB)
          snip

[mikesm]

Originally Posted by KJake

That's cool, I'm still running the custom 2.6.22-rc6 that I built for my Ubuntu installation. Will be nice to be on the stock kernel path again. I still haven't mounted the three 1TB disks that I bought the same time you did. I've been way too busy. I get time off around christmas so I should be able to do it then. I still have about 1.8TB free, but those drives were such a good deal, I couldn't pass on it.

Question about that migration process though. You said you were going to retire some old drives. How did you go about migrating the data from them? Does EVMS take care of all of that, where all you had to do was add in the new drives normally and remove the old drives and it said "please wait" for 20 minutes while it migrated everything? (I could probably look this up...but...*effort*).


Side-note:
Even with power outages, my NAS has been running full time since I turned it on downstairs. The outages have been less than 30 minutes, which my UPS can handle.

root@orion:/boot# uname -a
Linux orion 2.6.22-rc6 #1 SMP Sun Jul 22 18:50:14 EDT 2007 x86_64 GNU/Linux
root@orion:/boot# uptime
 08:38:32 up 110 days, 14:05,  1 user,  load average: 0.00, 0.05, 0.02
root@orion:/boot# ifconfig eth1
eth1    snip
          RX bytes:6216355724974 (5.6 TiB)  TX bytes:14758593649381 (13.4 TiB)
          snip

EVMS should be able to do that pretty easily. You are using LVM containers right? Just add more storage to the container and then remove the old RAID array and it should move everything over. Note it took my server almost a day to move 1.3 TB around, but it did it with the volume and serving all online. There are limits as to have fast this goes when you keep everything online.

110 day uptime? No urgent security patches that needed to be installed and generate a reboot? Oh, I forgot, this isn't windows. We can patch while keeping the system online... :-)

Yeah, 6 TB seems to be a sweet spot for HD packrats like us... :-)

Thx
Mike

[KJake]

Yup, I'm using LVM containers. I think that I'm going to put the new 1TB drives online and just put my 500GB drives offline and pull the power connector on them. Having 3 drives running instead of 6 will reduce the power consumption a little bit and I can just move them online if I end up needing the space.

I was being a bit facetious with the 20 minute time Even with the awesome throughput that we have on these systems, that's still a lot of data...and if your NAS is like mine, it isn't only for SageTV...so it would probably in-use during the migration.

[BitBass]

Take a look at this:

http://www.mythtv.org/wiki/index.php/LVM_on_RAID

Looks like it might be a good start to documenting this design. I wouldn't mind having something that discusses the hardware issues too.

[mikesm]

Originally Posted by BitBass

Take a look at this:

http://www.mythtv.org/wiki/index.php/LVM_on_RAID

Looks like it might be a good start to documenting this design. I wouldn't mind having something that discusses the hardware issues too.

Not too bad, but the GUI utilities to create raid arrays and deal with LVM volumes in Suse are pretty nice. It's a lot easier than doing it via the command line, though of course that still works. Some of the other distros have similar utilities (fedora for example).

A lot of the hardware items have been mentioned earlier in this thread, but here are some things I personally consider:

1) Stay away from the PCI bus. At only 133 MBytes/sec, it can be a big bottleneck for "overkill" performance. You want both the disks and the Gigabit ethernet controller on PCIE.

2) AHCI or SI3132 based SATA ports on the motherboard. These are fully supported for use with Port Multipliers, which really make these big RAID systems cost effective. If you need 12 drives and need 12 SATA ports to feed them all, you will pay a lot of money for a RAID controller that you don't really need.

3) Disks need to stay cool to last a long time. I really like the 5in3 SATA racks, not just because they make hotswap easy, but generally they provide good airflow and ventilation over the disks.

4) You want a big enough case to be able to hold a few of the 5in3 racks. I prefer the Coolermaster Stacker STC-T01 classic case, since you can put 3 (or even 4) of the 5in3 racks in them, and it has a 2nd slot for a second PSU if you need the amps.

5) CPU isn't really stressed in any of this, so you can go Intel or AMD. However, the Intel based boards tend to come with AHCI compatible SATA controllers, which are good. Some of the Nvidia based AMD motherboards also come with AHCI ports, but you have to be careful to make sure AHCI is support is there. You also want a few PCIE slots for SATA expansion if you need to go past what the motherboard has.

6) 1 GB of RAM is fine for most configs, but memory is cheap, so if you stick in 2 GB of ram, you can tweak the settings for readahead etc... and throw 1 GB of ram just for disk beuffering which makes things fly!

7) Always, always, ALWAYS (I mean it!) use a UPS. With the amount of RAM buffering you want to do with XFS, you really want controlled shutdown for powerfailure. Linux has a number of utilities that talk to APC supplies, both RS232 and USB based versions, so get one and configure it, test it, and make sure you pay attention to it. It will keep you out of trouble if you have a boatload of writing going on.

8) Don't bother with PATA disks if you can avoid it. The number of IDE ports you can put on a system is much smaller, you can't use PMP's, and if you use both the secondary and primary ports, you can end up with one disk failing and taking out it's neighbor on the bus as well. That's a big problem for RAID5, where 1 disk out is fine, but 2 down is a problem.

9) Get the highest density disk that is close to the disk pricing sweet spot you can. The fewer the spindles, the lower the power consumption, and that's good since you'll be keeping this 24x7. Also, denser disks get cheaper over time, and fewer spindles mean fewer 5in3 racks needed. The sweet spot today is 500Gb for $100, but occasionally you can find 1TB disks for $200. That is a very good deal and means a single 5in3 rack fully populated in RAID5 nets you 4TB of storage. Nice.

Hope this helps.

Thanks,
Mike

[BitBass]

Mikesm, very cool!

Anyone know if there's a motherboard where the onboard GE adapter is on the PCIE bus?

[mikesm]

Originally Posted by BitBass

Mikesm, very cool!

Anyone know if there's a motherboard where the onboard GE adapter is on the PCIE bus?

It is hit and miss. You have to look at the specifics. Sometimes they are on the PCIE and sometimes not. Most of the late model intel boards are on the PCIE as far as I can tell. The P35 Gbe is almost always direct to the southbridge - no PCI bus involved.

thx
mike

[BitBass]

Mikesm, did I see somewhere that you run the OS off of a flash drive and the spindles are strictly for the arrays?

Also, I'm looking into the opensuse gui method. I'm a router jockey so I'm perfectly happy with CLI but why beat your head against the wall if you don't have to, right?

[BitBass]

This motherboard looks like it has possibilities:
http://www.newegg.com/Product/Produc...82E16813135059

[mikesm]

Originally Posted by BitBass

Mikesm, did I see somewhere that you run the OS off of a flash drive and the spindles are strictly for the arrays?

Also, I'm looking into the opensuse gui method. I'm a router jockey so I'm perfectly happy with CLI but why beat your head against the wall if you don't have to, right?

No, I use one of the SATA slots for a system disk. But I don't have an optical drive on the system. I install opensuse from flash instead of DVD.

The CLI isn't the issue, it's with LVM you need to specify numbers of extenets and such, so it's just easy to point and click rather than typing in numbers that the previous command told you... :-)

thx
mike

[mikesm]

Originally Posted by BitBass

This motherboard looks like it has possibilities:
http://www.newegg.com/Product/Produc...82E16813135059

It looks like it has a non-PCI GbE, but it's an ECS board, which isn't the best quality, and only 2 PCIE slots, one of which is tied up for video. An Integrated video G35 based motherboard would avoid the need for a separate graphics card (onboard graphics are fine for a NAS console), and may have additional PCIE slots which are handly for SATA port expansion.

Some of the ABit P35 boards are onsale at various times and work well too.

Thanks,
Mike

[BitBass]

Originally Posted by mikesm

It looks like it has a non-PCI GbE, but it's an ECS board, which isn't the best quality, and only 2 PCIE slots, one of which is tied up for video. An Integrated video G35 based motherboard would avoid the need for a separate graphics card (onboard graphics are fine for a NAS console), and may have additional PCIE slots which are handly for SATA port expansion.

Some of the ABit P35 boards are onsale at various times and work well too.

Thanks,
Mike

Ok, am I understanding this right? P35's don't have onboard video but G35's do? I found a single g35 chipset mobo with everything. 3 PCIE slots.

http://www.newegg.com/Product/Produc...82E16813131237

How safe would a G33 chipset mobo be? There's seems to be a lot more options for that. Looks like AHCI support is hit or miss on that.

[mikesm]

Originally Posted by BitBass

Ok, am I understanding this right? P35's don't have onboard video but G35's do? I found a single g35 chipset mobo with everything. 3 PCIE slots.

http://www.newegg.com/Product/Produc...82E16813131237

How safe would a G33 chipset mobo be? There's seems to be a lot more options for that. Looks like AHCI support is hit or miss on that.

Depends on the board... G33 video is fine though... That asus board would make a great sage client though!

thx
mike

[BitBass]

Ok, if I were to get a motherboard that had several PCIE slots and just used 3132 cards, port multipliers and a PCIE GE card, would I need to be worried about the other details of the motherboard? Aside from onboard video just to save a slot.

I'm just wondering if it works out cheaper that way. You don't have to find the "right" motherboard. Just get any old one with a bunch of PCIE's.

For instance:

$20 Rosewill 3132 card: http://www.newegg.com/Product/Produc...82E16816132008

$30 GE card: http://www.newegg.com/Product/Produc...82E16833166019

$90 motherboard: http://www.newegg.com/Product/Produc...82E16813131096

Don't know if that would do the trick or not.

[mikesm]

Originally Posted by BitBass

Ok, if I were to get a motherboard that had several PCIE slots and just used 3132 cards, port multipliers and a PCIE GE card, would I need to be worried about the other details of the motherboard? Aside from onboard video just to save a slot.

I'm just wondering if it works out cheaper that way. You don't have to find the "right" motherboard. Just get any old one with a bunch of PCIE's.

For instance:

$20 Rosewill 3132 card: http://www.newegg.com/Product/Produc...82E16816132008

$30 GE card: http://www.newegg.com/Product/Produc...82E16833166019

$90 motherboard: http://www.newegg.com/Product/Produc...82E16813131096

Don't know if that would do the trick or not.

I don't think it's actually that hard.. :-) Onboard non-PCI GbE is pretty easy to find, and if you have a modern Intel MB, it will have AHCI SATA ports. Getting all of this plus integrated graphics is a little harder, but what I did on my NAS was use an old PCI VGA card I had lying around, and for a NAS that works fine, and uses a slot that isn't all that helpful to the central NAS role. If you don't have one, they can be found very cheaply.

If you can find though a board that has AHCI SATA, non-PCI GbE, a few PCI-E slots and integrated graphics, that will be a winner if it's not too expensive...

Thx
mike