Recommended NAS?

[mikesm]

Originally Posted by toricred

Now that's the link I was looking for. Unfortunately they are now sold out and it looks like the price is for today only. I'll have to keep a closer eye on this in the future.

How about this one? http://shop1.outpost.com/product/5478279

[ScottK]

Okay! This is what I ended up going with:

Motherboard: Gigabyte GA-EP35-DS3P (ICH9R, 6+2 SATA II, Realtek 8111B Gigabit)
CPU: Intel Core2Duo E4500 (2.2GHz)
RAM: 2GB DDR2-667
Case: CoolerMaster Centurion 590, 9x5.25 exposed bays, but includes one 4-in-3 module
PSU: CoolerMaster eXtreme Power Plus 460W
RAID5: 3x 1Tb Samsung SATA2 32Mb Cache; HDD; Serial ATA300 Hard Drive HD103UJ
OS Drive: Currently a whiny old IDE HD, but will switch to a SATA HD (much quieter) once this setup frees up some older HDs.

While the Stacker is pretty awesome, it was way overkill for what I needed. The CM 590 as a mid-tower still has quite a lot of bays, and seems to be really well designed for airflow -- the entire front panel is a mesh with an air filter, not to mention the numerous other vents. It came with one CM 4-in-3 module as well, which was the clincher.

I decided to stick with the Realtek Gb for now; I can easily add a PCI-e card with a better chipset later if needed.

As far as filesystems, I'm going to trial it with ext3 first, but with LVM2 on top of RAID it'll be easy enough to trial other things later considering that initially I'll also have a lot more space than I need due to the three-disk minimum.

Installed Ubuntu 7.10 and have created the RAID set with mdadm, and am quite excited! 190 minutes to go.

[mikesm]

Originally Posted by ScottK

Installed Ubuntu 7.10 and have created the RAID set with mdadm, and am quite excited! 190 minutes to go.

Great... 120 mins to go? For what? The OS should install a lot faster than that!

[ScottK]

Originally Posted by mikesm

Great... 120 mins to go? For what? The OS should install a lot faster than that!

Haha no, the initial RAID recovery to build the array.

[mikesm]

Originally Posted by ScottK

Haha no, the initial RAID recovery to build the array.

Oh, you had an existing array you were migrating? Because if you were just creating the array the first time, it should happen very quickly. 120mins is way longer than normal for that.

[ScottK]

# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] 
md0 : active raid5 sdc1[3] sdb1[1] sda1[0]
      1953519872 blocks level 5, 128k chunk, algorithm 2 [3/2] [UU_]
      [>....................]  recovery =  0.3% (3055488/976759936) finish=200.7min speed=80819K/sec

# mdadm --detail /dev/md0
/dev/md0:
        Version : 00.90.03
  Creation Time : Fri Apr  4 02:51:10 2008
     Raid Level : raid5
     Array Size : 1953519872 (1863.02 GiB 2000.40 GB)
  Used Dev Size : 976759936 (931.51 GiB 1000.20 GB)
   Raid Devices : 3
  Total Devices : 3
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Fri Apr  4 02:53:39 2008
          State : clean, degraded, recovering
 Active Devices : 2
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 1

         Layout : left-symmetric
     Chunk Size : 128K

 Rebuild Status : 2% complete

           UUID : 15e2eb2c:306f0692:34169379:6ac36897 (local to host Terra)
         Events : 0.4

    Number   Major   Minor   RaidDevice State
       0       8        1        0      active sync   /dev/sda1
       1       8       17        1      active sync   /dev/sdb1
       3       8       33        2      spare rebuilding   /dev/sdc1

I was creating a new array for the first time. I just assumed I had to wait for it to finish recovery, but perhaps I didn't. The actual array creation from mdadm was almost instant. Oh well, it was right before bed time anyway.

[mikesm]

Another 1 TB special, this time this weekend at Best Buy. The Seagate 1TB drive for $189...

http://www.flickr.com/photos/2507616...7604359799955/

[Slugger]

I'm looking to build a NAS to off load movies, DVDs, archived recordings, music, etc. from my single Sage drive. Looking in the closet I've found a rather old P3 700/512RAM. Is this a candidate to throw in a PCI SATA controller and install FreeNAS or some variant? I want to buy SATA drives because eventually I'd end up replacing this old server so I don't see a point in stocking it with older IDE drives. The motherboard is old - a few PCI slots and no onboard SATA connections.

If I can throw in a SATA controller and be on my way, which controller would be best? I think I'd just use softraid 1 from FreeNAS, though the 2port raid0/1 3ware controllers look nice, too.

Help appreciated.

[hdtvrocks]

Hi guys,

Saw MikeSM's posts on AVS regarding Linux software RAID5 and figured this would be a better place to post my questions as the other thread was Windows 2008-centric. Read through the first 12 pages of this thread but my eyes are starting to get sore. Anyhow, from what I've gathered, mdadm + LVM2 with XFS 128kbit chunks, with 64kbit cluster size is the recommendation. As well as tweaking with memory buffering with XFS.

Here's my scenario, I use Mediaportal which requires a Windows OS for TVServer. I would like to use ATA over Ethernet (AoE) to run my MP clients diskless. I prefer to have my server running linux for obvious reasons but I have a requirement for Windows for TVServer. I was originally going to use my DFI NF4 Ultra-D S939 mobo/X2 3800+ CPU setup for the server. However, no AHCI support, and it may be a little underpowered for what I want to do as it will not merely be a NAS. I'm thinking about picking up an Asus P5E-VM DO mobo with the Q35 chipset that supports Intel's new I/O virtualization extension VT-d. This will allow me to run Windows 2003 Server in a Xen guest virtual machine but hide my DVB cards from the host OS and give the guest OS direct access to the PCI cards. Then run OpenSUSE host for LVM2/mdm for RAID5 and AoE target work. But before I go ahead with this, just wanted to make sure I'm clear about a few things:

1. How fast is software RAID5? I saw some ppl getting 200MBps reads and 150MBps writes? I just ordered 3 750GB 7K1000 Hitachi's for $100 each after MIR.
2. I didn't buy 4 drives because I was assuming that I would be able to grow my array with LVM2, however I looked at the the LVM on RAID MythTV wiki and they stated "You need to do a little pre-planning as it is not necessarily easy to add drives to the array after it is constructed. LVM also adds to this problem." Is this still true or is that outdated?
3. MikeSM, I was going to stick 4GB of RAM in this server, would you say this would suffice? I could probably allocate 3GB to the host linux OS and get away with 1GB for the W2K3 guest as it will be pretty light duty.
4. The P5E-VM DO has the ICH9DO southbridge, will this support port replication? I saw someone list a bunch of chipsets supported for replication including ICH9R but not the DO.

I started my system rebuilds trying to just reuse parts I had leftover, but now it's grown into a moneysucking pit! =D Trying to sell off some older parts I got lying around to fund my server hardware.

Thanks in advance. Hope you guys don't mind helping out a non SageTV user. I'm still watching these forums for MDAPI support.

[mikesm]

Originally Posted by hdtvrocks

Hi guys,

Saw MikeSM's posts on AVS regarding Linux software RAID5 and figured this would be a better place to post my questions as the other thread was Windows 2008-centric. Read through the first 12 pages of this thread but my eyes are starting to get sore. Anyhow, from what I've gathered, mdadm + LVM2 with XFS 128kbit chunks, with 64kbit cluster size is the recommendation. As well as tweaking with memory buffering with XFS.

Here's my scenario, I use Mediaportal which requires a Windows OS for TVServer. I would like to use ATA over Ethernet (AoE) to run my MP clients diskless. I prefer to have my server running linux for obvious reasons but I have a requirement for Windows for TVServer. I was originally going to use my DFI NF4 Ultra-D S939 mobo/X2 3800+ CPU setup for the server. However, no AHCI support, and it may be a little underpowered for what I want to do as it will not merely be a NAS. I'm thinking about picking up an Asus P5E-VM DO mobo with the Q35 chipset that supports Intel's new I/O virtualization extension VT-d. This will allow me to run Windows 2003 Server in a Xen guest virtual machine but hide my DVB cards from the host OS and give the guest OS direct access to the PCI cards. Then run OpenSUSE host for LVM2/mdm for RAID5 and AoE target work. But before I go ahead with this, just wanted to make sure I'm clear about a few things:

1. How fast is software RAID5? I saw some ppl getting 200MBps reads and 150MBps writes? I just ordered 3 750GB 7K1000 Hitachi's for $100 each after MIR.
2. I didn't buy 4 drives because I was assuming that I would be able to grow my array with LVM2, however I looked at the the LVM on RAID MythTV wiki and they stated "You need to do a little pre-planning as it is not necessarily easy to add drives to the array after it is constructed. LVM also adds to this problem." Is this still true or is that outdated?
3. MikeSM, I was going to stick 4GB of RAM in this server, would you say this would suffice? I could probably allocate 3GB to the host linux OS and get away with 1GB for the W2K3 guest as it will be pretty light duty.
4. The P5E-VM DO has the ICH9DO southbridge, will this support port replication? I saw someone list a bunch of chipsets supported for replication including ICH9R but not the DO.

I started my system rebuilds trying to just reuse parts I had leftover, but now it's grown into a moneysucking pit! =D Trying to sell off some older parts I got lying around to fund my server hardware.

Thanks in advance. Hope you guys don't mind helping out a non SageTV user. I'm still watching these forums for MDAPI support.

Hi there, and welcome to the world of NAS. :-)

On #1, the performance depends on the number of drives and the various bottlenecks. 200 MBps reads are doable, but probably need about 6 drives in RAID5, with fast Sata controllers.

#2, you can easily do OCE with linux software RAID5 and LVM2. No issues at all. You first expand the array, and the extend the number of extents in the LVM volume. Then you grow the filesystem (XFS). I and others here have done this a bunch of times with no issues or downtime. It works fine.

on #3, 4 GB of ram is plenty, and will accommodate a windows VM in 1 GB just fine.

on #4, the ICH9 series, including the ICH9R technically handles port multiplier support, but only in command mode, and not with FIS. This is a HUGE performance hit, so you should stay away from PMP's on these ports. They are fine for single drives. If you have an ICH9 bord, get a cheap PCI-E SI 3132 based card and run the PMP's off them, but there is a 120 MB/s per port limit on those cards.

The new ICH10R's on the P45 chipsets will do PMP support with FIS and should be the fastest sata port solution around. If you can wait a month or two, then I would definitely get a P45 or G45 board instead. Also, the G35 is a bit of a dog... Asus is the only vendor who built one on the G35, and tons of people are building G45 boards, so I'd wait for that if I could.

Does all that help?

thx
mike

[hdtvrocks]

Originally Posted by mikesm

The new ICH10R's on the P45 chipsets will do PMP support with FIS and should be the fastest sata port solution around. If you can wait a month or two, then I would definitely get a P45 or G45 board instead. Also, the G35 is a bit of a dog... Asus is the only vendor who built one on the G35, and tons of people are building G45 boards, so I'd wait for that if I could.

Does all that help?

thx
mike

Thanks Mike! Helps alot! Actually I'm looking at the Q35 chipset which supports Intel VT-d, the Q45 will also support VT-d but won't be available until quite some time after the P45's unfortunately. Wish I didn't have this Windows requirement, would make my life so much easier and I could stay on my S939 for my server.

[mikesm]

Originally Posted by hdtvrocks

Thanks Mike! Helps alot! Actually I'm looking at the Q35 chipset which supports Intel VT-d, the Q45 will also support VT-d but won't be available until quite some time after the P45's unfortunately. Wish I didn't have this Windows requirement, would make my life so much easier and I could stay on my S939 for my server.

I'd just get a P45 and throw a $20 PCIE video card in it. It's just a server, and while it burns a PCIE slot, the P series also usually overclocks better than the G series.

thanks,
mike

[erik]

Did anyone reading this thread ever looked at unRAID?
Not really a traditional RAID5 solution but disk adding/replacement seems much simpler.

[hdtvrocks]

Originally Posted by mikesm

I'd just get a P45 and throw a $20 PCIE video card in it. It's just a server, and while it burns a PCIE slot, the P series also usually overclocks better than the G series.

thanks,
mike

Well actually I want Intel VT-d so P and G series doesn't seem to fit my reqs, need a Q series. Won't be overclocking a 24x7 server anyhow, if anything I'll be undervolting.

I have a question though, how do you read bonnie++ output?

Version 1.01d ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
host 4024M 17951 18 62505 26 26044 16 42170 55 55206 19 90.3 1
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
host,4024M,17951,18,62505,26,26044,16,42170,55,55206,19,90.3,1,16, +++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++

[hdtvrocks]

Got my array up and running with lvm2. 3 x Hitachi 7K1000 750GB drives will get another one in next few weeks or so. I wasn't able to mount my XFS filesystem with 64kb block size, is this not supported under linux yet like the manpage says? Tried searching how to change the linux pagesize but didn't find much info.

My bonnie++:
Version 1.03b ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
bkubuntu 4024M 41246 67 79456 14 35037 13 43403 76 154957 26 497.5 2
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 1758 19 +++++ +++ 1368 14 1571 15 +++++ +++ 1192 7
bkubuntu,4024M,41246,67,79456,14,35037,13,43403,76,154957,26,497.5,2,16,1758,19,+++++,+++, 1368,14,1571,15,+++++,+++,1192,7

[KJake]

@hdtvrocks: place code blocks around your text to keep formatting, it will make it easier to read on the forums (removes all HTML formatting).

Sample:

root@orion:~# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md2 : active raid5 dm-6[2] dm-7[3] dm-5[1] dm-4[0]
      2930279424 blocks level 5, 256k chunk, algorithm 2 [4/4] [UUUU]

[KJake]

@mikesm:
Do you see any reason why this wouldn't work for our setup (EVMS(XFS)/LVM2/MDADM)?

http://www.wlug.org.nz/SambaShadowCopyHowto

[mikesm]

Originally Posted by KJake

@mikesm:
Do you see any reason why this wouldn't work for our setup (EVMS(XFS)/LVM2/MDADM)?

http://www.wlug.org.nz/SambaShadowCopyHowto

Neat - I've never seen such a write up before, but yes, if you followed my advice in this thread in setting up a server using linux, XFS, LVM, and software RAID5, this should all work. No need to use EVMS, though it provides some snapshotting of it's own.

Next time I have some free time, I'll play with this. Very cool. Thanks for posting it.

[KJake]

Originally Posted by mikesm

Neat - I've never seen such a write up before, but yes, if you followed my advice in this thread in setting up a server using linux, XFS, LVM, and software RAID5, this should all work. No need to use EVMS, though it provides some snapshotting of it's own.

Next time I have some free time, I'll play with this. Very cool. Thanks for posting it.

I wish they had posted their scripts too

I'm not feeling very courageous right now, but I'll probably try to set this up. I have a separate drive that I'm backing up important files to, but shadow copies can still be useful.

Here's official documentation on the shadow_copy VFS module.
http://www.samba.org/samba/docs/man/....html#id430040

[mikesm]

Originally Posted by erik

Did anyone reading this thread ever looked at unRAID?
Not really a traditional RAID5 solution but disk adding/replacement seems much simpler.

Unraid is based on Freebsd, which has lackluster hardware support as compared with Linux (No Sata PMP's, etc...). It's filesystem is not well suited for media serving apps, something XFS excels at. Raid Performance is Ok, but the lack of tuning for large sequential media operations that XFS does well at hurts in benchmarks.

I don't think RAID management is that bad. Unraid may be easier for 4-5 disks, but when you hit 10 disks or more, the volume management drives you crazy. With Linux XFS and LVM on top of RAID5, you get enterprise grade features that makes it easy to export a single volume, and hide all the expansion and swapping under the covers. Good volume management means clients don't have to clean up a mess in organization at the server. Unraid is uninteresting in this aspect.

Also, many of us like to run other apps on the server as well, like commercial processing, etc..., so unraid isn't a good platform for that.

I also think that spinning up drives selectively is a lot more trouble than it's worth. I'd much rather have real RAID and real volume management.

For smaller installs, it may be fine, but then so would hanging a few disks on a windows machine. The hard part is dealing with lots of disks and huge filesystems. That needs enterprise grade tools that linux has.

Now, if unraid added ZFS support, that would be another story and clean up a bunch of the problems with both filesystem and volume management, but for me at least, lack of SATA PMP support is a killer, and that doesn't appear to be changing anytime soon.