Recommended NAS?

[Fuzzy]

I understand that Linux is the solution for what you are wanting, but I didn't see in the thread what the problem it was the solution for. That is what I was asking for. I have been using direct attached storage in windows on my sage server since day one, and never felt the need for the NAS.

[mikesm]

Price alert!

Seagate 1 TB 7200.11's now $139.99 with free shipping and no tax in ca and many other states here:http://www.pcconnection.com/IPA/Shop...SourceID=k1971

[chrisc16]

So I finally got 3 Seagate 1TB disks and built my first linux raid 5 array - it was easy! Almost too easy, and I'm wondering if I did everything correctly. I'm copying a lot of files over from my ReadyNAS, and it's writing to the array at about 14.4 MB/sec according to nautilus. Seems pretty fast to me. I remember way back in the beginning of this thread, that mikesm had an optimization script. Is this something I should run at startup too? And how else can I test the speed? I assume it should be with local files, to eliminate the network speed.

Thanks,
Chris

[mikesm]

Originally Posted by chrisc16

So I finally got 3 Seagate 1TB disks and built my first linux raid 5 array - it was easy! Almost too easy, and I'm wondering if I did everything correctly. I'm copying a lot of files over from my ReadyNAS, and it's writing to the array at about 14.4 MB/sec according to nautilus. Seems pretty fast to me. I remember way back in the beginning of this thread, that mikesm had an optimization script. Is this something I should run at startup too? And how else can I test the speed? I assume it should be with local files, to eliminate the network speed.

Thanks,
Chris

Sure, you can run the optimizer script - it should speed things up. A quick test of speed is to "dd if=filename of=/dev/null bs=1m" where the filename is a file on the array, preferably multi-gigabyte in size. That will tell you your net read performance through the filesystem.

And yes, it's really not that hard. :-)

[hdtvrocks]

Just added 2 750GB drives, did a successful vgextend and lvextend. However when I run xfs_growfs <mountpoint> I get the following error:

meta-data=mountpoint isize=256 agcount=38, agsize=4915200 blks
= sectsz=512 attr=0
data = bsize=4096 blocks=183500800, imaxpct=25
= sunit=0 swidth=0 blks, unwritten=1
naming =version 2 bsize=4096
log =internal bsize=4096 blocks=32768, version=1
= sectsz=512 sunit=0 blks, lazy-count=0
realtime =none extsz=4096 blocks=0, rtextents=0
xfs_growfs: XFS_IOC_FSGROWFSDATA xfsctl failed: Input/output error

UPDATE: I rebooted the machine for something else and now my RAID array isn't coming up I hope I haven't lost all my data.

From /var/log/messages:

Sep 27 01:23:05 bkcentos kernel: md: Autodetecting RAID arrays.
Sep 27 01:23:05 bkcentos kernel: md: autorun ...
Sep 27 01:23:05 bkcentos kernel: md: considering sde1 ...
Sep 27 01:23:05 bkcentos kernel: md: adding sde1 ...
Sep 27 01:23:05 bkcentos kernel: md: adding sdc1 ...
Sep 27 01:23:05 bkcentos kernel: md: adding sdb1 ...
Sep 27 01:23:05 bkcentos kernel: md: adding sda1 ...
Sep 27 01:23:05 bkcentos kernel: md: created md0
Sep 27 01:23:05 bkcentos kernel: md: bind<sda1>
Sep 27 01:23:05 bkcentos kernel: md: bind<sdb1>
Sep 27 01:23:05 bkcentos kernel: md: bind<sdc1>
Sep 27 01:23:05 bkcentos kernel: md: bind<sde1>
Sep 27 01:23:05 bkcentos kernel: md: running: <sde1><sdc1><sdb1><sda1>
Sep 27 01:23:05 bkcentos kernel: md: kicking non-fresh sde1 from array!
Sep 27 01:23:05 bkcentos kernel: md: unbind<sde1>
Sep 27 01:23:05 bkcentos kernel: md: export_rdev(sde1)
Sep 27 01:23:05 bkcentos kernel: md: kicking non-fresh sdc1 from array!
Sep 27 01:23:05 bkcentos kernel: md: unbind<sdc1>
Sep 27 01:23:05 bkcentos kernel: md: export_rdev(sdc1)
Sep 27 01:23:05 bkcentos kernel: md: kicking non-fresh sda1 from array!
Sep 27 01:23:05 bkcentos kernel: md: unbind<sda1>
Sep 27 01:23:05 bkcentos kernel: md: export_rdev(sda1)
Sep 27 01:23:05 bkcentos kernel: raid5: automatically using best checksumming function: generic_sse
Sep 27 01:23:05 bkcentos kernel: generic_sse: 5742.800 MB/sec
Sep 27 01:23:05 bkcentos kernel: raid5: using function: generic_sse (5742.800 MB/sec)
Sep 27 01:23:05 bkcentos kernel: raid6: int64x1 2042 MB/s
Sep 27 01:23:05 bkcentos kernel: raid6: int64x2 2724 MB/s
Sep 27 01:23:05 bkcentos kernel: raid6: int64x4 2309 MB/s
Sep 27 01:23:05 bkcentos kernel: raid6: int64x8 1876 MB/s
Sep 27 01:23:05 bkcentos kernel: raid6: sse2x1 3757 MB/s
Sep 27 01:23:05 bkcentos kernel: raid6: sse2x2 4131 MB/s
Sep 27 01:23:05 bkcentos kernel: raid6: sse2x4 6322 MB/s
Sep 27 01:23:05 bkcentos kernel: raid6: using algorithm sse2x4 (6322 MB/s)
Sep 27 01:23:05 bkcentos kernel: md: raid6 personality registered for level 6
Sep 27 01:23:05 bkcentos kernel: md: raid5 personality registered for level 5
Sep 27 01:23:05 bkcentos kernel: md: raid4 personality registered for level 4
Sep 27 01:23:05 bkcentos kernel: raid5: reshape will continue
Sep 27 01:23:05 bkcentos kernel: raid5: device sdb1 operational as raid disk 3
Sep 27 01:23:05 bkcentos kernel: raid5: not enough operational devices for md0 (3/4 failed)
Sep 27 01:23:05 bkcentos kernel: RAID5 conf printout:
Sep 27 01:23:05 bkcentos kernel: --- rd:4 wd:1 fd:3
Sep 27 01:23:05 bkcentos kernel: disk 3, o:1, dev:sdb1
Sep 27 01:23:05 bkcentos kernel: raid5: failed to run raid set md0
Sep 27 01:23:05 bkcentos kernel: md: pers->run() failed ...
Sep 27 01:23:05 bkcentos kernel: md: do_md_run() returned -5
Sep 27 01:23:05 bkcentos kernel: md: md0 stopped.
Sep 27 01:23:05 bkcentos kernel: md: unbind<sdb1>
Sep 27 01:23:05 bkcentos kernel: md: export_rdev(sdb1)
Sep 27 01:23:05 bkcentos kernel: md: ... autorun DONE.
Sep 27 01:23:05 bkcentos kernel: md: md0 stopped.

mdadm --examine results for each disk: http://pastebin.com/m7af8982

[hdtvrocks]

I was able to get my array up again in degraded state. I had to modify /etc/mdadm.conf and change it from 3 devices to 4 devices. Was able to force md0 to start and it's now in recovering state with the 4th drive removed. However, my logical volumes are Not Available, I hope my data is still there. I tried to readd the missing 4th device but for some strange reason it states that it doesn't exist. I'll wait until the array is finished building. Can someone tell me what's the procedure to grow an XFS volume? xfs_growfs obviously doesn't seem to work for me

[mikesm]

Originally Posted by hdtvrocks

I was able to get my array up again in degraded state. I had to modify /etc/mdadm.conf and change it from 3 devices to 4 devices. Was able to force md0 to start and it's now in recovering state with the 4th drive removed. However, my logical volumes are Not Available, I hope my data is still there. I tried to readd the missing 4th device but for some strange reason it states that it doesn't exist. I'll wait until the array is finished building. Can someone tell me what's the procedure to grow an XFS volume? xfs_growfs obviously doesn't seem to work for me

First off, all you needed to do was to do something like an mdadm --re-add /dev/md0 /dev/sd3a or whichever disk went offline. You should review the log files to see why the disk was removed from the array. There could be a problem with the disk or the cabling that needs attention.

You can also force a restart of a raid5 array in degraded mode by using the force option when doing an --assemble operation in mdadm. Read the mdadm man page, it's a ton of good stuff in it.

If the array is up, try doing an lvm start and see if that brings the lvm volume up, and then a mount -a after that.

Weird error from xfs.growfs. You did a pvresize right after adding space to the raid5 volume? vgextend doesn't do what you want.

This guys page shows how to grow a raid5 array, resize and grow the lvm volume and then finally grow the xfs filesystem. http://scotgate.org/?p=107

vgextend adds physical volumes to the volume group. You do NOT want to do that, as the physical disks are being added to the existing raid5 array. If you attempt to extend the volume to a disk already part of a raid5 array, bad things will happen to the volume and the array if something doesn't prevent you from doing it. In a single raid5 configuration, you have only one logical volume and one physical volume. You want to resize the physical volume, then RESIZE the logical volume, and then grow the filesystem. If took a new physical disk, and extended a logical volume onto it, you end up with two physical volumes, and that disk cannot be part of the raid array.

[stanger89]

Well, (hopefully) I'm finally getting ready to update/upgrade/replace my storage system. Long story short I'm down to two options:

1) Infrant/Netgear ReadyNAS Pro
2) Replace/update the RAID array in my current server.

I've already got a ReadyNAS X6, and I really like the features and functionality of it, but the performance from it's Arm (I think) processor leaves a bit to be desired. The ReadyNAS Pro looks killer based on an Intel processor, reports are it can easilly hit over 100MB/sec. With 6 bays and 1.5TB drives, that's 7.5TB total which would roughly double my current capacity. It's not cheap, but would be large and fast enough to support everything I need from a storage system.

My other option is probably an Adaptec 5805 card and associated equipment. This would get me a top of 10.5TB and potentially eliminate one box from my setup. The card alone is much cheaper than the ReadyNAS pro option, but that's not the only thing I'd need. I'd like to put hotswap enclosures in, and I'd need a PCIe motherboard, so that would probably drive me to upgrade my server as well.

So, main question is does anyone know any good SAS or SATA hotswap drive enclosures? Probably 4-in-3 type.

[mikesm]

Originally Posted by stanger89

Well, (hopefully) I'm finally getting ready to update/upgrade/replace my storage system. Long story short I'm down to two options:

1) Infrant/Netgear ReadyNAS Pro
2) Replace/update the RAID array in my current server.

I've already got a ReadyNAS X6, and I really like the features and functionality of it, but the performance from it's Arm (I think) processor leaves a bit to be desired. The ReadyNAS Pro looks killer based on an Intel processor, reports are it can easilly hit over 100MB/sec. With 6 bays and 1.5TB drives, that's 7.5TB total which would roughly double my current capacity. It's not cheap, but would be large and fast enough to support everything I need from a storage system.

My other option is probably an Adaptec 5805 card and associated equipment. This would get me a top of 10.5TB and potentially eliminate one box from my setup. The card alone is much cheaper than the ReadyNAS pro option, but that's not the only thing I'd need. I'd like to put hotswap enclosures in, and I'd need a PCIe motherboard, so that would probably drive me to upgrade my server as well.

So, main question is does anyone know any good SAS or SATA hotswap drive enclosures? Probably 4-in-3 type.

I'd go with a 5in3 sata enclosure like the Supermicro or AMS one. They work just fine. But for the price, you might think about going to a norco 4u 4020 case that comes with 20 sata trays and backplane, and is cheaper than going the 5in3 rack route. You get lots of room for expansion, but it's a rackmount system.

thx
mike

[hdtvrocks]

Originally Posted by mikesm

First off, all you needed to do was to do something like an mdadm --re-add /dev/md0 /dev/sd3a or whichever disk went offline. You should review the log files to see why the disk was removed from the array. There could be a problem with the disk or the cabling that needs attention.

I tried that. Didn't work. It would fail with no devices in array error.

Originally Posted by mikesm

You can also force a restart of a raid5 array in degraded mode by using the force option when doing an --assemble operation in mdadm. Read the mdadm man page, it's a ton of good stuff in it.

Tried that too. Again no devices in array error. But when I did edited /etc/mdadm.conf to show devices=4, --assemble --force worked.

Originally Posted by mikesm

If the array is up, try doing an lvm start and see if that brings the lvm volume up, and then a mount -a after that.

Weird error from xfs.growfs. You did a pvresize right after adding space to the raid5 volume? vgextend doesn't do what you want.

This guys page shows how to grow a raid5 array, resize and grow the lvm volume and then finally grow the xfs filesystem. http://scotgate.org/?p=107

vgextend adds physical volumes to the volume group. You do NOT want to do that, as the physical disks are being added to the existing raid5 array. If you attempt to extend the volume to a disk already part of a raid5 array, bad things will happen to the volume and the array if something doesn't prevent you from doing it. In a single raid5 configuration, you have only one logical volume and one physical volume. You want to resize the physical volume, then RESIZE the logical volume, and then grow the filesystem. If took a new physical disk, and extended a logical volume onto it, you end up with two physical volumes, and that disk cannot be part of the raid array.

I checked my history, I did a pvresize, then lvextend, sorry not vgextend. Anyhow, I currently have my array back and all the data seems to be there. However, I did a mdadm --stop on it and then restarted it and the 4th disk was removed again. I'm not sure why, mdadm --detail shows 4 active, synced disks and mdadm --examine on the 4th drive matches the others. I think I know what it is, I didn't zero out the superblock and didn't properly label the disk as fd.

[jgourd]

I personally would not record to a NAS. This is especially true for me since I have two HDHR tuners and that alone eats up bandwidth. I have great luck with USB2 externals in BTV and I expect I'll have the same luck with Sage once I shut down the BTV server and move some of its drives to Sage.

I do recommend a NAS for static media like music and video collections. I personally recommend the Drobo with a DroboShare for that. I recommend the Drobo for the following reasons:
1. The data is protected
2. Unlike RAID5 you can have discs of different sizes.
3. When it fills up, you take out the smallest or oldest drive and replace it with the largest one you can get for the best price.
4. The DroboShare has an open API and there are streaming apps you can get for it.
5. The DroboShare can run two Drobos at once.
6. When replacing a drive you still have access to all the data while it is working to re-protect the array.

[stanger89]

Well, 1 and 6 really aren't reasons to choose a Drobo over a NAS, as a decent NAS (home built or otherwise) does the same thing.

2, 3, that's nice, but some of the better NAS options out there can do something similar via multiple volumes.

4, most of the good NAS options (especially home built) can do that as well.

5, I don't really understand what it's better than.

But here's the kicker, the Droboshare is (according to the Engadget review) quite slow. They got 6-8MB/sec from the Droboshare over Gig-E, in comparison a ReadyNAS NV+ will do more like 30MB/sec (either way) and the new ReadyNAS Pro (or home built NAS) should easilly do 100MB/sec.

And at about $700 for the combo, it's not really any cheaper either.

[stanger89]

Originally Posted by mikesm

I'd go with a 5in3 sata enclosure like the Supermicro or AMS one. They work just fine.

Thanks, I've seen a bunch of them on newegg, but it's so hard to tell if they're crap or not...

But for the price, you might think about going to a norco 4u 4020 case that comes with 20 sata trays and backplane, and is cheaper than going the 5in3 rack route. You get lots of room for expansion, but it's a rackmount system.

That is interesting, but "cheaper" is complicated. I don't really plan to go down the "massive" road, I'm looking at more like an 8 drive array. So I don't need 20 bays. So while for 20 bays it's definitely cheaper, 20 bays is actually more expensive because it's more than I need...

Hm ~$1000 for the base hardware to do it myself, but I get a new server out of the deal...

[jgourd]

Originally Posted by stanger89

Well, 1 and 6 really aren't reasons to choose a Drobo over a NAS, as a decent NAS (home built or otherwise) does the same thing.

And at about $700 for the combo, it's not really any cheaper either.

I never said it was cheap. It is however very reliable and doesn't use a lot of electricity. I have a RAID5 system that I home built two years ago and the cost of running it 24/7/365 became prohibitive.

Anyway, I am not discounting anything anyone else is using, I just wanted to share my thoughts.

[chrisc16]

Originally Posted by stanger89

So, main question is does anyone know any good SAS or SATA hotswap drive enclosures? Probably 4-in-3 type.

I probably could have found something cheaper, but I went with the Addonics 5SA disk array. The construction and quality are great.

Or are you looking for an external enclosure?

-Chris

[stanger89]

Nope, internal.

[mikesm]

Originally Posted by stanger89

Thanks, I've seen a bunch of them on newegg, but it's so hard to tell if they're crap or not...



That is interesting, but "cheaper" is complicated. I don't really plan to go down the "massive" road, I'm looking at more like an 8 drive array. So I don't need 20 bays. So while for 20 bays it's definitely cheaper, 20 bays is actually more expensive because it's more than I need...

Hm ~$1000 for the base hardware to do it myself, but I get a new server out of the deal...

The AMS, addonics, and a couple others all use the same basic OEM 5in3 frame, which works just fine. I have 2 of them. The supermicro CSE-M35T-1B works great too (I have one of them too), and has better ventilation. But all of them just a fine job is this role.

If you aren't rackmounting stuff, the 5in3's will be fine. But they are a little hard to squeeze into a 4U case for some reason (I had a friend who had issues mounting thing in a 4U rackmount case). So if you are going rackmount, I would recommend a case that has the sleds builtin, even if it's less than 20. If you are using something like a stacker (which I use), the 5in3's work great in that. No issues with mounting.

[Peter_h]

This is a little bit off topic but I figured the most relevant answers would come from the people in this post.

What do you guys do about commercial detection?

I've been following the latest com skip thread on h.264 formats, link here

http://forums.sagetv.com/forums/show...t=35381&page=5

and the guys are saying that they see 40% CPU usage across all four cores of a quad core system for one file.

In this instance would you have the sage server handle the commercial detection or the NAS?

I guess i am asking if you record the file to your sage server and have the sage server do the commercial detection and move it to the NAs when it is done.

Or do you record straight to the NAS and have the sage server still do the commercial detection...

OR do you record straight to the NAS and have the NAS do the commercial detection.

-Peter

[mikesm]

Originally Posted by Peter_h

This is a little bit off topic but I figured the most relevant answers would come from the people in this post.

What do you guys do about commercial detection?

I've been following the latest com skip thread on h.264 formats, link here

http://forums.sagetv.com/forums/show...t=35381&page=5

and the guys are saying that they see 40% CPU usage across all four cores of a quad core system for one file.

In this instance would you have the sage server handle the commercial detection or the NAS?

I guess i am asking if you record the file to your sage server and have the sage server do the commercial detection and move it to the NAs when it is done.

Or do you record straight to the NAS and have the sage server still do the commercial detection...

OR do you record straight to the NAS and have the NAS do the commercial detection.

-Peter

I use showanalyzer - works great. I record directly to NAS, and another (windows) server runs showanalyzer across the network reading from the NAS. I just got VMWARE running up on the NAS itself (it is linux based), and so will move commercial processing to the VM there, which should reduce traffic and improve I/O. The NAS has a quad core CPU on it and plenty of ram, so no issue with it running there.

[stanger89]

Originally Posted by Peter_h

In this instance would you have the sage server handle the commercial detection or the NAS?

In general, I'd say you wouldn't want to try it on a "NAS", since "NAS" implies that it's a comparatively low power machine. Personally I run SA on my desktop, and if you've got another PC that's on anyway, that would be my recommendation. Next to that I'd run it on the Sage server.

I guess i am asking if you record the file to your sage server and have the sage server do the commercial detection and move it to the NAs when it is done.

There's no reason to move the file, just scan it in place.