Recommended NAS?

[mikesm]

PS One more thing - in another thread in the forum, folks have talked about getting showanalyzer and comskip running under linux using wine. This has a big advantage running on the NAS, since the actual files are on the system, and it removes a lot of network traffic and possibly computational load from the Sage server.

If you want to go this route, then I would highly recommend an Intel Motherboard and fast CPU, as the commercial processing is pretty intensive. If folks every can get h.264 transcoding for archiving off loaded onto Linux, then that would really be a winner for the same reason, and would probably push me to a quad core CPU for the NAS.

For NAS stuff alone though, you don't need a fast CPU at all - just modern I/O.

thx
mike

[BitBass]

So what you're saying is, I'm overthinking it?

Ok, this board seems to have it all, but only two PCIE slots. Runs about $75:
http://www.asus.com/products.aspx?l1...72&modelmenu=2

Or better yet, this one looks good too although I'm not certain the GE isn't PCI. More PCIE slots though. Runs about $100:
http://www.newegg.com/Product/Produc...82E16813121099

That might be the one. Either way, I'll give it a rest.

[mikesm]

Originally Posted by BitBass

So what you're saying is, I'm overthinking it?

Ok, this board seems to have it all, but only two PCIE slots. Runs about $75:
http://www.asus.com/products.aspx?l1...72&modelmenu=2

Or better yet, this one looks good too although I'm not certain the GE isn't PCI. More PCIE slots though. Runs about $100:
http://www.newegg.com/Product/Produc...82E16813121099

That might be the one. Either way, I'll give it a rest.

That intel board looks nice, and I'm sure it's a non-PCI GbE controller. Just no overclocking capability... The ASUS P5K-V version with the G33 should have overclocking capabilities too, and a few more SATA ports. All very nice actually.

Thanks,
Mike

[KarylFStein]

I'm using the Intel DG965WH mainly because it was one of the couple boards I could find at the time with 6 SATA ports and onboard graphics. (I'm using the ICH8R for RAID and didn't want port multipliers.) It's been running very well with an inexpensive E4300 CPU.

[BitBass]

Karyl, I think I like that board a lot! Thanks!

[mikesm]

An interesting except from the linux-raid mailing list (not that I would recommend running a production system in raid5 with more than about 8 disks):

On Tue, 18 Dec 2007, Thiemo Nagel wrote:

> Dear Norman,
>
>>> So... we're curious how Linux will handle such a beast. Has anyone run MD
>>> software RAID over so many disks? Then piled LVM/ext3 on top of that? Any
>>> suggestions?
>>>
>>> Are we crazy to think this is even possible?
>
> I'm running 22x 500GB disks attached to RocketRaid2340 and NFORCE-MCP55
> onboard controllers on an Athlon DC 5000+ with 1GB RAM:
>
> 9746150400 blocks super 1.2 level 6, 256k chunk, algorithm 2 [22/22]
>
> Performance of the raw device is fair:
> # dd if=/dev/md2 of=/dev/zero bs=128k count=64k
> 65536+0 records in
> 65536+0 records out
> 8589934592 bytes (8.6 GB) copied, 15.6071 seconds, 550 MB/s
>
> Somewhat less through ext3 (created with -E stride=64):
> # dd if=largetestfile of=/dev/zero bs=128k count=64k
> 65536+0 records in
> 65536+0 records out
> 8589934592 bytes (8.6 GB) copied, 26.4103 seconds, 325 MB/s
>
> There were no problems up to now. (mkfs.ext3 wants -F to create a filesystem
> larger than 8TB. The hard maximum is 16TB, so you will need to create
> partitions, if your drives are larger than 350GB...)
>
> Kind regards,
>
> Thiemo Nagel
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>

Quite slow?

10 disks (raptors) raid 5 on regular sata controllers:

# dd if=/dev/md3 of=/dev/zero bs=128k count=64k
65536+0 records in
65536+0 records out
8589934592 bytes (8.6 GB) copied, 10.718 seconds, 801 MB/s

# dd if=bigfile of=/dev/zero bs=128k count=64k
27773+1 records in
27773+1 records out
3640379392 bytes (3.6 GB) copied, 6.58454 seconds, 553 MB/s

[KJake]

Ack! What am I doing wrong? Is it the port multipliers? I have nothing like those speeds!

# dd if=/dev/md1 of=/dev/zero bs=128k count=64k
65536+0 records in
65536+0 records out
8589934592 bytes (8.6 GB) copied, 67.8733 seconds, 127 MB/s

# dd if=/dev/md0 of=/dev/zero bs=128k count=64k
65536+0 records in
65536+0 records out
8589934592 bytes (8.6 GB) copied, 61.7992 seconds, 139 MB/s

[mikesm]

Originally Posted by KJake

Ack! What am I doing wrong? Is it the port multipliers? I have nothing like those speeds!

# dd if=/dev/md1 of=/dev/zero bs=128k count=64k
65536+0 records in
65536+0 records out
8589934592 bytes (8.6 GB) copied, 67.8733 seconds, 127 MB/s

# dd if=/dev/md0 of=/dev/zero bs=128k count=64k
65536+0 records in
65536+0 records out
8589934592 bytes (8.6 GB) copied, 61.7992 seconds, 139 MB/s

Well, 22 disks in RAID5 or the use of raptors helps. What kinds of disks you have? How many, chunk size, etc...

PS You can still saturate a GbE with that...

thx
mike

[KJake]

I think it is technically .7 or .8 Gb, but close. I guess those raptors must be the 15k ones?

#  cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md1 : active raid5 dm-9[3] dm-8[2] dm-7[1] dm-6[0]
      2197715712 blocks level 5, 256k chunk, algorithm 2 [4/4] [UUUU]

md0 : active raid5 dm-5[5] dm-4[4] dm-3[3] dm-2[2] dm-1[1] dm-0[0]
      2441918720 blocks level 5, 256k chunk, algorithm 2 [6/6] [UUUUUU]

The md0 drives are 500GB ST3500630AS drives.
The md1 drives are 750GB ST3750640AS drives.

They're 7200rpm drives - I suppose that doesn't help things? And they're connected to port multipliers. Just wondering then if the speed I quoted there is a good one for my setup?

[mikesm]

Originally Posted by KJake

I think it is technically .7 or .8 Gb, but close. I guess those raptors must be the 15k ones?

#  cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md1 : active raid5 dm-9[3] dm-8[2] dm-7[1] dm-6[0]
      2197715712 blocks level 5, 256k chunk, algorithm 2 [4/4] [UUUU]

md0 : active raid5 dm-5[5] dm-4[4] dm-3[3] dm-2[2] dm-1[1] dm-0[0]
      2441918720 blocks level 5, 256k chunk, algorithm 2 [6/6] [UUUUUU]

The md0 drives are 500GB ST3500630AS drives.
The md1 drives are 750GB ST3750640AS drives.

They're 7200rpm drives - I suppose that doesn't help things? And they're connected to port multipliers. Just wondering then if the speed I quoted there is a good one for my setup?

You don't have any drives with that AAK version firmware problem in there do you? See here: http://www.fluffles.net/articles/seagate-AAK-firmware

Thx
Mike

[KJake]

Originally Posted by mikesm

You don't have any drives with that AAK version firmware problem in there do you? See here: http://www.fluffles.net/articles/seagate-AAK-firmware

Thx
Mike

:cry:

All the 500gb drives are AAK:
"Model=ST3500630AS , FwRev=3.AAK"

Whereas all the 750GB drives are AAC:
"Model=ST3750640AS , FwRev=3.AAC"

Drat, so I need to try and get Seagate to replace these? Thanks for the info, I would have never thought of this.

[mikesm]

Originally Posted by KJake

:cry:

All the 500gb drives are AAK:
"Model=ST3500630AS , FwRev=3.AAK"

Whereas all the 750GB drives are AAC:
"Model=ST3750640AS , FwRev=3.AAC"

Drat, so I need to try and get Seagate to replace these? Thanks for the info, I would have never thought of this.

Ah well, that explains it. Yeah, there is a big outcry in the performance community about this. Some even accused them of dumping AAK based units at a discount through fry's etc...

You should contact them and see if they'll do a swap. Though in real world performance in your application it's probably not that big a deal. Still, if I had all AAK, I'd probably work a swap with them.

PS Did you look at that article I linked? Look at the delta in sequential read/write performance. That's the heart of what we do in this business. :-) A big deal for media serving I would say.



Thx
mike

[pez]

So, after reading this thread, i'm interested in making a Linux RAID 5 NAS box. I have moderate Linux experience but have good friends that are sysadmins for linux/unix and are willing to help.

I have a horribly slow SW Raid5 (5x250GB) array for my SD Sage server. It is not capable of recording 2 HD streams while viewing 2 HD streams. I need more performance and I like the security of RAID as I often record an entire half season of a show before I watch it.

I have some WD10EASC 1TB GP drives that i've been using as backup disks to other systems. If I reuse these in a raid array, the array can hold the backs.

These drives do not have the TLER (a way to gouge enterprises) that the RE or enterprise drives have and can therefore experience a long access delay when correcting a disk error. So they cannot be used with HW Raid controllers.

Can the Linux Raid SW deal with this or are drives so good now-a-days that I will likely never experience a problem?

If not I'd likely sell them and by Seagate Barracuda 7200.11 ST31000340AS 1TB.

thanks,
Pez

[KJake]

Originally Posted by mikesm

Ah well, that explains it. Yeah, there is a big outcry in the performance community about this. Some even accused them of dumping AAK based units at a discount through fry's etc...

You should contact them and see if they'll do a swap. Though in real world performance in your application it's probably not that big a deal. Still, if I had all AAK, I'd probably work a swap with them.

Thx
mike

Good thing I've got those Hitachi drives standing by. I'll pop those in within the next week, have EVMS migrate the 500gb drives out, and then work on getting Seagate to swap them for a different firmware.

FYI - I got the 500gb from NewEgg, scary.

[mikesm]

Originally Posted by pez

So, after reading this thread, i'm interested in making a Linux RAID 5 NAS box. I have moderate Linux experience but have good friends that are sysadmins for linux/unix and are willing to help.

I have a horribly slow SW Raid5 (5x250GB) array for my SD Sage server. It is not capable of recording 2 HD streams while viewing 2 HD streams. I need more performance and I like the security of RAID as I often record an entire half season of a show before I watch it.

I have some WD10EASC 1TB GP drives that i've been using as backup disks to other systems. If I reuse these in a raid array, the array can hold the backs.

These drives do not have the TLER (a way to gouge enterprises) that the RE or enterprise drives have and can therefore experience a long access delay when correcting a disk error. So they cannot be used with HW Raid controllers.

Can the Linux Raid SW deal with this or are drives so good now-a-days that I will likely never experience a problem?

If not I'd likely sell them and by Seagate Barracuda 7200.11 ST31000340AS 1TB.

thanks,
Pez

Pez, I am not familiar with the specific issue that you are talking about, though the linux raid code is very robust. If it did stop responding for some time, it may cause the drive to fault and be taken offline. I suspect it'll be fine though. Best to query folks on the linux-raid mailing list and get their take on it.

I take it the existing system is a windows based SW RAID system? If so, it'll be hard not to get better performance. :-)

PS I always grab hold of my wallet when I hear people making "green" tech equipment. It's usually a sign they are peddling substandard equipment and looking for a gimmick that will make people want to buy it.

thx
mike

[mikesm]

Originally Posted by KJake

Good thing I've got those Hitachi drives standing by. I'll pop those in within the next week, have EVMS migrate the 500gb drives out, and then work on getting Seagate to swap them for a different firmware.

FYI - I got the 500gb from NewEgg, scary.

I thought I remembered you getting a set of those. :-) In that case, heck yeah. Call Seagate and demand an exchange - set it up, and then install the 1 TB drives and do the migration. I would not have recommended you take the 500's offline like that for a swap, but since you have the 1Tb drives anyway, why not?

BTW, I know seagate has an option in the exchange program where if you pay something extra they will send you the new drives first, and then you ship them back the old ones. That means a minimum of downtime. A good idea for other folks that may be having these problems but didn't get in on the BB 1 TB deal. :-)

thx
mike

[pez]

Thanks for the response Mike.

Originally Posted by mikesm

Pez, I am not familiar with the specific issue that you are talking about, ...

TLER in the RE and enterprise drives is a FW feature that stops an attempt at recovering a bad sector after 7 seconds so the HW Raid controller does no drop the disk. I would guess that if I'm writing 2 20Mbit/s HD streams, and 2 6.7Mbit/s SD streams, the SW would need to buffer 6.676MBytes/s. At 7 seconds that almost 50MB. How much buffering does the Linux RAID SW do? The GP can take up to 2 minutes before it gives up.

Originally Posted by mikesm

I take it the existing system is a windows based SW RAID system? If so, it'll be hard not to get better performance. :-)

Yes it is, when i build it, I had just been through a rash of drives dying on me. So I decide to suffer with performance since it didn't affect SD recordings. Now I have an HDHR and it no worky.

Originally Posted by mikesm

PS I always grab hold of my wallet when I hear people making "green" tech equipment. It's usually a sign they are peddling substandard equipment and looking for a gimmick that will make people want to buy it.

I agree, but i bought these because they run cool (28C idle and 38C load) and are cheaper than other 1TB. As is live in the desert, keeping my Den (server room) cool is an issue.

Thanks,
Pez

[mikesm]

Originally Posted by pez

Thanks for the response Mike.



TLER in the RE and enterprise drives is a FW feature that stops an attempt at recovering a bad sector after 7 seconds so the HW Raid controller does no drop the disk. I would guess that if I'm writing 2 20Mbit/s HD streams, and 2 6.7Mbit/s SD streams, the SW would need to buffer 6.676MBytes/s. At 7 seconds that almost 50MB. How much buffering does the Linux RAID SW do? The GP can take up to 2 minutes before it gives up.



Yes it is, when i build it, I had just been through a rash of drives dying on me. So I decide to suffer with performance since it didn't affect SD recordings. Now I have an HDHR and it no worky.



I agree, but i bought these because they run cool (28C idle and 38C load) and are cheaper than other 1TB. As is live in the desert, keeping my Den (server room) cool is an issue.

Thanks,
Pez

Again, on the specifics of the SATA kernel driver operating in RAID mode doing bad block recovery, you should post a note to the linux raid list. The folks there do have intimate knowledge of the kernel, so they would better be able to answer that. Of course, if the disk hangs during a read, the buffering isn't going to help as the filesystem will go into an i/o wait state before being able to fulfill the request, readahead buffering notwithstanding.

As far as generic filesystem buffering, using XFS and RAID5 together with the tweaked settings I posted awhile ago, I use about 700 MB of RAM for filesystem buffering. That certainly helps optimize disk activity for performance.

I think the # of disks makes a lot bigger difference than the GP "tuning" of the disks in terms of power savings. Going with a high efficiency CPU and high efficiency PSU and a smaller number of high density disks are about the best you can do for a NAS. Doing staggered disk spinup will allow you to get by with a smaller PSU which means that it can operate in the higher efficiency range during typical operation will also help. I have 15 disks spinning off a system with a single high efficiency 480W PSU.

But at 24x7 operation, it's a good idea to be thinking about energy use.

[pez]

So does the Linux Raid or disk management SW allow for controlling staggered spinup or is that something your (5in3) rack does?

Thanks,
Pez

[mikesm]

Originally Posted by pez

So does the Linux Raid or disk management SW allow for controlling staggered spinup or is that something your (5in3) rack does?

Thanks,
Pez

Depends on hardware, but my PMP connected drives seem to do that. The rack is passive from an SATA perspective.

thx
mike