It seems there's never enough storage. To address growing storage needs within our faculty, we looked at various storage solutions. FreeNAS, an open-source solution, stood out because our unit works with a razor thin budget. We ordered our first FreeNAS system last year.
This year we looked at building another FreeNAS system for our satellite campus to use as an off site backup of the primary system. FreeNAS uses the ZFS file system and can easily replicate one system to another. The off site system provides a good level of protection and redundancy. Should the main server fail, the secondary server can take over.
The first step was to spec and build the hardware. We looked at brand name systems, but the prices all quickly scaled beyond our budget once we added the RAM and storage space we desired. FreeNAS recommends 1GB of RAM for each terabyte of storage, and we wanted lots of storage. The configurations we looked at all ended up being over $8,000, with some of the high-storage options going over $10,000.
We decided to look at building our own system.
After a little shopping, I came across the Supermicro 6027R-72RF 2U barebones server. It has 8 SAS/SATA III hot-swap drive bays, SCSI SAS controller with SAS expander backplane, a dual socket Xeon motherboard, redundant power supply, CPU heatsinks, fans, and integrated video. Pretty much everything you need other than CPU, RAM, and hard drives. This nearly complete system was just under $1,600!
We added in a 6-core/12-thread Xeon processor, 32GB of ECC RAM, and 8 4TB Western Digital RE (RAID Edition) hard drives, and the grand total was just under $5,800 CDN (before tax). It would have almost been possible to order two of these systems for the same price as a similarly spec'ed brand name server.
Putting all of the parts together was a breeze. The whole system was assembled and booting up FreeNAS in just a few hours.
Of course, things weren't going to be perfect. :)
The support from FreeNAS of the integrated SCSI chip was pretty much non-existent. The recommendation from the FreeNAS forums was to purchase another SCSI card. Despite the information in the forums, I had no trouble running the drives in the proper JBOD mode. The biggest challenge seemed to be FreeNAS' lack of integrated support for monitoring the health of the hard drives. From what I could tell, this support is in the mfip.ko module in FreeBSD (the OS FreeNAS is based on), but that module isn't in FreeNAS.
FreeNAS does include the MegaCli utility which can be used to read a lot of information from the drives, including the SMART values. FreeNAS also has an alert system based on Python scripts. It was relatively easy to write a custom script to monitor the health of the hard drives and use FreeNAS' integrated alert system.
The system has been up and running solidly for a few months now. The next topic will describe how we're letting our users take advantage of the extra space.
