FreeNAS and ZFS; I'm officially impressed.

[R.G.]

I've been chasing a ZFS backup and utility server for a while now, and finally got one running stably under FreeBSD and FreeNAS.

I was doing my first massive file movements into the new Big Bucket when one of two mirrored disk drives started clicking, and a while later died stone-cold-dead. I was sitting here listening while the clicking got more frequent, then suddenly stopped.

It didn't even stop the file transfer that was going on. It kept on moving files, noted a dead disk drive in an email to my account, and started scrubbing itself to look for data errors **while** it kept on doing file transfers. No data errors.

I'm impressed.

[bob p]

I built a Freenas server back in October-November as my first experiment with the distribution. I built a doubly-redundant Zpool, so that two drives could fail without having the pool lose it's "healty" status. Much to my chagrin, one of the brand new WD 2 GB Green drives was dead coming out of the anti-static bag, and this fact wasn't recognized when the pool was built. So I started off with a non-healthy pool. I had to RMA the drive, run on a non-healthy pool for a couple weeks, and then reinstall the new drive. (Good thing that I built a doubly fault-tolerant zpool!)

I'm happy to say that the Zpool worked just fine without a hiccup the entire time that the pool was down from 4 drives to three. I pounded on the zpool for a solid week, loading the server with several TB of files over the network. I'm not kidding -- it took a solid week for Rsync to transfer the files to the ZFS box. The process ran for a solid week... without a single hiccup other than that one bad drive.

The problem came along when I had to replace the defective volume. Be forewarned everyone, that the process of taking a drive in a z-pool off-line for replacement is a very specific, detailed process, and you have to perform every step in the proper order or the results will be catastrophic. I learned the hard way.

IMO the FreeNAS interface is very poor when it comes to exchanging/replacing drives. I mis-understood where some of the menu items were hidden, and made the mistake of installing the new drive as striped against the zpool instead of adding it to the zpool as a replacement, all because of one mistaken mouse click. Unfortunately, that mistake could not be undone, and my only option was to reformat the entire Zpool, losing all of my data. I spent another week uploading data to the server.

Spending that extra week uploading data was really annoying. I tried just pulling a data drive and moving it into the ZFS box to copy the data over the PCI bus, but wouldn't you know it -- FreeBSD won't read EXT4 drives. So I was back to spending another week loading up the file server via ethernet.

All things considered, I think that ZFS is robust, but I think that FreeNAS is very poorly documented and the browser administrative interface leaves a lot to be desired. when I ruined my zpool in trying to swap/replace the defective drive, I was really disappointed. Googling a bit, I learned that many people have made the exact same mistake that I made, and suffered the exact same fate -- data corruption -- all because of the lousy documentation and administrative GUI. This would never have happened if the documentation were better in FreeNAS.

All things considered, I'm pretty happy with FreeNAS now that it's up and running. I particularly like the fact that I can set the Zpool to go through a scrub on sunday nights while I'm sleeping, and that the system is ready to start a new week on Monday morning with freshly scrubbed data. I am very disappointed though, with the poor state of documentation for FreeNAS. While ZFS is extremely robust and fault-tolerant, the poor state of the documentation means that your greatest risk in running FreeNAS comes in administering the system. FreeNAS and ZFS are very complicated systems (perhaps unnecessarily so) and the greatest risk of data loss comes from administrative error rather than hardware failure. I'm a pretty experienced linux network sysadmin, having done it for over 10 years. I really wish that I didn't have to resort to running BSD to get ZFS.