Hard drive backups

[Steve A.]

I learned a lot about backups when a friend's RAID 0 drive set crashed last spring. (RAID 0 is striping the data between two hard drives to increase the speed- it is very important to back up on a daily basis since if either one of the two drives was to fail your data would be trashed and very expensive to recover.)

I asked the recovery tech about all of the different backup methods. I thought that RAID 1 would be ideal: "RAID 1 mirrors the contents of the disks, making a form of 1:1 ratio realtime backup. The contents of each disk in the array are identical to that of every other disk in the array."

RAID 1 is good but you still do need to keep backups; while it will protect your data in case one of the two drives develops a critical hardware problem, it will not help if your OS screws up and starts sending out crap bytes to both of the drives. Or if you accidentally delete or overwrite an important data file.

So whether you use SATA or RAID 1, you really want to keep regular backups on a separate physical drive (external or internal). Like a full backup once a week or once a month, and incremental backups every night.

I also like to keep my important data files backed up to a DVD-R. Well, its better to be safe than sorry.


I wish I could say that I have been following all of my advice regarding backup but I have not. Last summer I had an external drive fail- this was a regular SATA drive in a SanDigital enclosure which is like a small computer case. I have been able to save some of the files from but there are countless CRC errors which cause a file copy to abort. So while the bad news is that I have lost a lot of files, the good news is that most of them were live and studio recording of Bob Dylan. (It has been suggested that I try to use something like Ghost to save an image of the drive since it will supposedly ignore the CRC errors and just copy everything that it can.)

I think I know why that drive failed- I forgot to keep the dust off the air intake screen and it overheated a lot (I usually keep these drives on 24/7). When it would overheat it would shut off and anything that Windows was writing to it larger than the cache would be trashed.

Oh well, live and learn. At least it was just my 100GB collection of Bob Dylan recordings I lost and not I really like...

Steve Ahola

[dai h.]

fwiw, if you need to take out a RAID 1 disk and use it to access the data using an external drive case or the like, you have to re-write the file table (or something like that) which can be done with a free piece of software called TestDisk:

Marstran.com • View topic - useful FREE computer tools (NoScript, TestDisk)

maybe two external USB drives (one data/one backup) is easiest (affordable, easy use, easy to re-connect to most any newer PC)? IIRC I have three copies of the most important stuff since I read somewhere that was the best practice. Was thinking about getting a 2TB. Best prices seem to be around a bit over (around) USD130 for the Hitachi/IBM one (here anyway).

[Steve A.]

fwiw, if you need to take out a RAID 1 disk and use it to access the data using an external drive case or the like, you have to re-write the file table (or something like that) which can be done with a free piece of software called TestDisk:

Marstran.com • View topic - useful FREE computer tools (NoScript, TestDisk)

maybe two external USB drives (one data/one backup) is easiest (affordable, easy use, easy to re-connect to most any newer PC)? IIRC I have three copies of the most important stuff since I read somewhere that was the best practice. Was thinking about getting a 2TB. Best prices seem to be around a bit over (around) USD130 for the Hitachi/IBM one (here anyway).

I was just reading somewhere that they actually recommend using an external drive these days instead of an internal drive for your audio files- I would think that the internal bus would be faster than USB but I guess that the internal bus is more likely to be interrupted- something like that.

For external drives I had nothing but problems with the self contained units like the My Book. Without exception after a year or two they would crash.

At least until the drive failure last summer (mainly my own fault for not cleaning off the dust) I have had great luck with the SanDigital external hard drive enclosures. I ran into problems with a few of the early cheap enclosures- like having the computer recognize them; perhaps the new ones are more reliable because the SanDigital enclosures are not cheap (the 4 drive models are selling for around $150- and I never see them go on sale anymore!)

Steve

P.S. Thanks for the tip about TestDisk- there are not a lot of truly free file recovery programs out there that are worth a crap. They are always a sales tool to get you to buy the premium version.

[MistaT]

I recently read that one weak link with raid hard drives is the controller. Usually bothe drive use the same one. So if your controller card craps out, yes your data is probably ok, but you are dead in the water until you get a new controller card.

Personally, I like to get my hands on an old PC and put a big hard drive in it and load linux on it (my personal favorite "flavor" of linux is Ubuntu..). Usually you can scrounge an old PC from someone else that is throwing it away. Then go buy a 500 G hard drive to throw into it for the data drive.

Set is up as a server, then have a BAT file do an xcopy perodically that is called from the task scheduler, then it is automatic.

If you have a MAC, (my son does..) then use the Rsync command. Additionally if you DO have a mac, the rsync command can be run from the backup system / server automatically. Set it up once and forget it.

Just check your backup files say once a month so see that it is actually working.

Yeah its a lot of work, you only have to do it once, and then forget about it. Been doing that for years, and it just works.

[Steve A.]

I recently read that one weak link with raid hard drives is the controller. Usually bothe drive use the same one. So if your controller card craps out, yes your data is probably ok, but you are dead in the water until you get a new controller card...

What I was told by the recovery tech when my friend's ~7 year old computer with the RAID 0 drives that went South was that you can't just replace a bad controller card because the new one will not be able to read the old drives. (I had thought that it was just like IDE and SATA drives where you can replace the controller with no problems.) Of course he may have just been feeding me a line.

I have good reason to suspect his integrity. His company's policy is that they can recover only 50% of the data on the drive you only pay 50% of the quoted price. On the form you fill out you are told to list what types of files were on the drive(s). It just so happened that they were able to recover the full 80GB so the full bill had to be paid.

However once my friend started looking through the files she found a whole bunch which were not hers. Like a bunch of Christian DOC files (she is Buddhist). I had suggested that some programs had installed sample files on her hard drive and that was where they might have come from. It occurred to me much later that these guys were a bunch of crooks, padding the recovered drive with crap files. Another issue to ponder- the bad drives had something like 82GB worth of files. Since they recover *only* data files and not the operating system files and not all of the programs installed under Program Files, how were they able to recover 82GB of data...?

Steve Ahola

P.S. I like your idea to set up a junk computer with Linux to act as a server and have it back up your data files automatically. I believe that Windows used to lock some of the data files that were opened- how does your future server handle that or is it no longer a problem? I like running HD utilities when Windows is still in its pre-boot state since you don't have to deal with all of the Windows crap.

[MistaT]

Yeah that data recovery tech sounds fishy to me too.. Sorry to hear that.

Yeah I think you are right about windoZe locking open files, but I think it is more application dependent. I have found some programs will open "exclusive" which will prevent others from reading the file, and other applications will open and share nicely.

Can you tell I'm rather fed up with Micro$oft products? I am goiing in a Linux direction, Ubuntu specifically, and it handles all of the file locking much better. Also Rsync is usually run to only backup files that have changed. So just run in once an hour and then you only loose an hours time even the file is locked.

[Steve A.]

Yeah that data recovery tech sounds fishy to me too.. Sorry to hear that.

Yeah I think you are right about windoZe locking open files, but I think it is more application dependent. I have found some programs will open "exclusive" which will prevent others from reading the file, and other applications will open and share nicely.

Can you tell I'm rather fed up with Micro$oft products? I am goiing in a Linux direction, Ubuntu specifically, and it handles all of the file locking much better. Also Rsync is usually run to only backup files that have changed. So just run in once an hour and then you only loose an hours time even the file is locked.

Linux is starting to sound *real* good to me. One problem is that most the programs I need to use (like Adobe Audition) do not support that platform (as far as I know).

It would be really cool if someone could write a program that would allow you to run Windows applications in Linux- without having to load the whole Windows colonel, er, kernal.

As far as locking files I guess it probably depends on the application. I think that most of the newer programs load the data file into memory (virtual or real) and then close the data file- very important to keep it from getting overwritten.

One test would be to open a data file in a specific application and then see if you can copy the file using Windows Explorer. And then just for the heck of it see if you can move the file (extra credit points here if you can). I have noticed that sometimes when I move files if one of them is open the file will be moved but the folder will remain. I suspect that is because when the program that has opened the file is ready to save your edited file the folder will still be there.

Question: does Linux do "preemptive" multi-tasking? That was always my complaint with doing digital audio recording on a Windows computer. If for any reason Windows decides that something is more important than the task you are running you get bumped, at least for a split second which can be enough to create a glitch. It would seem to me that now that we have dual and quad processors, one of them could handle the Windows interrupts and another one could run your program. No more glitches and hiccups! At one point I was getting so many pops and clicks in my recordings that I had to run them through the click repair software that I use for vinyl rips.

Steve Ahola

P.S. The more I think about it the more I like idea of having a Linux server but run Windows in at least one of the clients. Another question: I have about a dozen 1TB external drives (internal SATA in pro quality enclosures). So Linux does support USB- right? My question is what would the throughput be if I was writing my files on USB external drives plugged in the Linux server- going through an ethernet cable? (I think that they have 1000mbps wired routers these days so it might be worth it to upgrade to something like that.)

I think that your suggestion was to use the Linux machine just for the backups- I am wondering about having all (or most) of my external hard drives plugged into the Linux server.

Thanks for your posts- I learn a lot from these threads!

Steve Ahola

P.S. "Also Rsync is usually run to only backup files that have changed. So just run in once an hour and then you only loose an hours time even the file is locked." No problem- what bugs the sh*t out of me is how Windows will abort the operation of moving or copying multiple files if it runs into some problem like an open file. C'mon- go ahead and move or copy the rest of the files first and *then* ask me how to handle the files in question. Dumb, dumb, dumb...

[defaced]

It would be really cool if someone could write a program that would allow you to run Windows applications in Linux- without having to load the whole Windows colonel, er, kernal.

There is at least one: WineHQ - Run Windows applications on Linux, BSD, Solaris and Mac OS X

USB is fully supported in Linux.

You will have to check with the distribution (flavor) of Linux you're looking at for particulars about how they handle multitasking. I'm sure someone has tweaked the kernel to do what you want it to. There's alot to learn if you want to go to Linux. Some trivial, some not so.

[Steve Conner]

I ran a Linux server at home for years and backed my other computers (both at home and at work) up to it with rsync. I installed Cygwin on the Windows machines and wrote some shell scripts to do the backing up. It works just fine, and I used it for lots of other things, like running a development web server, a CVS repository for a programming project I was working on, and a kind of remote-controlled jukebox using mpd and PHPMP.

You can't back up the Windows system files with rsync though: the best way to do that is to use something like Norton or Acronis to make a disk image. I know that Acronis at least can send the disk image over your network to your Linux server, if it supports SMB or FTP.

But be aware of the old adage "if it's online, it's not a backup". A backup drive unplugged and stashed in a filing cabinet is safer than one spinning in a server.

Yes, like all modern OSs, Linux has preemptive multitasking. The only ones that didn't were Windows 3.1 and the old MacOS. But even with preemptive multitasking, you can still get those clicks and pops: in fact it sometimes makes them worse, when your audio app gets preempted by something else.

There are patches available to optimize the Linux kernel for audio, and you can get ready-made distros like Ubuntu Studio with the patches already in.

The main thing that Linux lacks is the "killer apps": you don't get any of the popular audio software for it. Audacity is the best audio editor the FOSS crowd have, and it can't touch the likes of Sound Forge. There are open-source DAW programs available too, but it's a while since I looked at them. The last one I tried was Rosegarden and it was pretty awful.

WINE certainly works, but I wouldn't be optimistic about the real-time audio performance.

[MistaT]

Steve, as far as backing up a windows PC on the linux box, honestly I haven't looked into it, but a linux box WILL mount to a windowZe PC. So I'm guessing that its possible to run rsync via an smb mount. That would be a good question to ask Mr. Google.

As far as running WindoZe apps inside linux, again, I haven't done it, but lately a lot of people have been using WMware. I have seen it run, and it seems to work. Now as far as latency fo audio, I don't know. I have also heard of wine, and I haven't used that one either, so I don't know which is better.

I work at a radio station, and all of our files are WAV's. When I started, they had a small peer to peer network all windowZe based, and it was crap. I got a hold of two old PC and threw in 500G drive in each and installed (ubuntu) linux on each. One server is the backup and the other is the main server on a standard 100MB neetwork. The workstations, as well as the PC that controls the on-air audio and satellite triggers are all windoZe PC's. The audio production PC uses adobe audition, with all audio files saved on the server. Additionally the on-air also pulls off the server, and there is no latencey.

Our server is just a generic 2 gHz PC with an ide drive. Nothing special or even really "server worthy" however when three of our 4 PC's are hitting the server hard, the server is only running 1% utilization.

Concerning an audio application, there is a free application called Audacity that is similar to adobe's autition. It was originally a Linux program, but it has been ported over to windows platform. My boss uses it on his windoZe PC from his house, then connects to the server and uploads his audio on snow days if he has to.


For me, the more I mess around with linux, the more I realize how little I know about it. Its pretty amazing. Like the Physics professor that got tired of purchasing time on a Cray computer for his research, and figured out /rewrote the kernel to run on a PS3, then clustered 8 of them together. Now he has the same processing power as a cray in his own lab for what his annual budget was to bu time on one. IIRC it was something like $1500. I'm sure Mr. Google can also find this guy too.

[R.G.]

Wow. Amazing convergent evolution. I've been working on a solution to random backup disks for some months. Here's what I've come up with, just as some ideas.

1. I need a server machine. No more wandering from machine to machine with a usb drive. Between my wife and myself, there are seven working and in-regular-use machines in the house. It's a wonder that I haven't had a big data loss. In the last year, I have had one motherboard just die (decoupling capacitors shorted) and one power supply simply quit. The disks had no failed files that were not on a usb drive. But it's a PITA to get this done, and one day I'll lose something big. So I need a back-end, back-up server.

2. I worry about bit rot. Simply putting data on CDs and then DVDs will eventually lose all your data, as the media has a several-years life. That sounds good until I think that I've been doing serious computing at home for nearly two decades. If you back up on DVDs, go get DVDisaster. It's free, effective, and creates error correction data for recovering optical disks from the inevitable bit rot on them.

2a. Hard drives have both sudden failures and bit rot. The sudden failure gets all the press, but drives can correctly write an incorrrect bit, or byte, or sector, or address if they are fed such from the non-EEC memory on a motherboard, or their own cache. Soft errors ... do ... happen in memory, and a hard disk will happily write them into hard errors.

3. I want to have reliability for more than one disk having a hard failure. There is a line of reasoning that says that withstanding one disk failure is not enough. This is because the number of bits on a disk is growing much faster than the rate at which you can fill a replacement disk through the I/O interface. That means that when one disk fails, the time to re-fill the replacement disk is a window during which any second disk failure loses you your data.

4. RAID often a trap. RAID is a trick to get good performance, not reliability. By spreading blocks to be read/written over N disks, you get N times faster read/write times, but 1/N reliability because losing any disk loses all your data. Hence RAID mirroring (the data is always doubly redundant) and RAID 5 and on up. This also brings in the write hole. Normal RAID setups write data and parity/checking data as separate blocks. If you have a bit error after the write of the last data block and before the write of the checking data, the drives thing all is well, but the data is hosed. Writing is non-atomic for normal RAIDs, and cannot be made so.

All this led me to zfs.

Zfs was designed at least partially for eliminating write holes and bit rot. ZFS does:

- copy on write; data to be written is written as a new copy, then the checking/repair blocks are written. When this is all over, a pointer is changed to point to the new copy. The old copy may be released as free, or may be retained - at NO performance expense - as a snapshot. By design, there is no write hole.
- It's big. You basically can't ever manufacture enough disks with any forseeable increase in density to get too big for it.
- it scrubs data in the background. This is what hooked me. You can tell the OS to run a data scrub in background, and it will spend its spare time not in an idle loop but running checksums against their data and correcting bit rot.
- not incidentally, the statistics on scrubs can tell you when you're going to have trouble in much the same way that SMART can tell you that your disk is getting near the edge
- it's free. zfs was designed for Solaris and is now part of the free Solaris downloads, as well as Open Solaris, where you also get the source code.

ZFS is available as a NAS setup for free, as well as a layer on top of BSD, perhaps others by now.

I just took delivery of my server hardware: triple core Athlon 2, 4G memory, motheboard, six 750Gb disk drives and a 4U rack mount case. So far it's about $600 outlay.

... now all I need is time to breath life into it...

[Steve Conner]

I don't see any tube sockets on there?

[Steve A.]

... All this led me to zfs.

Zfs was designed at least partially for eliminating write holes and bit rot. ZFS does:

- copy on write; data to be written is written as a new copy, then the checking/repair blocks are written. When this is all over, a pointer is changed to point to the new copy. The old copy may be released as free, or may be retained - at NO performance expense - as a snapshot. By design, there is no write hole.
- It's big. You basically can't ever manufacture enough disks with any forseeable increase in density to get too big for it.
- it scrubs data in the background. This is what hooked me. You can tell the OS to run a data scrub in background, and it will spend its spare time not in an idle loop but running checksums against their data and correcting bit rot.
- not incidentally, the statistics on scrubs can tell you when you're going to have trouble in much the same way that SMART can tell you that your disk is getting near the edge
- it's free. zfs was designed for Solaris and is now part of the free Solaris downloads, as well as Open Solaris, where you also get the source code.

ZFS is available as a NAS setup for free, as well as a layer on top of BSD, perhaps others by now.

I just took delivery of my server hardware: triple core Athlon 2, 4G memory, motheboard, six 750Gb disk drives and a 4U rack mount case. So far it's about $600 outlay.

... now all I need is time to breath life into it...

R.G. As soon as you said BSD I said "over my head, over my head!"

I finally learned what BJ stands for the other day and now you are throwing another anagram at me! ["What's that? Oh- acronym! Well, they both start with A and end with M...]

I get it now- ZFS will work on a Unix platform but not on Linux (because of legal issues). So what is Unix up to these days? I hear about everyone running Linux at home- never Unix.

So you can have a Unix machine as the file server on your Windows network? That sounds great.

Right now I have Norton AV set to do scans when my computer is idle. So far I think it has found one new threat; everything else are cracked programs and patches, and Mr. Norton has pretty much wiped out my collection going back over 10 years! I tried to turn off the idle scans but it is too late- Norton has a mind of its own and it is in charge of my computer now.

Thanks for all of the suggestions!

Steve Ahola

P.S. Note to self: be sure to copy all of the cracked programs you download to a DVD disc. Let's see what Norton can do with a read-only data disc. In your face, Peter!

[R.G.]

I get it now- ZFS will work on a Unix platform but not on Linux (because of legal issues).

So you can have a Unix machine as the file server on your Windows network? That sounds great.

Actually, the simplest thing is to download Open Solaris and use it as is, rather than unix in one of its various forms. There's some adapting to do with grafting zfs (and, in general, anything) into Unix or it's descendents, including Linux. There's a CD download to just set up OS and zfs. The only reason you have to consider this at all is that there are some arcane commands to be used, but they're no worse than linux.

See
ZFS NAS Wiki: Welcome to your new Wikidot site
and
EON ZFS Storage (NAS) (EON ZFS Storage)

One advantage of a NAS is that backup can be done over a network (i.e., as little as a network switch/router between two computers) and intermittently if you like, turning the NAS on when you want to do a backup if you like to keep the power-on-hours down on your backup storage. You have to balance power-on-cycles against power-on-hours if you're into this kind of thing.

Right now I have Norton AV ...

Say no more. Go get Avast (free) and zone alarm free firewall.

Norton is regarded by most people who think about antivirus things as very much like Dune's Baron Von Harkonnen - bloated, mentally and physically diseased, probably smelly if you get too close, and at its center, pure evil, any good it does being an accidental byproduct of whatever evil it was trying to do. Did I mention that I don't like Norton AV?

be sure to copy all of the [anything you want to keep] to a DVD disc.

Also notice that DVD-W, DVD+W, DVD-RAM, DVD-RW and so on have storage life that's indeterminate, a few years is all you could reasonably count on. Having found out the hard way about bit-rot on CD+/-W and CDRW, I turned to DVDiaster. This lets you recover the data on disks that have gone not too far south. So once a year or so, you'd run a scan on each of your backup DVDs with DVDisaster, and it would tell you if it could not read a sector. If that happened, you'd pull out the checking data, reconstitute the bad data from the original plus checking data and write it onto a new media. The way the computing world is going, about every five-ten years, a new hard media comes out and you can condense the backups about 5-10 to one. The next step is already over the horizon. Blue-ray DVDs hold about 25GB each, instead of the 5GB of the original DVD.

So archiving a 1TB (!) drive is 40 BDs. There I go with those acronyms again. DVDisaster already supports BD.

I got into this because I finally ripped my entire music collection to both FLAC (lossless, for archiving) and MP3 for (listening, until something better happens) and was faced with the question - OK, now what do you do if you have a disk crash? I have something like 400GB of music in FLAC form so I can recover the original bits. Took about six months of "spare" time to get that done. No way I'm losing the bits now.

Then there's the years of digital photos. Even bigger.

[Steve A.]

... Say no more. Go get Avast (free) and zone alarm free firewall.

Norton is regarded by most people who think about antivirus things as very much like Dune's Baron Von Harkonnen - bloated, mentally and physically diseased, probably smelly if you get too close, and at its center, pure evil, any good it does being an accidental byproduct of whatever evil it was trying to do. Did I mention that I don't like Norton AV?

Okay, I'll say it... "no more."

I, too, had been boycotting Mr. N. for many, many years and had used AVG, Kaspersky and then AVG again before running across Avast, which I have been recommending to everybody I know. But last summer I picked up a really nasty virus and I tried everything to get rid of it but nothing worked so I got NAV with its boot-time scan and it cleared it up. The boot-time scan feature in Avast did not work in this case (although I have used it to clean up a lot of other computers); I think that the reason NAV worked is that it ran from a boot CD (I suspect that the virus had compromised my Avast install).

If I didn't pay for the darned program I would have switched back to Avast... So why doesn't Norton have a reasonable return policy- like "If you are not 100% satisfied just return the unused portion of our product and we will gladly return the unused portion of your money!"

As much as it pains me to discard something that I already paid for, I guess I really ought to dump NAV and go back to Avast. One local computer store offers to install Avast on all of the computers that they repair or sell- along with Malwarebytes, Ad-Aware and Spy Bot.

Steve Ahola