Thursday, May 8, 2008

Back it up Buddy!

OK so it's not easy to get the message home about backups without being a complete nag. I know it's important; You know it's important; We all know it's important so stop nagging me about it, already! Right? - Well no not quite!

You see there's knowing that something is important and understanding that it's important. "What's the difference?" you may ask. I think we can know something on an intellectual basis but it's when we feel it's consequences on an emotional level that we really understand.
When I tell you that backups are important in case you have a hard disk failure, you may nod sagely and say "How true!". That's knowing the importance of your backups.
When your hard disk has failed and I ask, "Do you have a good backup of your company documents?" the thoughts rushing through your mind at that moment are a profound understanding of the importance of that backup.
For my sins I've been party to these conversations many times. I've often felt when I'm nagging someone about the importance of backups that I'm not really getting the message through. I've also had to inform businesses (both large and small) that have had a disk failure that they didn't have a valid backup. Honestly it's a wretched task. Reactions include panic, tears and aggression (usually toward the messenger). The consequences aren't pleasant either; Professional embarrassment, lost work, cost of recovery efforts, even dismissal.

Here are some of the causes that I've seen in the past:


1) The backups were simply not done. -- Very common!
There was no backup strategy at all or it was not carried out. No-one changed the tape or disc.
2) There was only one backup copy
This one is painfully common. Someone saves money by only having one backup media. Say a tape or a usb drive. Then there's a catastrophic failure during the backup. Now the original data is gone because of the failure and the backup is bad because it has just been (halfway) overwritten. It's a false-economy; Spring for a second usb drive or make sure that you have at least 5 tapes.
3) The backups were being performed but no-one checked that they were being successful.
I've seen this one more than once. Heck, I've had this happen to me more than once. This is where you've got your backup system going. Say you backup to tape and you swap the tape every evening without fail but at some point the backups started to fail and you didn't know it because you didn't read back a tape to check that what you expected to be on it was there.
There are a myriad of reasons why a perfectly good backup will stop working. Here are a few:
a) The amount of data got too big for the tape.
b) Someone moved the data directory but forgot to update the backup tool.
d) Folder permissions were changed and the backup process no longer had rights to the data.
4) The backup media was over-used.
This is common with tape backups. They don't last forever you know. Eventually they wear out and the magnetic stuff comes off the ribbon. I've seen this one happen in a large company. All the tapes were bought together and they all wore out together. When we needed a backup we found errors on the first two tapes that we tried. That was the longest few hours while we tried each backup in turn. I had a nightmarish vision that all five tapes would be bad. Incidentally this would have been caught if the veracity of the backups was being checked regularly.
5) Incremental backups. -- This is a real pitfall.
Incremental backups are where you backup all the data on Monday night. Then you only backup the changes on subsequent nights. If you need to restore then you first restore the full backup and then apply each incremental backup until you have the full restore. There are a few pitfalls with this:
a) By the end of the week you are relying on 5 backups so you're 5 times more likely to have a failure. If any of them fail then you have a problem. If the full backup at the beginning of the week fails then the incrementals are useless and you have no backup at all for the week. If you are (foolishly) working with a single set of 5 tapes (or disks) then you have NO BACKUP AT ALL
b) In small companies that don' t have a professional admin person it is really easy to put the wrong tape in and overwrite a previous increment.
c) My advice is ... Don't do incremental backups unless you have it down pat.
5) Backups were successful but not protected.
Well they were probably successful but we'll never know for sure. Sometimes people think that the only thing they have to worry about is their hard drive failing. It is the most common failure but I've seen a small fire take out the computer server and the backups that were in a plastic box beside it. Backups need to be cycled off-site or stored in a fire-proof safe. A small fire-proof safe only costs a few hundred dollars. I'd recommend getting one rated for at least 2-hours.

To Sum up ...

One of the advantages of going paperless is that it lets you backup all of your documents easily. It could take months with a photocopier to accomplish the same task for paper documents. However the disadvantage is that barring a natural disaster such as flood or fire your a hard-drive is more likely to lose your documents than an old fashioned file cabinet. So be prepared and make sure those (easy to do) backups are done.

I must go... I've got to check my backups.