So, I get to work this morning, and immediately about 10 people surround me to advise me that the data on the main data drive “looks funny”.

I check, and it is indeed “funny”.

It appears that our 24,000 directories (folders to you newies) containing 136,000 files, and around 40GB of corporate data have been deleted overnight, before the two backups (one to another server on a different site, the other to tape) occurred. So, now we’re missing all our data after July 26th. Great.

It took all day from 9:00am until 4:00pm, but we managed to recover the data from 12:30pm on the 27th using the incredibly awesome Shadow Copy that comes with Windows Server 2003. We also restored the tape backup from the night of the 26th over the top, only creating files that were missing (under 200 out of 136,000 isn’t bad).

Half way through the day, I learn that today is Sysadmin Day. Amazingly convenient for such a data loss to occur!

Given that all our backups from last night were completely screwed, I’m impressed that we only lost a few hours of data (and most of a day where people couldn’t use that file share). Stressful but today has certainly gone fast. Maybe I’ll now have a lunch break!