Approximately 7 minutes ago a good friend of mine accidently typed delete from customer; and pressed enter.

Needless to say, this caused quite a stir amongst our little techie department when faithfully the mysqld responded Query ok, 270249 rows affected. Oh yea, and our backup machine (that's what the sticker on the front of it says anyway) powers up, but doesn't boot. Nice, huh?

We were all about to flip out, especially the nameless soul who made this slip of the keys. Fortunately for all parties concerned that table is used primarily for billing and tech support. Our billing cycle doesn't run until tomorrow morning so that means it's only tech support who's affected. The Yeti (who's intimately familiar with the database, God bless him) had an old backup of the table stored in an alternate location on the database server and restored it just seconds ago. Our database guys are now in the process of recovering the missing two months worth of data from a logging table used by the signup process.

With no backup server this could have been a huge disaster. Luckily we're able to reconstruct the data from other sources. So the moral of the story boys and girls is ALWAYS BACK UP YOUR DATA!!