Oh Shit!

Originally published at ace's blog. You can comment here or there.

So I had one of those oh shit moments that you have when you f*ck up and replace some of the production tables (files for the Lansa crowd; I never understood why Lansa insisted on their database TABLES being called files...they aren't files pig fukkers, they are TABLES...ya damn idiot).

Whoa! Where'd that come from?! Ok, back to my point. I totally fucked up. Shit, I said fuck. When I looked down at my screen and saw that I was manipulating the production DB, a giant wave of warmth went over my head, down my body and stopped at the pit of my stomach. You know that feeling you get when your stomach just isn't feeling right? There's a whole lotta gurglin goin on in there? That's how I felt at that moment, minus the gurgling, I just had that "oh now that's just not good, not good at all" feeling. You know the one.

Ha! That was forced, I'm not lauging nor am I in a haha fucking mood; my bad. It is funny amusing how some words can seemingly minimize a very drastic circumstance. I said "manipulating the production DB" like I changed a few records (tuples, rows, whatever) in there when that simply was not the case. I replaced three major files in the system with backups from the day before. No big deal you say?! Bullshit. You didn't do it; it was all me baby!

So I call my buddy and say "hey dude, restore this file from the backup from last night after all of the important stuff ran". He says "Ok" and then I start to think "wow, maybe this wasn't so bad after all...where'd that silver lining come from?" Then I get home and get a not so welcome phone call from my earlier buddy: "Hey, um, the file you wanted restored has zero records in it. Is that what you expected?" Fuck me. No! That is not what I expected. Can you recount them and make sure there are zero there?!

Well sunuvabitch. So if all the important stuff ran last night and this backup is from after that point, then there was just no data in the file right? Whatever, I can deal with that. That file wasn't THAT important, just one of those nice-to-have kinda things. Well then I start thinking (why did you just say uh oh?). I start thinking about this other file that I replaced.

Dammit, that will need to be restored as well. Ok, no biggie; we still have the backup tape on-site. So I restore that file. It takes about 2 hours. Then I look at the file that was restored and it has the exact same record count as the fucking file from the day before. So what does that tell me? That the backup time I was told was wrong and must've happened before the important stuff ran.

So now here I am. It is 11 pm. I'm not tired and am worried sick. I've had this shitty acid reflux feeling/taste in my throat since I discovered I was playing with production data at 3:30 this afternoon. The important stuff eluded to earlier is now running again (should be anyway; I have not checked) for today (it's an everyday thing). I need to sleep so I can wake up bright-eyed and bushy-tailed and ready for my morning of "hey, can someone help me? I'm a moron/idiot".

So anyway, it is now that time to check and see what's up in production right now. Time to worry more. Time to go to bed and just lay there till my alarm goes off at 4:15.

Happy happy joy joy. Anybody remember Ren and Stimpy? They were awesome.