It was 3 in the morning. I just got one of the pages (well, now text messages) that you dread. Site’s down. Shit. This is the 4th unexpected “Availability Event” in the past 6 months. What is it now?
I fire up the grid control panel on my laptop. Well, try to. The browser is just spinning. Grrr. Let’s see if I can SSH into the grid controller. Nope, just spinning there too. What the heck man? All the websites and applications are down. You’ve got to be kidding me!
I fire off a quick helpdesk ticket and then call the emergency line for our guys. Leave a voicemail. Get a callback in 5 minutes from them. Hmm, they are awake already?
John Doe: ”Mike, just calling you back.”
Me: “What’s the story?”
John Doe: “Yeah, well you see, the Data Center is doing network maintenance. And we just got the email 20 minutes ago, ourselves.”
Me: “?!??!?!!”
John Doe: “Yeah, that’s what we said. There is nothing we can do but wait it out.”
You know, this one wasn’t their fault. But it was the last straw. It is time to cut our losses and run. RUN!
<Read on…>