What a night. We had quite severe weather through here last night. Winds of 60-70mph, torrential downpours and all the lightning you could stand. (Yes, I did go out and shoot some — hopefully they’ll be posted in a day or two.) There’s still over 100,000 folks without power, and no ETA on when they’ll be brought back online.
We had dinner with the gang after the first wave came through at a local Mexican restaurant. As always, good food and great fun. While we ate, the fire engines raced east and west on Manchester, from several firehouses, and apparently responding to all kinds of calls. We had heard that Ballwin and Manchester were hard hit, but had no idea how bad. The news today makes that area look like a war zone. Unbelievable.
We’d decided to come to the house, make a bunch of drinks — Mary got a new “bullet blender” for her birthday Friday, so we decided to break it in — and play some poker. We hit the front door about 9pm, and just about as soon as we were in the house, my pager started going off. And it was bad, bad news.
Through a series of unfortunate (and unexplained, at that time) events, the power to our data center was gone, the generator didn’t take over, and we were on batteries at the site. We all scrambled to kill apps, kill servers, and try to get things switched to the backup site before the batteries ran out. We were informed that the power came back up, the datacenter was cooling from 100+ degrees, and that things were starting to get back to normal from a facilities perspective about 10pm.
However, the team on the call (and several others) continued to try to get things working well, and we were on the line until 3am this morning. An ugly, ugly night. I’ve slept the better part of the day, and am just now getting to feel like myself.
In just a little bit, we’re gonna rip the bandaids off the owwies from last night, and move everything back to where it’s supposed to be. Hopefully, this will be a short deal, and nowhere near as painful as last night’s events.