power=internet

[anivair.livejournal.com]

A few moments ago power went out in the entire building. my UPS and the server's UPS kicked in and I started rebooting things.

As I was getting ready to do so, I encountered an employee in the bleak darkness of the hallway: "Fire fox is giving me an error message."

Ya think?

I work with geniuses.

Comments

[torkell]

...and I started rebooting things

That doesn't sound good. the whole point of a UPS is to kick in *quickly* so things don't have to be rebooted.

Reminds me a bit of a time when the power was glitching slightly - all you'd see were the lights flickering for a moment, but it was *just* long enough to reboot the computer I was using. This is why I like laptops.

[anivair.livejournal.com]

not the UPS stuff, other smaller things. Sometimes the routers like to be power cycled after the power does that and while everyone is waiting to get back on I oblige them.

[torkell]

Ah. That makes more sense.

Then again, at my last school before they replaced the UPS the batteries lasted for about 30 seconds. This meant the sysadmin got to the server room just in time to see everything die. (now he's got a much nicer 1-hour runtime)

[oh-kevin.livejournal.com]

sounds all too familiar.

you'd think a basic level of computer skills, and common sense skills would be required for these office jobs.

or you'd like to think...

[darkblade1.livejournal.com]

I remember one day at the call center here where the power went out and the UPS didn't kick in on time....

...in fact it didn't kick on at all.

It was the best 1/2 we had at work.

[redqueenmeg.livejournal.com]

During that big power outage where basically the whole northeast of the US was out, we got hundreds of calls from users wanting to know when their servers would be back on.

[xdownfornowx.livejournal.com]

heh... guy I knew that worked at time warner ran a fridge of the generator that kept power to the data center. They could have closed shop if one segment of new jersey would have blacked out. at least they had cold soda.

[jecook]

Heh.

This reminds me of a story that my ex boss at Global crossing told me.

He was working in the Network Operations Center in Sunnyvale, CA, and there was a massive power outage. Fortunately, the UPS systems kicked on, but the generator to the data center did not. It would not even turn over. Upon further investigation, it was discovered that the battery was dead. So my boss had to run to a auto parts store to buy a new battery for the generator, with only about 10-15 minutes of run time left on the batteries. He didn't make it.

After the dust cleared, they discovered that one of the overnight operations staff had a dead battery in their vehicle (I was told it was a motorcycle, but who knows) and swapped batteries. That person was fired very shortly thereafter.

[jecook]

Oh yeah..

Then there was the time shortly after 9/11 where one of our intercontinental fiber landing sites was low on fuel, because no one was sure that it was ok to send a tanker truck down there to refuel the site's generators...

That one I don't talk about too often, though. Too many bad memories attached to it.

[eightofspades.livejournal.com]

Reminds me of an afterhours call. I support a 24/7 operation, but fortunately I don't have to be bothered by petty stuff.

"spades, we haven't been able to update the shift log for about two hours now."

I do some routine remote stuff - can't get in. I've had issues with VPNing before - usually a router loop out in the WAN, which I don't have jurisdiction over. That hurts *me*, but their shift log is kept locally on a server, and only internal problems would affect it. I get a few other error messages out of him. Then...

Me: "ok, I'm going to have to call you back, I need our desk to tell me if we have an external issue stopping me from getting in."

him: "oh, well here, call this number back. we lost phones about twenty mintues ago" (current time of call, eighteen minutes)
ooooOOoo...

me: "are there any other issues?"

him: "Yea, half of the building doesn't have power. The electricians know about it, though."

me: "Did it occur to you that the other half of the building is where my server room is, and that the phone system's power supply probably just ran out too?"


Turned out an electrician sometime in my building's undocumented past had set an external outlet on the same circuit as part of the room (still trying to get that fixed) and a janitor had plugged in a floor buffer.

That was fun to clean up..