*facepalm*

[jprocksmyworld.livejournal.com]

For a couple of weeks now, on random days, the server in the P office has been dropping off the network. It's been happening between 5:30 and 6PM. The staff finish at 5:30. It's not the cleaner. There's a sign on the server "Do Not Turn Off". There's apparently a sign on the plug "Do Not Unplug". I've got no idea since they swear blind they aren't unplugging it or turning it off. It always comes up when they push the power button in the morning.

We clean the server internals. I ship a "hot swap" replacement. It stays up over the Xmyth period.I'm thinking it was an overheat problem or the like....

Today, when the P branch staff are leaving at 3:30PM, the server drops off the network again at 3:35. I notice. I call the branch. I get them just as they are leaving. I ask them to check it. It's off. They plug a radio into the socket. Dead. I'm thinking blown fuse or tripped circuit breaker. Maybe dodgy wiring that when they turn the lights out... Then I get told that the server's plugged into an extension cable. I tell them to trace it back. It's plugged into another multi-way. Which also has a users PC plugged into it. The user is turning their PC off at the wall. Which has the multi-way plugged in.

The user doesn't always turn their PC off at the wall. The user isn't always in the office. Sometimes their machine doesn't get turned on. This explains the apparent randomness.

When they come in in the morning, the first thing she does is turn her PC on - this means the server's got power- so when they come to check it for me, it powers up on the push of the button.

Todays lesson: get them to trace the cable *all* the way back to the *wall* socket, not just "the socket".

[x-posted to my journal]

Comments

[indigo-max.livejournal.com]

oy, that would be aggravating. Reminds of some stories that I have heard of in the past. Cleaning crews unplugging a server so as to plug in a floor buffer or a security guards magnetic flashlit being slapped up against a computer during nightly patrols.

Max...

[tertiumquid.livejournal.com]

I've seen a couple of similar situations.

1. Anesthesia machine in an operating room. There's an outlet strip made onto the back of the machine. It's plugged into another outlet strip. The other outlet strip is plugged into...
you guessed it. The back of the anesthesia machine
Funny how that recycling electricity doesn't work.

2. Horrid wiring at an HVAC company.
We notice that every once in a while the entire network goes down. At the same time the phone system also dies.
The entire server room, including file server, the WAN switching gear, Modems, and the PBX are all plugged into one little 500VA UPS. The UPS battery hasn't been cycled, much less checked in years.

Unfortunately, the outlet that all of this is plugged into isn't even on it's own circuit. There's an old Xerox around the corner that's rarely used. It's plugged into an outlet that happens to be on the same circuit.
Whenever someone uses the copier, the excess current drain of this dinosaur drops from the nominal 117V to less than 90V across the entire cirsuit. This kicks in the UPS which has a dead battery and brings down the entire system.

[jecook]

I suppose I count myself as lucky then, that the refridgerator that is plugged into the same outlet as our web/backup proxy server has not cause any problems, although I'm hoping to remedy the electrical "issues" when me move this office.


Issues like:

no UPS on anything: not on the phone system, networking gear, or the aforementioned server. just a "surge" strip which looks like it's seen it's tour of duty.

[sunfell.livejournal.com]

I've had to do some repairs to our network cabling because of people who like to run vacuum cleaners over our wiring. Some of the wiring under the desks are snakes nests, and in spite of our efforts to keep it neat, we always get at least one call a month for a nonop network cable.

Hah...

[coyoteden.livejournal.com]

In your case it wasn't the cleaners, but I have seen that happen.

And no, it wasn't the infamous "floor polisher polishes off life-support patient." Life-support machines have batteries, alarms, and radio alerts to the watch desk if the power is cut.

I did helldesk at a university, and we had a problem with an entire building's LAN and every building that fed into it just dropping off for an hour every few nights. We checked the switches, routers, fiber, everything.

Then we realized the backbone fiber transciever was in a utility closet and powered by an ordinary wall-wart. Apparently the cleaning staff would unplug it to plug in their vaccuums and polishers.