EB Emergency Room Doctor

[http://users.livejournal.com/_caecus_/]

We have had problems with a unix cluster here at work, lately. The problem is, some of the memory was going out in the Prod 1 server, which is 1 of 5. Normally, it wouldn't be that big of an issue, as there are other servers to handle the load. The problem is, the software on the cluster is throwing errors, because the software is still trying to pull from Prod 1 and it's an intermittent failure. We wind up having to do an emergency down time, and I have it announced overhead. The system will be down for approximately 30 minutes, which is a generous estimate, as it should take more like 15 or 20. There is a down time system that handles all requests, it's just a little old, slower, and takes a little more work on the part of the clinical staff. I will also add that the system had become inaccessible a few times last night, because of the hardware related errors. That's a huge deal when it just stops working now and then. The down time was to prevent it completely going down and the hospital not being able to work. This is the system that handles all medical records.

I get a call from Dr. Douchebag, who will we call Dr. D for the sake of this post. First of all, he's yelling right off the bat, and I can't make out half of what he's saying. I get a few bits and pieces of how this is going to affect patient care efficiency, but I honestly couldn't make out most of it because it was almost like he was trying to eat the phone while yelling into it.

A few minutes later the House Supervisor comes down and asks if I can explain to Dr. D why the down time is necessary. I explain that he already called me yelling. He asked if I would have the administrator for the system, who is also my manager, call and explain it, because she has more authority than I do. I say okay, and I transfer my manager to him. The House Supervisor, which is Dr. D's supervisor, is telling me that if this doesn't work, he's going to have to call either a VP or the President at home to get them to handle it. He said he was honestly wondering if he was going to have to have the physician restrained by security when they were having this conversation, as the physician was acting like he was going to become violent.

Anyway, long story short, I didn't hear anything else about it. The system went down, came up, and ran like a champ after the down time. I had it announced overhead. I am amazed that this doctor could not be inconvenienced for twenty minutes. He spent more time ranting about the system being down, than he would have lost if he had just worked through it.

Comments

[ace-brickman.livejournal.com]

heh - at our shop, we'd say that [physician] was too busy "walking on water" to respect the reality of the situation. Both area hospitals (different owners, different on-site IT depts, we don't touch any of their servers) have had Citrix client issues ever since they decided that a 300% increase in server load wouldn't require an upgrade.

"This affects patient care" - Ugh, the 4-word phrase that sends maniacal shudders down my spine... Happy Monday!

[http://users.livejournal.com/_caecus_/]

That sounds almost like something I'd expect to see here. I know what you mean, because that means it will most likely bite you at some point. Monday is always happy for me. I work Weds-Weds, so it means I'm nearing the end of my work week, my days get shorter, and we are more heavily staffed. ^_^

[mouser.livejournal.com]

These are the doctors that can't type their passwords in to change them twice in a row, that can't type in "Password0" as the default starter password, that ALL need the tablet computers for seeing patients, that can't follow simple instructions ("Log off when you leave a computer, don't just lock it.")

Our nurses aren't as bad. They just suck down ALL the bandwidth listening to internet radio. And forget their passwords every Monday.

[gholam.livejournal.com]

Regarding internet radio, I generally just block access to all streaming (<3 Fortinet), open up one or two sites with specific exceptions, and QoS it down to hell.

[http://users.livejournal.com/_caecus_/]

We use Websense and most people don't have access to anything outside of the intranet. The ones who do are highly monitored, and most categories are filtered, anyways. There is also a quota system for power users where they have 60 minutes of free quota time in a 24 hour period to goof off.

[falnfenix]

damn, you folks are STRICT!

[http://users.livejournal.com/_caecus_/]

Well, it got to a point where you could barely access the internet when you needed to, so they fixed it. ^_^

[mouser.livejournal.com]

We use Cymphonix - it's tied into their Windows ID by policy.

We don't disable anything but the worst sites - we just set the bandwidth to "PAINFUL" levels. They load so slow they quit bothering after a few tries.

(It's really fun when they call in complaing. "Show us" and they only use the work sites - which work perfectly...)

[http://users.livejournal.com/_caecus_/]

We do something similar to that, and they download web definitions of a lot of sites that makes most of them inaccessible. It also filters by categories in meta tags. Legitimate sites we use and things related to medicine, it automatically lets through. Most people do not have any access to the web at all, though. Your manager has to justify why you need it and submit a request to be administratively approved by our security people to get access at all.

[http://users.livejournal.com/_caecus_/]

Yes. The sites used to work purposes are in the ACL. It's anything remotely questionable like music, games, entertainment, etc that are filtered out. You do have to click a tab that says, "Use Quota Time," and shows your remaining minutes, though. They break it up into 10 minute blocks.

[falnfenix]

your comment makes me happy that at my old job:
tablets were banned
internet radio was blocked
we had ONE GENERAL PASSWORD for all basic public workstation functions (even though they'd lock that password nearly every morning, it became routine to pull up that profile and check the unlock box around 8:30 daily)

[jimbojones.livejournal.com]

suck down ALL the bandwidth listening to internet radio

Ah, CHRIST how I hate "internet radio". The bane of network throughput at nearly every fucking client I have.

Every. Single. Fucking. One. of the offenders always has a perfectly functioning REAL radio right there in their cubicle, too... and more often than not, it's a LOCAL RADIO STATION that they're streaming over the net.

Then they complain because "the VPN is slow".* SIGH.

* in 5+ years of steadily recommending that sites with mission-critical VPNs get a second, cheap-as-hell DSL line for "regular" internet traffic while reserving the expensive, stable internet connection they already have for VPN traffic, not a single fucking one has taken me up on it yet.

[bothunter.livejournal.com]

Muhaha.. I stream KEXP's (http://www.kexp.org) 1.4Mbps uncompressed stream, and their studio is 4 blocks from my office.

[gesundyke]

maybe they'll find the stick at dr. d's next colonoscopy.

probably not, but hope springs eternal...

[qextor.livejournal.com]

I know a group of employees (Nurses and others) in a Seattle emergency department, who were quite annoyed they couldn't access petfinder.com because of the filters and such. So they spent the next 3-4 hours, in between patients (it was a slow night), to find a work around. They managed to make (spoof?) petfinder a mirror of a medical site that was allowed.

[http://users.livejournal.com/_caecus_/]

It's the same with us on those, fax machines, and copiers. We contract them from a local company. Half the time people act like we are lying when we say we don't service them. They drive me insane.