OMG.

[phaedra-13.livejournal.com]

Mini-background:

Integration has begun and the country is on a schedule of who goes live when. Dallas decides it wants to go live two weeks early, so I am asked to "change" three printer queues and create two new ones...at 8:00 am on SATURDAY. I do this and find out the printers aren't getting moved for another 4 hours. WTF. Two hours after they actually moved, I am told I have to delete two of the queues I re-created because they gave us IPs that had already been assigned to another group's printers. K. Fine. Done. The rest of Saturday goes by without a hitch (or a phone call).

However, today I have spent 90% of my day trying to troubleshoot three of these printers. Mind you, today is Sunday, so I was at my mom's place nearly 50 miles west of here. Yes, I brought the laptop...I am on call after all. But I sure as hell didn't expect THIS.

The first call came in at 11:13 am this morning. The printers aren't working. Spoke with Field Tech #1 and did some testing. Jobs are appearing in the print queue, but stalling out. Found the ping times to be EXTREMELY high and there was some packet loss going on. Told him to check with the networking guys to check the ports, etc.

11:43 - Talked to FT1 again. He says this is not a network issue. Did some more futzing with it. Told FT1 that if the jobs are appearing in the queue, then our software is functioning properly. This would require vendor servicing as it seems to be a physical printer issue.

12:53 - Incident Management calls and says this issue has resulted in an E4 ticket (this means it has to be resolved within 4 hours and a whole group of people - including me - gets contacted every 15 mins until it is resolved or downgraded, whichever comes first); I advised him of what I already told FT1. He asked me to make notes in the ticket to that affect. Went hunting for the ticket and cannot find it in our workgroup or under FT1's name.

1:31 - Called Incident Management to find out what ticket this is. I made some notes and saved it.

2:26 - Incident management calls to tell me Field Tech #1 has re-escalated to our workgroup. I am severely frustrated at this point and call my boss.

2:30 - My boss calls me back. She agrees that this is likely no longer our problem, but I can contact the VMS admin and see if he has suggestions. So, I call him and he says that it's possible the listener port is wrong - turns out Axis print servers are supposed to use the same listener port as JetDirect. OK, fine. I go through and delete the print queues and rebuild them with the new listener port. Testing seems to be successful.

Within a few minutes, I get a call back saying they still can't print. Yeah. THIS time, the fucking users are sending forms to the printer, but have the printer set to BLANK. Fucking dumbasses.

3:14 - FT1 calls yet again to tell me one printer is still not functioning properly. Ran a print job, pinged the IP...everything looks kosher. Told him my laptop battery was going to die very shortly and that I would finish working on this when I got home. I had JUST started eating dinner when this call came in...so damned if I was gonna dump an entire plate of food to go rushing home. I went back to a cold plate of food, re-heated it, and went back to eating it.

4:20 - FT1 calls back, FREAKING out because I downgraded the ticket. I remind him that I can't very well work on the issue without power to the laptop and that I would begin working on the situation again when I got home. "But this CAN'T be an R3..." Dude. Listen to me. I am not going to take three days to resolve this. I am going to work on it as soon as I get home, but not one minute sooner. I have approximately 20 minutes of battery time on the laptop so finishing this up now is NOT GOING TO HAPPEN. I am downgrading it so I don't have to get a phone call every 15 minutes as I am trying to drive home on the Chicago Tollway.

6:57 - Finally got home (traffic sucked balls), plugged in the lappy and logged into the network. Called Field Tech #2 (since FT1 is not answering) and began working with him on the whole situation. This printer is receiving print jobs and they are disappearing (not stalling) as though they ARE printing, but nothing is actually printing. Hmmmm. Called the VMS admin again for ideas...he says it's likely a cabling issue or something with the hardware.

Good. This means I can stop dealing with it 'cause our group doesn't support that shit.

So I call FT2 again and ask him to check the cables. Tried sending another print job with the same outcome. FT2 tries swapping out the parallel port on the bad machine with one from a good machine. Yep...it works now. Hardware issue - gets sent back to the group in AZ to have a vendor come out and make the replacement. In the meantime, I hammer out a workaround for the users with FT2 and walk him through the process. Test the workaround...it works. Yay! Finished up some loose ends with FT2 and got his OK to re-assign the ticket. Sent a quick e-mail to my group advising the situation since a couple of them are in the office before I am.

A bit after 7:30, I finally get a chance to rummage for a snack and watch TV.

What a fucking day.

And somehow, I get the feeling FT1 is gonna spaz out to my boss tomorrow (if he hasn't already)...

(x-posted to my personal journal)

Comments

[jecook]

Call your boss. demand hazard pay. :)

I have our company's On call cell phone this week. (it rotates between all the support staff) The magic time for it to go off would be between midnight and 5 am. Although I am fortunate that it has not gone off so far (the hand off occurs on wendsdays) my brane has insisted on kicking me away at 2 AM local time to check the damned thing. As a result, I have not gotten a whole lot of useful or productive sleep this week. :(

The upside? If I have to go in to work as a result of it, It's an automatic 2 hours at overtime rates. :)

[phaedra-13.livejournal.com]

On-call pay is a standard $60/day. No more. No less. So unfortunately, hazard pay is out of the question. :(

Our on-call rotates between 5 of us with the handoff occurring Monday mornings. So I am only on-call every 5 weeks. That's assuming I don't swap with someone. My brain used to wake me up once and hour every night I was on-call..for about four days. I quickly learned NOT to do that. :)

[omg-teh-funnay.livejournal.com]

I used to work for a group home for disabled adults - live in care.

My hours were from 6AM-9AM, then 3PM-10PM. If you worked more than 5 hours between 10 and 6, they paid you for all 8. Which I thought was cool. Because, I worked a LOT of 5-hour nights...

That was the job with $1000/week takehome at an hourly rate of $8.25...