techrecovery | (no subject)

I'm a network technician for a major telecoms company. Weeeeeee! I'm not sure I like the job yet, but it's double my salary and half the work, so I'll stick it out for now...

Well, it only took two weeks for the asshattery to start.

Amongst other things, I handle some escalations from the customer-facing NOC. Some dude calls the NOC and says his public LAN IP subnet isn't working. And then it gets punted to me.

First of all, the guy in the NOC e-mails me two traceroutes, asking why the first one goes through Comcast. The second one stops at our (Cisco) edge router in Dallas "even though he is based in Austin". Grinding my teeth I pore over his e-mail and very carefully type a response that won't get me fired.

"[Felipe],

The first traceroute goes through Comcast because you have mistyped the IP address. The IP in question belongs to Comcast (verifiable through a WHOIS) and is not one of our /14s. The second traceroute stops at Dallas because this is the edge router his T1 is connected to. This is the last hop that responds to ICMP requests. The customer has a firewall that is configured to deny ICMP requests. There is no issue with routing, as evidenced in this output:

show ip route [x.x.x.x] [Output snipped; shows correct serial port for the subnet to routed to]

Kind regards,

[Me]"

Thirty minutes later he e-mails me again saying, "The customer is still complaining something is wrong, please check these show ip route outputs. The subnet has a /30 mask on one router (referring to our redundant edge router) but a /28 on the other. Please advise."

Howling like a banshee and foaming at the mouth, I respond:

"[Felipe],

There is nothing wrong with routing. You have mistyped the IP address in your show ip route command to show routing for a /30 block. Here is the correct subnet, as well as an output that shows routing is perfectly fine. We cannot ping the customer's side of the WAN nor his LAN IPs because his firewall does not permit us to ping them:

[Snipped]

[Me]"

I understand why people disable ICMP, but it's fucking annoying when it's a new service and we're trying to verify connectivity. Jesus, nobody's going to DoS you as soon as your connection comes up/up.

(FWIW, I think the guy has his firewall misconfigured. But of course he's going to call us first.)