Yay, bureaucracy
Jun. 29th, 2007 11:17 amPart of my job involves turning up ports on our switches for new server deployments. You'd think it would be easy enough - stick the port in the right VLAN, set speed and duplex, slap on a port description and turn up the port! 30 seconds worth of work, right?
Wrong.
In order to configure ports, we must first submit notification to do so, including a deployment plan (including our "proposed" configuration), a backout plan (in case something goes wrong...including config removal commands), which requires the approval of:
- The originator (ie. me)
- A secondary approver (usually the person requesting the configs)
- Our "change administrator" (some dude I've never even met)
- My direct manager
- His manager
- Our director
- Our VP
Change requests must be submitted AT LEAST five working days in advance of the change.
...oh but it gets better! If something goes WRONG (say, our cabling guys plug something into the wrong switch or server - which happens a LOT), then I must either open a ticket with the NOC - note that I have to call the NOC, not open the ticket myself - provide the ticket number to my manager, e-mail the ticket number to our VP requesting permission to change the port assignments/configs and then await him to e-mail the word "approved" to me before I can change three lines of config on the switch.
And this is only if the server is in production. Servers not in production must go through the 5-day approval process all over again.
Add to the fact that our SLA for creating deployment plans is 10 days (5 days to create the deployment plan taking into consideration our current workload, 5 days for the approval process), we could be talking almost a month just because someone wants me to fucking CONFIGURE A SWITCH PORT.
This is a Fortune 50 company as well. I wish I was making this shit up.
Wrong.
In order to configure ports, we must first submit notification to do so, including a deployment plan (including our "proposed" configuration), a backout plan (in case something goes wrong...including config removal commands), which requires the approval of:
- The originator (ie. me)
- A secondary approver (usually the person requesting the configs)
- Our "change administrator" (some dude I've never even met)
- My direct manager
- His manager
- Our director
- Our VP
Change requests must be submitted AT LEAST five working days in advance of the change.
...oh but it gets better! If something goes WRONG (say, our cabling guys plug something into the wrong switch or server - which happens a LOT), then I must either open a ticket with the NOC - note that I have to call the NOC, not open the ticket myself - provide the ticket number to my manager, e-mail the ticket number to our VP requesting permission to change the port assignments/configs and then await him to e-mail the word "approved" to me before I can change three lines of config on the switch.
And this is only if the server is in production. Servers not in production must go through the 5-day approval process all over again.
Add to the fact that our SLA for creating deployment plans is 10 days (5 days to create the deployment plan taking into consideration our current workload, 5 days for the approval process), we could be talking almost a month just because someone wants me to fucking CONFIGURE A SWITCH PORT.
This is a Fortune 50 company as well. I wish I was making this shit up.