On-Prem

Sysadmin left finger on power button for an hour to avert SAP outage

Tedious Y2K maintenance made chap a bit vague, then a bit sore


Who, me? Welcome to the seventh instalment of Who, me? The Register's new column in which readers share stories of the times they broke stuff without any help at all from users.

This week, meet "Jeremy" who back in 1999 scored his first "real" IT job "as part of a team sent out to run the IT at a big publisher."

Said team was working on Y2K-proofing the publisher, which among other things meant updating firmware on a big fleet of old Compaq ProLiant 1000 and 1500 servers.

"These servers had mechanisms to prevent people from shutting them down by accident," Jeremy told us, going on to explain "small plastic cages in which the power button was hidden and could only really be pushed with a screw driver or a ballpoint pen."

Jeremy and his colleagues made decent inroads into the server fleet, but on the 100th upgrade may have let their concentration waver a little. They got as far as Windows NT's "You can now power off your computer" dialog, at which point Jeremy asked his colleague to push the power button of said server.

Which he did … but on the wrong server. And not just any wrong server, but a server running SAP. In production, natch.

Intellisense was off and developer learned you can't code in Canadian

READ MORE

At this point the nature of that power button became critical, because its design meant that so long as it was not released, the juice would flow.

And thankfully Jeremy's mate had pressed the button, but not released it.

The next hour of their lives was interesting. Jeremy's co-worker was left unable to move while the IT team scrambled, "getting everybody to log off from SAP and getting approval for emergency downtime. Then waiting for the SAP admins to shut down the server gracefully, before J could finally release the switch."

Jeremy told Who, me? that his mate asked to be relieved, as he was in a bit of pain. Those requests were denied due to the risk of the power going off and also out of a desire to make the poor chap suffer for his error.

What have you switched off by mistake? Click here to write to Who, me? with your story and perhaps we'll give it a run on a future Monday. ®

Send us news
131 Comments

SAP ordered to pay $26.4M in South Africa energy firm dispute

SA Special Investigating Unit orders payment within 7 days following alleged breach of public finance laws

SAP users aren't keen on upping spending right now

Cloud and upgrade conversions remain steady if sluggish, according to ERP spending bellwether DSAG

RISE with SAP plan fails to hit go-live date in West of England council

Delays spark fears of £500K legacy support bill, plans for new financial year

SAP accused of age discrimination, retaliation by US whistleblower

Complainant moved to 'retire in place' role after Sarbanes Oxley Act report, filing alleges

DBA made ten years of data disappear with one misplaced parameter

Greybeards thought it was clever, making this an educational experience in more ways than one

Yes, I did just crash that critical app. And you should thank me for having done so

Quick thinking turned poor judgement into genius proactivity

March Patch Tuesday sees Hyper-V join the guest-host escape club

Critical bugs galore among 61 Microsoft fixes, 56 from Adobe, a dozen from SAP, and a fistful from Fortinet

Intern with superuser access 'promoted' himself to CEO

Older and wiser colleagues couldn’t see the funny side

Health system network turned out to be a house of cards – Cisco cards, that is

Pride came before a fall for techie who thought he knew it all

If we plug this in without telling anyone, nobody will know we caused the outage

The whole incident could have been avoided by spending some spare change on a fastener

City of London ditches Oracle for SAP in search of ERP enlightenment

Four years after first approaching market for new system, public body hopes to start working with SI

Self-taught-techie slept on the datacenter floor, survived communism, ended a marriage

This is what happens when you get promoted too fast, too soon