Sysadmin admits trying to axe California power grid
Homer Simpson-style rage attack
Regcast training : Hyper-V 3.0, VM high availability and disaster recovery
A sysadmin last week pleaded guilty to attempting to disrupt the power grid in California by shutting down a data center that managed the state's electricity supply.
Lonnie Charles Denison, 33, of South Natomas in California confessed to breaking a glass cover and pushing an emergency power off button at the Independent System Operator's (ISO) data center near Folsom on 15 April. The contract Unix sysadmin was upset with his employer and co-workers at the time. Denison reportedly snapped shortly after discovering his computer privileges had been revoked.
His actions prevented the ISO from communicating with the electricity market for about two hours, leaving California vulnerable to blackout conditions. In the event no blackout occurred because the incident happened late on a Sunday night, when demand was low, so California had no need to buy in excess generating capacity from other states.
Nonetheless, the incident cost the centre $14,000, UPI reports. It took 20 computer specialists about seven hours to restore the system.
Folsom data centre blues
Prosecutors allege Denision compounded his offences by sending a threatening email to an unnamed California ISO employee the next day implying he planted a bomb at the facility. The email said: "Hey, at one point I respected you... you have a new kid. So this is only because of him. Get out before the timer expires. Not long now. Take care."
ISO responded by evacuating 500 workers from the facility and transfered control of the grid to a second control centre.
Denison pleaded guilty to attempted damage of an energy facility, a felony offence punishable with up to five years' imprisonment and a $250,000 fine, at a hearing in Sacremento on Friday. The rogue employee faces a sentencing hearing scheduled for 29 February. ®
COMMENTS
Water and electrickery do NOT mix well
I worked in a medium-sized data centre (four big mainframes, four VAXclusters, numerous Windoze servers and a few PDP-11s(!!)) on the ground floor of a three-storey building with a flat roof.
Most of the roof drains got blocked by leaves (that was the official excuse - no trees nearby!) and all the rainwater tried running down one 6"-diameter pipe... the pressure forced an inspection cover partway open and it was only cos I noticed the sound of running water in the data centre (funnily enough, not a normal feature!) that we discovered the floor void filling with the water from the roof.
At the deepest point, the water was about two inches or so below the trunking carrying the 3-phase power cable. For those who don't know, UK 3-phase runs at 415V and lots of amps...
Our EPO cutoff switch killed all power instantly, as it was supposed to.
The same cannot be said for a certain datacentre in Scotland where the EPO was tripped when *they* flooded, but the UPSs kept some of the server and disk cabinets and comms racks in the machine room live for almost twenty minutes... kudos for the person who specced the capacity needed, but just as well the hardware guys told the Data Centre Manager to take a running jump when he told them to go in and start fixing things before the kit ran out of juice...
I wonder
You'd think he'd realise that the time of least demand isn't the moment to strike, wouldn't you? I'd be interested to find out
1/ Why his admin privs had been pulled, but not his building access? Why was someone who needed to have this prives revoked allowed into such a sensitive area? Why didn't they remove him to the canteen, and then tell him when he's out of harms way?
2/ What did he think he's achieve with a false bomb threat, other than making himself look an even bigger tit and making his future prospects even LESS rosey?
How to fire a sysadmin
At my place of work, we fire our sysadmins properly. That is, we get them off-site; preferably for an hour or so (lunch tends to work for us.) When the person exits the rotating door on the way out (i.e. finishes entering the big blue room), the person at the back of the line to go out remembers something, runs back to his desk, and sends a ping to the person doing the main deed.
Access is terminated by the time said individual gets to whatever transport is being used for lunch. This includes both computer access and physical access authority.
When they return from lunch (or whatever outing they were on), there is one or more boxes, containing all of their worldly possessions they had left on-site.
This way, they have no access to any systems, inside or outside the computer room.
(Note: back when we had the blackberry ssh program in testing, the software supporting that service would flake out every so often. Mysteriously, about 5% of the time it did this, someone was fired. Oddly enough, this equated to 100% of the time we were firing someone with sufficient access to be able to do anything with it - not that they could manage anything, with their access revoked, but we didn't want lunch disturbed by them realizing they were fired.)
IT infrastructure monitoring strategies
Agentless Backup is Not a Myth
Top 10 SIEM implementer’s checklist
Steps to Take Before Choosing a Business Continuity Partner
Requirements Checklist for Choosing a Cloud Backup and Recovery Service Provider
