Reg HPC man relives 0-day rootkit GROUNDHOG DAY

Okay, campers, rise and shine, and don't forget your booties...

High performance access to file storage

HPC blog This is a difficult article to write, and I’ve put it off for way too long. But it’s time to bite the bullet and make an embarrassing admission to the Register audience. I’ve been hacked and hacked hard.

Admitting this publicly to Reg readers is like chumming shark-infested waters with my own blood. Or like telling people that I have a lice infection: sure, some will say that being infested with lice doesn’t have anything to do with your personal hygiene, but who believes that?

When it comes to computing hygiene, I think I run a fairly tight ship. Every system on our little network is up to date when it comes to operating systems (Windows 7) and applications – including updates and patches.

Every system also has a full, and up-to-date, security suite which includes antivirus, anti-malware, anti-spyware, and anti-everything else. Our network is protected by our ISP’s firewall and security infrastructure plus our own name-brand hardware firewall.

I’m also pretty careful about what I download and install on any of these systems. I only download apps that I’ve checked out beforehand, from locations I trust.

Even with all that…

I got hacked. I first noticed it just after Christmas. My main business system suddenly started playing random audio. It sounded like snippets from radio or TV broadcasts, including short news blurbs, commercials, and bits of music.

I figured the problem was something associated with my browsers. I typically have a couple up at any given time, all with multiple open tabs. Sometimes embedded videos, commercials, etc, will play when the sites automatically refresh. Restarting the browsers seemed to do the trick… at first.

But it came back. Even after locking down the media options for webpages. Then it started happening even when I didn’t have a browser open – or any applications at all. Oh-oh.

I did my routine checks: full system scans (no problems noted), looking at the processes and services that were running (nothing unusual), and paring back programs that started on boot to see if that made a difference.

I also took a hard look at any programs I had installed prior to the start of the problem, uninstalling them and deleting all references to them just in case. But I just wasn’t able to find the offending program or process.

Then it was on to Google to see if anyone else reported this problem and how they fixed it. There were a few examples, and all signs seemed to point to some flavour of rootkit, which made my blood run cold.

My Problem metastasises

My first move was to check my other systems to see if they exhibited the same behaviour. I found that my laptop had the same problem as my main business system, which could mean that I'd picked it up on the road.

I immediately stopped using email and stopped using these systems to access our shared NAS. I didn’t want this thing to spread any further. And I certainly didn’t want to become a carrier by spreading it to clients or business contact via emails, documents, or any other befouled form of communication from my sick machines.

Attempted cures fall short

I tried most, if not all, of the suggested procedures that worked for other users. I downloaded the recommended rootkit detectors and put them through their paces. None of them found my rootkit or detected the problem I was dealing with.

(I’m not naming names here. I don’t want to slime them just because they couldn’t find my particular rootkit. I have reason to believe that my infection was somewhat unique at the time, or at least rare, and you can’t expect any tool to be 100 per cent accurate all the time. But rest assured that I tried the most prevalent and best-reviewed tools out there.)

Bringing in the big guns

All of the above steps took time, and I was falling farther and farther behind on work while trying to fix it. I finally threw in the towel and brought in the professionals. All of the major security firms have subscription or one-time services that almost guarantee a fix for your system. They use a remote agent to take control of your PC and apply hardcore tech antibiotics.

I cut a deal with one of the largest and most respected security firms, signing up for around $100 bucks to start up a multi-system subscription. Since I could cancel the subscription after the first month, I figured it was a good deal if the virus had spread to other systems in our little infrastructure.

They got to work the next day. I booted my system and stood by while they loaded tools, ran scans, etc. I could watch what they were doing and noticed that it all looked pretty familiar – full system scan, looking at processes, running rootkit detectors, etc. I finally got bored and wandered off to work on my goal of doubling my body weight in less than six months.

After they logged off, I fired up the system and was greeted 10 minutes later with exactly the same random audio.

The next day was the same routine, but with second-line support – smarter guys, I guess. I booted up, they played around for a few hours, and then shut it down. I reboot, find the problem is still there, and call them back. The second-line tech confessed that he couldn’t find a virus or anything out of the ordinary on my box.

It was time to bring in the A Team – third-line technicians. Guys who have Jedi-like powers over all things virus-related. They’d beat the hell out of the virus, and for no extra charge, find the guy who wrote it and kick his ass for me. But that would probably have to happen tomorrow or the next day, because they were busy.

After some bureaucratic snafus, I finally got to talk to third-line tech guy. His voice had that world-weariness that comes from having seen too much darkness in too short a time. We talked about Trojans, rootkits, data hostage schemes, and other disasters. He was definitely the right guy to tackle my problem.

He did say one thing that was very disquieting: “Dan, I do have to let you know something. I’ve seen these symptoms once before, and I wasn’t able to fix or save the system. So be prepared for that, just in case.”

Huh? Wasn’t able to save the system? What the hell? How well could this thing be hiding itself? And how did it get in?

Turns out that there are things that even a Jedi can’t do, and these include identifying and fixing some rootkits. Even figuring out how the rootkit got in can be beyond their powers because, according to the Jedi, rootkits can be piggybacked onto normal software updates and the like. Once inside, they burrow deeply into the morass of files, often disguising themselves as something innocuous.

After delivering that warning, he got to work and promised to call me when he had any news. After a few hours, I finally went to check on the system and found an open Notepad session on my desktop. It read:

“As I mentioned, this is a brand new infection that not even our most advanced tools will clean. I would recommend using a previous image as we discussed to put the pc back a few days. Please give us a call if there is anything further we can assist with.”

And that was that.

High performance access to file storage

More from The Register

next story
Seagate brings out 6TB HDD, did not need NO STEENKIN' SHINGLES
Or helium filling either, according to reports
European Court of Justice rips up Data Retention Directive
Rules 'interfering' measure to be 'invalid'
Dropbox defends fantastically badly timed Condoleezza Rice appointment
'Nothing is going to change with Dr. Rice's appointment,' file sharer promises
Cisco reps flog Whiptail's Invicta arrays against EMC and Pure
Storage reseller report reveals who's selling what
Bored with trading oil and gold? Why not flog some CLOUD servers?
Chicago Mercantile Exchange plans cloud spot exchange
Just what could be inside Dropbox's new 'Home For Life'?
Biz apps, messaging, photos, email, more storage – sorry, did you think there would be cake?
IT bods: How long does it take YOU to train up on new tech?
I'll leave my arrays to do the hard work, if you don't mind
prev story


Securing web applications made simple and scalable
In this whitepaper learn how automated security testing can provide a simple and scalable way to protect your web applications.
Five 3D headsets to be won!
We were so impressed by the Durovis Dive headset we’ve asked the company to give some away to Reg readers.
HP ArcSight ESM solution helps Finansbank
Based on their experience using HP ArcSight Enterprise Security Manager for IT security operations, Finansbank moved to HP ArcSight ESM for fraud management.
The benefits of software based PBX
Why you should break free from your proprietary PBX and how to leverage your existing server hardware.
Mobile application security study
Download this report to see the alarming realities regarding the sheer number of applications vulnerable to attack, as well as the most common and easily addressable vulnerability errors.