SaaS data loss: The problem you didn’t know you had

Sysadmin blog Right on cue, Java has responded to my hatred in kind. Shortly after I awoke to discover my previous article denouncing the language had been published, a client called to inform me his computer had contracted some malware. Java has, if you'll forgive the anthropomorphization of a bytecode virtualization engine, decided to exact its revenge.

Closer inspection of the infection revealed deep network penetration that the installed antivirus applications were completely unable to cope with. The chief financial officer of the company relies on cloudy applications that require Java-in-the-web-browser. Contrary to early reports that we should only fear Java 7, this beauty crawled in through a fully up-to-date Java 6 browser plugin and installed some friends.

The payload: a software nastie called Sirefef. This itself is actually irrelevant; even Microsoft Security Essentials can find and kill most variants. The purpose of Sirefef is to serve as the staging component for the coup de grace: the highly sophisticated Zeroaccess rootkit (Sirefef downloaded some other friends too, but once the rootkit is dealt with, they are easily dispatched.)

Zeroaccess is a nightmare. It creates a hidden partition to run components from, deletes the BITS and Windows Update services, infects system restore and then removes the system restore interface from Windows. It locks you out of various sections of your file system it has decided to secrete backup copies of itself into. (C:\Windows\Temp, C:\Windows\System32\Config\Systemprofile and so forth.)

Zeroaccess knows all the standard tricks; it hides itself from Trend Micro's virus scanner Housecall, kills industrial-strength bleach Combofix (attempting to run this tool will freeze the system), resists cleaning by SurfRight's Hitman Pro, Symantec's resident AV and so forth. If you delete the hidden partition after booting from a Linux Live CD, chances are you didn't get every last remnant of the thing and it will be back in due time. It also prevents remote support app Teamviewer from starting properly with Windows.

If any residue of the rootkit lingers, or if Sirefef and/or its downloaded friends remain, they will all download and reinstall one another and we get to play whack-a-malware one more time. Bonus points were awarded for exploiting known Windows 7 vulnerabilities to infect every other machine on the network; that was a nice touch that really made my Friday.

Cleaning up this one Trojan-horse town

So what's the solution? It turns out that some combination therapy kills the Zeroaccess variant in question. The solution I have settled upon is this:


1. Disconnect every Windows system from the network; if one is infected, they are all infected. (I have absolutely no idea what they used to get through the firewalls on client PCs, but it was effective.) You need to clean all systems one at a time on a quarantine basis. If you have a way to automate the rest of this list for enterprise deployment, please let me know.
2. Create a new local user with admin privileges, reboot and log on as that user. (You need as clean a profile as possible.)
3. Download and run Symantec's Zeroaccess removal tool. It will ask you to reboot; do so. A widget will pop up when you next log in that says the rootkit was not found. This is a lie. The removal tool got rid of it, and you have already been reinfected. Fortunately, it can't do anything until the next reboot.
4. Run Trend Micro's Housecall; kill all the things. Do not reboot.

5. Repair the background intelligent transfer service (BITS).

6. Repair the Windows automatic updates service. (If you get the popup for the "Microsoft Fixit" tool, use it. It will fix your broken Windows update service.)
7. Install Windows updates. Do not reboot.

8. Run Microsoft Security Essentials; kill all the things. Do not reboot. At this point, you should have killed all of Zeroaccess's little friends.

9. Re-run the Symantec Zeroaccess removal tool. It should kill the newly reinfected (but still dormant) variant of Zeroaccess.
10. Reboot. When the system comes back up, make sure you log in as the "new" local administrative account you created.

11. Run Combofix. If it doesn't lock up your system, you're good!
12. Reboot back into your regular account, and delete the local account you created for this process. You win.

If you are infected with Zeroaccess, exercise extreme caution. Someone is actively versioning this rootkit. I detected at least three different variants on one network alone. More to the point, the little friends that serve as satellite malware are also seeing some rapid evolution; what worked for me today may not work a week from now.

This incident should serve to underscore exactly how serious the Java exploits in question are. If you can, uninstall Java. If you must use Java, keep it as up-to-date as possible and see if you can disable or remove the plugins for your browsers. (In an attempt to help resolve the current crisis, Ninite is offering free access to the pro version for a limited time; it can really help with the updating.) If you absolutely must use Java-in-the-browser then it's time to start taking security very seriously; break out the tinfoil and start making some shiny hats.

Java-in-the-browser absolutely must be treated as "already compromised". There is no wiggle room here. Do not under any circumstances run Java in the browser on any production system or any client system in which any other application is used. Go buy another Windows licence and put Java inside a virtual machine.

Ring-fence the virtual machine by placing it on its own VLAN and subnet. Keep that virtual machine's traffic as separate from the rest of your network and system as you possibly can: Java-in-the-browser is a live grenade and you can't afford to have it go off inside your network. If you can, deploy the virtual machine from a managed template; the ability to destroy it at the end of the day and revert to a "known good" is a huge advantage when dealing with a threat of this magnitude.

Even if Oracle gets its act together and solves the immediate issues, this is only the latest in a long line. Java is simply is not developed with an adequate "security first" approach; Oracle is used to dealing with large corporations, not consumers. It doesn't have the experience to fight these kinds of rapidly escalating arms races, and it shows.

There isn't time to wait for Oracle to overcome its corporate inertia. It is time for systems administrators to act. It is our duty to depopulate Java with extreme prejudice. ®

Steps to Take Before Choosing a Business Continuity Partner