Reg HPC man relives 0-day rootkit GROUNDHOG DAY

Rebuilding my tattered life

The next steps were straightforward, albeit hugely tedious. The first was to call Big Security Company, cancel my security subscription, and get a refund for my initial payment. Then began the process of going to my backups, putting an image on a clean hard drive, then testing to see if this new image had the same virus as the old one.

The virus’s erratic behaviour – sometimes starting right up with the inane audio, other times waiting for as long as an hour or two before tormenting me – made a long job longer. Couple that with not being able to pin down exactly when the virus first emerged, and you end up with a long and tedious job.

Fortunately, I have firm backup procedures in place. Every key system is backed up incrementally daily, with a clean image saved weekly. All of these backups are stored for 60 days just in case of, well... this.

It took what seemed like forever to find an image that didn’t have the virus on it. I ended up going back several weeks, which made quite a bit of work and raw materials (video, notes, etc) disappear. Those files had to be brought in individually and tested, just in case one of them was the virus carrier. That process accounted for another long period of time... sigh.

Targeted or unlucky? Plus lessons learned

I kept wondering how it happened and how I could prevent it in the future. Since this was a new virus, according to Big Security Company, why were my well-protected computers the first to get hit?

Did the Trilateral Commission finally decide to even the score with me? Or the Yakusa? Russian mobsters? Or someone who was just looking to mess with me? It could have been all of the above, or maybe it was just my time to be a zero-day guy. I do have some slight grounds to suspect that I was targeted and if I find out more, I’ll let you know.

What did this experience teach me? To be more fearful than ever. I still don’t have any idea what I was infected with, how I picked it up, or how to prevent it in the future. According to third-line tech Jedi at Big Security Company, this is the world we live in today. Our cyber safety is under constant attack, and the bad guys have the first-mover advantage.

The most valuable lesson? It pays to back up, and an investment in fast and solid NAS boxes (shout out “Thank you, Synology!”) is worth every penny. My terabytes of backups saved the day and got me back in business. Without them, I’d be looking at clean installs of everything and then a file-by-file inspection and test of all of my stored data. Yikes.

I don’t want to think about how long this process would have taken if I were trying to do this number of restores via the cloud. I have a fairly fast pipe into the home office; it typically tests out at 20Mbit per second. But when you’re talking about full-sized images of around 150GB, it would take anywhere from 15 to 18 hours to complete a single download. My local NAS was able to copy these images over in half an hour or so.

Now that I’m back, it’s time to start dealing with the backlog. I have plenty to tell including some interesting and compelling experiences at the SC13 supercomputing conference, stories on my trip to the second annual South Africa Student Competition, info on the upcoming ISC’14 Cluster Challenge, and the usual HPC industry happenings. Stay tuned. ®