If you're interested in the sweep of history, as I am, then you really should find the time to read through Jacques Barzun's magisterial From Dawn To Decadence: 500 Years of Western Cultural Life: 1500 to the Present. In particular, Barzun delivers a wealth of telling anecdotes that perfectly illustrate a point he's trying to make about human behaviour throughout history. One of my favourites is concerned with trains.

On 15 September 1830, passenger travel on steam trains was inaugurated with a 30-mile train line between Manchester and Liverpool. Guests on the world's first passenger train that day included the Duke of Wellington, Doctor Brandreth, and William Huskisson, a man noted as an economist, Member of Parliament, and President of the Board of Trade. After moving along at a steady (and incredibly fast, for the time) rate of 20 miles per hour, the cars stopped about halfway along the journey for water. During the stop, the passengers got out and milled about but, unfortunately, another train approached and struck Huskisson, fatally injuring him. Reports from the time explain that Huskisson was confused by the noise and the movement, and couldn't get out of the way in time, which makes sense. Here was a radically new thing - a large man-made moving object on land - which required a radically new way of adjusting to it. Huskisson didn't adjust in time, and paid with his life.

Security professionals are faced with new changes in technology all the time, and we have to adjust as well. Even more so, we have to shepherd the "normal" users we oversee through these changes, and sometimes - no, most of the time - that can be quite difficult. Unlike William Huskisson, we usually don't have to worry about someone losing a life if things go wrong; no, instead we have to worry about something many people would consider worse: a lawsuit.

Let me explain by asking you two questions. First, do you have a data retention policy in place? I'm sure you do. You have a well-thought out and well-planned backup policy, and your backups are run every day - or perhaps several times a day - and regularly tested to make sure that all is working as planned. Your policy is communicated to your users, and they understand that their data is saved and can be recovered if necessary. Life is good.

Now my second question: do you have a data destruction policy in place? I don't mean a policy avoiding data destruction. I mean a policy specifying what data is actively destroyed when, by whom, and how. Are you actively destroying - not merely throwing out, but removing from existence - data? If you're not, you should be.

Danger, Will Robinson! Danger!

Of course, many individuals and companies are prohibited by law from destroying some data. For instance, the US Securities and Exchange Commission has set in place regulations requiring all firms that handle investments to keep all business records for three years. This includes client transactions, monetary records, and emails. Yes, emails. Failure to keep email records has in fact led the SEC to fine some firms millions of dollars.

The Internal Revenue Service recommends that companies and individuals keep tax records for three years after filing a return. The Occupational Safety and Health Administration requires the employers keep some records about the health of employees for 30 years after they leave their jobs. And if you or a client is in the midst of an investigation or a lawsuit, it's always a very bad idea to start shredding records - right, Oliver North? Right, Arthur Andersen?

So I am not suggesting that you start getting rid of everything willy-nilly, in a paroxysm of nihilism. Not only could you very easily cross legal and ethical boundaries, but you may get rid of something important or even vital. However, you should understand that legally, data destruction can benefit you, especially if there is even the hint of a lawsuit. I am certainly not a lawyer, (and if you're not either, you really must consult with a lawyer before putting any policy in place) but it's my understanding that certain standards of law have developed over the years that you need to be aware of, such as:

"If you throw something out or give it away, you have just revoked any legal right to privacy for that object.

If you're sued, and the folks suing you want to look at your computers for deleted files, you have to provide them with access to those computers.

If you're sued and the folks suing you want electronic data, you can't instead provide with stacks of printouts. If they want something in its electronic form, and you have it in electronic form, you'd better give it to them in electronic form.

If you're sued, and you know about the existence of electronic documents, you must reveal information about them.

If you're sued, suddenly instantiating a policy for data destruction will not be seen favourably by the court. No, not at all.

If you're sued, the existence of a haphazard, sloppy policy for data retention and destruction may result in a painful and costly penalty.

If you're sued, the existence of a consistent, reasonable, and enforced policy for data destruction can shield you from legal liability; in other words, you can legally justify a failure to produce documents if you really don't have them and in fact got rid of them according to policy."

Again, I'm not a lawyer. Before doing anything pertaining to destroying data, consult with your organisation's lawyers so that everyone does the right thing.

Crush, Kill, Destroy

When it comes to deciding what data you should jettison, I'm going to have to leave that up to you and your legal counsel (did I mention that I'm not a lawyer?). Basically, if you're not legally obligated to keep it, and there's no real technical reason, I'd encourage you to get rid of it. Erase it, and make sure that you remove the backups as well. If you're really going to destroy data, then make sure you get all of it, which of course leads to another important consideration: where is the data we want to get rid of stored?

One of the most obvious locations of data that should be removed is the hard drive. Unfortunately, this is also where a lot of mistakes are made. Simson Garfinkel recently published a fascinating and revealing article titled "Remembrance of Data Passed: A Study of Disk Sanitization Practices" (882 kb PDF) that every security pro should read. Garfinkel and his associate acquired more than 150 hard drives from eBay and offline sales, many of which were supposed "erased". They were able to recover data from 64 per cent of the drives. Some of the data they recovered included:

• 3,722 credit card numbers
• Bank account numbers, access dates, account balances, and even ATM software from a hard drive used in an ATM machine in Illinois
• Memos about corporate personnel issues
• Email messages
• Pornography

Garfinkel's larger point in his article is that merely formatting or fdisking a drive really does very little (in fact, fdisk only overwrites about 0.01 per cent of the drive's sectors). Data is still easily recoverable. To really erase the contents of a drive, sanitization software is required. Fortunately, there is some very good, very inexpensive (or even free) software available that can take care of this problem, no matter if you're using Windows, Mac OS, or *nix. Garfinkel recommends some software, but you can easily find other lists elsewhere.

Keep in mind that the software above will truly erase a drive, but still leave it usable. If you really want to make sure that data from a drive can't be recovered, and you want to work out your frustrations, then get a ball-peen hammer and beat the hell out of that drive until it's in pieces. Ain't no recovering that data.

Of course, you need to worry about more than just hard drives. What about USB drives? Palms and Pocket PCs? Cell phones? Laptops used by employees? Backup tapes, CDs, and DVDs? Routers? All these devices have the ability to store data on them that you may want to wipe. Heck, even the Furby - a talking doll - was a problem for the NSA a few years ago. All of these devices need to be covered by your policy, since any of them could be requested in a lawsuit.

Never fear, Smith is here

The items I mentioned above are relatively easy to handle because destroying data usually means wiping the device clean. It gets much more complicated when you need to keep some data while getting rid of other bits of information. For instance, what if you want to delete mail logs or spools while keeping the rest of the server up and running? Do you know what log files are being generated? Can you safely delete them?

And here's another thing that many people forget about: what about destroying data inside files? I'm talking, of course, about the famous problem of Microsoft Word's ability to keep deleted information behind in a file, which has caused embarrassment and worse for the British government. Word is not the only program with this ability - digital imaging software has also caused embarrassment for some individuals, and Adobe Acrobat does not always hide things as well as some users think.

Recently, Michal Zalewski wrote an informative piece titled "Strike that out, Sam" in which he reported the results of an interesting experiment. Zalewski wrote a spider that searched microsoft.com for Word documents and then grabbed the 10,000 or so that it found. He then found the documents that had used change tracking and still had deleted text left behind, which narrowed his set down to 500. Then he started looking at the Word documents using software that shows the complete text of the documents, deletions and all. He was pretty nice about his results, posting fairly innocuous examples, but even those don't exactly paint the company in a favourable light.

As Zalewski points out, his findings are particularly ironic considering that Microsoft recently released a tool to take care of this very problem. Too bad that it only works for documents created using Office XP and Office 2003, and only on Windows XP - maybe the recent embarrassment caused by Zalewski's article will encourage Microsoft to release a tool that would be useful for all the folks still using prior versions of Office and Windows.

One final thought: I've been focusing a lot on electronic data, but as security pro's, we need to think about non-electronic data as well. Getting rid of stuff we don't need also means shredding the pounds and pounds of paper that every office has in it (and then recycling that now-useless stuff, if you can). Too often we focus on computers, because that is what many of us use every day, but there is still danger lurking in printouts, and we would be remiss to forget them.

It's time for us to accept that we live in an environment with a great many dangers lurking in it, dangers that we can lessen as long as we create a policy that everyone in our organisation can understand and actually use, as long as we discipline ourselves to delete the stuff we just don't need, and as long as we remember to look both ways before crossing the railroad tracks - or the lawyers.

Copyright © 2004,

Scott Granneman is a senior consultant for Bryan Consulting Inc. in St. Louis. He specializes in Internet Services and developing Web applications for corporate, educational, and institutional clients.