Researcher attempts to shed light on security troll

Moreover, Krawetz's forensics analysis does not stand up well, said Carole Chaski, a forensic linguist and principal researcher at the Institute for Linguistic Evidence, Inc. Chaski makes a living out of researching ways of identifying authors and has analysed suicide notes and threatening email in criminal cases to determine authorship.

She stressed that techniques such as measuring vocabulary, spelling errors and grammar errors are not good methods of identifying a document's author.

"This whitepaper floats some of these erroneous ideas which have already been shown by research, which the whitepaper is apparently unaware of, to be fairly unreliable for authorship analysis," Chaski said in an email interview.

Chaski also took to task the paper's assertion that, if the errors in the methods were consistent in either being right or wrong, then the accuracy of the identification is improved.

"There is an old expression 'two wrongs do not make a right' which applies here," Chaski said. "If one method is only 60 per cent accurate and it is combined with another method - for the nonce let us assume an independent method - which is only 60 per cent accurate, than the combined method has an accuracy rate of 36 per cent, far below chance. One would be better off guessing than combining unreliable methods."

Security consultant Krawetz started researching the identity of n3td3v after another Full Disclosure subscriber challenged him to use linguistic forensics - a topic on which Krawetz presented at this year's Black Hat Security Briefings - to uncover the person's identity. He started with the hypothesis that n3td3v and Gobbles were the same and used methods of gender determination as well as analysing lexicon, word frequency, punctuation frequency and preferred sentence length to create a profile of the two groups.

The analysis seemed to indicate that n3td3v had three members and Gobbles had three or four main members. The characteristics of three members of each group seemed to match up.

Krawetz acknowledges that some aspects of his analysis could have significant error. For example, this is the first time that he attempted to analyse documents to match up three different authors, a research problem that ILE's Chaski believes to be untenable.

Krawetz hopes that, if his analysis is shown to be in error, it will ultimately benefit his research.

"Very few people are using these techniques just because they are new technologies," he said. "On the other hand, if someone comes out and shows me I'm wrong, it could help me tune my system."

This article originally appeared in Security Focus.

Copyright © 2006, SecurityFocus