Original URL: http://www.theregister.co.uk/2008/09/19/educating_stob_ieee_enquiry/

Stob latest: IEEE flags dodgy paper

Cleaning up after peer review

By Andrew Orlowski

Posted in Developer, 19th September 2008 14:15 GMT

For the past few weeks I've been trying to piece together an explanation for Verity Stob's extraordinary adventure in academia, published here on Monday. Why did the Open University set as a marked assessment for postgraduate students a plagiarized piece of garbage, then admit they hadn't really read it? Why had a prestigious peer-reviewed journal at the IEEE published it in the first place - when to anyone with a knowledge of the subject, it made no sense at all?

These were just two of the questions raised by Verity's story. However, many more arose, as I found myself being sent from pillar to post. Repeated emails and phone calls produced the response that everything was fine. The institutional operating manual was recited back to me, as if this answered all possible questions. Both institutions had clammed up - no one wanted to admit there was anything amiss. I'll shortly tell the full saga - but a minor breakthrough came on Wednesday last night.

To recap: A postgraduate course at the Open University, M885, required students to answer questions on a paper published in the IEEE's Software journal in 2004. The paper, Open Source Reuse in Commercial Firms by T R Madanmohan and Rahul De', was part of a teacher-marked assessment (TMA). Verity queried the paper - but was told to carry on as if nothing was amiss. Then, after discovering compelling evidence it was plagiarized, Verity informed the IEEE.

The current editor of the journal, Hakan Erdogmus (who took up the post in 2007, long after the original atrocity was committed), instigated a plagiarism investigation. The conclusion was that the paper drew on earlier work and failed to credit it. Yet it was deemed only a "borderline Type 4" transgression, not serious enough to withdraw the paper or flag it as a suspect work. Erdogmus remained concerned, and set off another investigation into the observation that the authors also drew on earlier work and passed it off as their own. This concluded that there was nevertheless enough material in the paper - and again, it was left intact, with no indication that concerns had been raised.

(Note: The IEEE's investigations did not examine whether the paper made any sense at all - or whether bizarre statements such as the one that "artificial intelligence" could be used to locate software components were defensible. Because these can't be made "objectively", according to the IEEE procedures, they're not made at all. The computer says no.)

Software's current editor, Hakan Erdogmus - the only person to emerge with any credit in this saga - referred our questions to the board. The chair of the plagiarism investigation failed to respond. Finally, a breakthrough.

After a series of emails with IEEE board member and the institute's director of products and services, Evan Butterworth, we received this statement:

First, after a review of IEEE publication policies and procedures, we've determined that it would be appropriate to flag the article in our database. We are currently addressing that, and the flag should appear the next time we do a download of articles into the database - some time within the next 5 to 10 days. The flag will clearly alert researchers that the content of the article may not be reliable. I've appended the full text of the flag below my signature.

Second, it is our practice to regularly review and revise our policy manual, and we intend to consider what we have learned from this situation as part of that process.

Again, thank you for your diligence in bringing this situation more broadly to our attention. The IEEE is committed to ensuring the integrity of our scholarly publishing process, and the soundness of our research database.

And here's the new text appended to the Madanmohan and De' paper:

Notice of Violation of IEEE Publication Principles

“Open Source Reuse in Commercial Firms” by T.R. Madanmohan and Rahul De in IEEE Software, Vol. 21, Issue 6, November/December 2004, pp. 62-69

After careful and considered review of the content and authorship of this paper by a duly constituted expert committee, this paper has been found to be in violation of IEEE's Publication Principles.

This paper contains portions of original text from the sources cited below. Text from Paper 1) was reused without attribution. Text from Paper 2) was reused with attribution but without being clearly delineated from the above authors’ own text.

1) "Open-Source CMS: Prohibitively Fractured?" by Tony Byrne in CMS Watch, http://www.cmswatch.com/Feature/89-Open-Src 14 May 2003, pp 94-99

2) “The Amos Project: An Approach to Reusing Open Source Code” by Manuel Carro, Germán Puebla and Carlo Daffara in Presente y futuro de la ingeniería del software libre URJC, Madrid, May 21, 2003

As Verity noted, the paper continues to be widely cited.

Stay tuned - there's much more to follow. ®