Fortify and the Java open review project

Is open source software better than you think?

Choosing a cloud hosting partner with confidence

Comment I got some flack recently for daring to suggest (or appearing to) that open source software (OSS) should be "fit for purpose" (here).

After all, since all those saintly OSS developers are working for nothing, why should we expect their software to work? Well, I can't imagine a company with any hope of staying in business using software that isn't "fit for purpose", OSS or not. But, luckily, as I personally believe in OSS, there is objective evidence that it really can be pretty good quality.

This comes from the Java Open Review, an open source project sponsored by Fortify Software which uses Fortify SCA tools and Findbugs to look for defects in software – as a service. It publishes aggregated statistics but has a "responsible disclosure policy", which means details of bugs found are fed back only to the authors.

The project has recently analysed some common Java packages of the sort used to build/support other applications, including Hibernate, Struts, Spring, and Tomcat. These did pretty well, as you might expect, averaging under 0.1 defects per kloc (kilo lines of code), as opposed to the expected 20-30 defects per kloc reported by Carnegie Mellon's Cylab Sustainable Computing Consortium (although one should be a little cautious comparing such studies as even the definition of a "kloc" could differ).

Java, the most popular OSS language by far, appears to be more reliable than C/C++ - which is not exactly news, but it is always good to actually confirm what is obvious.

And it's useful ammunition for developers wanting to exploit OSS in conservative companies, as it appears that OSS may contain at least an order of magnitude fewer bugs than commercial software. Although the sample sizes are rather small as yet, and you can probably find buggy OSS if you look, I am pretty impressed not by the absolute figures so much as by the OSS community supporting an open assessment of OSS quality – this bodes well for OSS quality generally, if the project excites interest in the community. Perhaps we should revisit his project in a few months and look at interest levels.

Now, can you imagine Microsoft, IBM, or BEA publishing their defect statistics in any useful way? I can't, but if I'm wrong, please tell me. To be fair they'd all have to do so, I suppose, in some sort of race where no one wanted to be first.

But in the meantime, this from the Colorado State University makes interesting reading. It suggests that defect rates in open source operating systems are comfortably lower than those in Windows, although those in Windows really aren't too bad. Unfortunately, only a beta version of Windows XP was available, so you'd expect its defect rates to be higher – which perhaps ought to worry users of Web 2.0 applications where beta software sometimes seems like the norm.

You can find the Java Open Review study here (registration needed). Some other quality nuggets from this project are that cross-site scripting is the most common vulnerability you should be considering these days, and that even if Java packages are pretty good, the code samples supplied with them often don't reflect good security practice – just one reason why basically good OSS code is often used in insecure ways by developers.

Oh, and while I'm talking about security and finding defects, one of my pet hates is employing ex-hackers as penetration testers. Penetration testing has its place as a sort of acceptance testing or threat assessment, but it is really too late in the lifecycle to find defects anyway.

But, if you employ ex hackers to do it, how do you know they've really reformed (do you really want to give them low-level access to commercially sensitive or personal data in your systems), and how do you know they're as good as they say they are? Well, now there's an analysis tool from Fortify called Tracer which looks at the executables being penetration tested and reports back on coverage etc. That could sort out the sheep from the goats! ®

Business security measures using SSL

More from The Register

next story
'Windows 9' LEAK: Microsoft's playing catchup with Linux
Multiple desktops and live tiles in restored Start button star in new vids
Not appy with your Chromebook? Well now it can run Android apps
Google offers beta of tricky OS-inside-OS tech
New 'Cosmos' browser surfs the net by TXT alone
No data plan? No WiFi? No worries ... except sluggish download speed
Greater dev access to iOS 8 will put us AT RISK from HACKERS
Knocking holes in Apple's walled garden could backfire, says securo-chap
NHS grows a NoSQL backbone and rips out its Oracle Spine
Open source? In the government? Ha ha! What, wait ...?
Google extends app refund window to two hours
You now have 120 minutes to finish that game instead of 15
Intel: Hey, enterprises, drop everything and DO HADOOP
Big Data analytics projected to run on more servers than any other app
iOS 8 release: WebGL now runs everywhere. Hurrah for 3D graphics!
HTML 5's pretty neat ... when your browser supports it
prev story


Providing a secure and efficient Helpdesk
A single remote control platform for user support is be key to providing an efficient helpdesk. Retain full control over the way in which screen and keystroke data is transmitted.
Saudi Petroleum chooses Tegile storage solution
A storage solution that addresses company growth and performance for business-critical applications of caseware archive and search along with other key operational systems.
Security and trust: The backbone of doing business over the internet
Explores the current state of website security and the contributions Symantec is making to help organizations protect critical data and build trust with customers.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.