Feeds

Fortify and the Java open review project

Is open source software better than you think?

Business security measures using SSL

Comment I got some flack recently for daring to suggest (or appearing to) that open source software (OSS) should be "fit for purpose" (here).

After all, since all those saintly OSS developers are working for nothing, why should we expect their software to work? Well, I can't imagine a company with any hope of staying in business using software that isn't "fit for purpose", OSS or not. But, luckily, as I personally believe in OSS, there is objective evidence that it really can be pretty good quality.

This comes from the Java Open Review, an open source project sponsored by Fortify Software which uses Fortify SCA tools and Findbugs to look for defects in software – as a service. It publishes aggregated statistics but has a "responsible disclosure policy", which means details of bugs found are fed back only to the authors.

The project has recently analysed some common Java packages of the sort used to build/support other applications, including Hibernate, Struts, Spring, and Tomcat. These did pretty well, as you might expect, averaging under 0.1 defects per kloc (kilo lines of code), as opposed to the expected 20-30 defects per kloc reported by Carnegie Mellon's Cylab Sustainable Computing Consortium (although one should be a little cautious comparing such studies as even the definition of a "kloc" could differ).

Java, the most popular OSS language by far, appears to be more reliable than C/C++ - which is not exactly news, but it is always good to actually confirm what is obvious.

And it's useful ammunition for developers wanting to exploit OSS in conservative companies, as it appears that OSS may contain at least an order of magnitude fewer bugs than commercial software. Although the sample sizes are rather small as yet, and you can probably find buggy OSS if you look, I am pretty impressed not by the absolute figures so much as by the OSS community supporting an open assessment of OSS quality – this bodes well for OSS quality generally, if the project excites interest in the community. Perhaps we should revisit his project in a few months and look at interest levels.

Now, can you imagine Microsoft, IBM, or BEA publishing their defect statistics in any useful way? I can't, but if I'm wrong, please tell me. To be fair they'd all have to do so, I suppose, in some sort of race where no one wanted to be first.

But in the meantime, this from the Colorado State University makes interesting reading. It suggests that defect rates in open source operating systems are comfortably lower than those in Windows, although those in Windows really aren't too bad. Unfortunately, only a beta version of Windows XP was available, so you'd expect its defect rates to be higher – which perhaps ought to worry users of Web 2.0 applications where beta software sometimes seems like the norm.

You can find the Java Open Review study here (registration needed). Some other quality nuggets from this project are that cross-site scripting is the most common vulnerability you should be considering these days, and that even if Java packages are pretty good, the code samples supplied with them often don't reflect good security practice – just one reason why basically good OSS code is often used in insecure ways by developers.

Oh, and while I'm talking about security and finding defects, one of my pet hates is employing ex-hackers as penetration testers. Penetration testing has its place as a sort of acceptance testing or threat assessment, but it is really too late in the lifecycle to find defects anyway.

But, if you employ ex hackers to do it, how do you know they've really reformed (do you really want to give them low-level access to commercially sensitive or personal data in your systems), and how do you know they're as good as they say they are? Well, now there's an analysis tool from Fortify called Tracer which looks at the executables being penetration tested and reports back on coverage etc. That could sort out the sheep from the goats! ®

New hybrid storage solutions

More from The Register

next story
New 'Cosmos' browser surfs the net by TXT alone
No data plan? No WiFi? No worries ... except sluggish download speed
'Windows 9' LEAK: Microsoft's playing catchup with Linux
Multiple desktops and live tiles in restored Start button star in new vids
iOS 8 release: WebGL now runs everywhere. Hurrah for 3D graphics!
HTML 5's pretty neat ... when your browser supports it
Mathematica hits the Web
Wolfram embraces the cloud, promies private cloud cut of its number-cruncher
Google extends app refund window to two hours
You now have 120 minutes to finish that game instead of 15
Intel: Hey, enterprises, drop everything and DO HADOOP
Big Data analytics projected to run on more servers than any other app
Mozilla shutters Labs, tells nobody it's been dead for five months
Staffer's blog reveals all as projects languish on GitHub
SUSE Linux owner Attachmate gobbled by Micro Focus for $2.3bn
Merger will lead to mainframe and COBOL powerhouse
iOS 8 Healthkit gets a bug SO Apple KILLS it. That's real healthcare!
Not fit for purpose on day of launch, says Cupertino
prev story

Whitepapers

Providing a secure and efficient Helpdesk
A single remote control platform for user support is be key to providing an efficient helpdesk. Retain full control over the way in which screen and keystroke data is transmitted.
WIN a very cool portable ZX Spectrum
Win a one-off portable Spectrum built by legendary hardware hacker Ben Heck
Storage capacity and performance optimization at Mizuno USA
Mizuno USA turn to Tegile storage technology to solve both their SAN and backup issues.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
Security and trust: The backbone of doing business over the internet
Explores the current state of website security and the contributions Symantec is making to help organizations protect critical data and build trust with customers.