Open and closed source software defects reloaded

Rhyme and Reasoning

Open source and closed source software packages start out with roughly equivalent defect rates even though code inspection gives OS packages an edge over time.

That's the conclusion of a pair of studies by software tools vendor Reasoning.

In the first study, published in February, 2003, Reasoning found that the TCP/IP protocol stack implementation in version 2.4.19 of the open source Linux kernel has fewer defects than the protocol stacks of several commercial equivalents.

Reasoning found eight defects in 81,852 lines of Linux kernel source code - the fewest number of defects of the various implementations of TCP/IP inspected by the company as part of its study.

In a second study, published yesterday, Reasoning found the software quality of Apache v2.1 roughly on par with commercial Web server equivalents.

The defect density of the Apache code inspected was 0.53 per thousand lines of source code, while the commercial average defect density came to 0.51 per thousand lines of source code. In all 31 software defects were found in 58,944 lines of Apache v2.1.

Apache v2.1 is a less mature open source package than the Linux kernel reviewed in Reasoning's February study. Reasoning uses this factor to explain the discrepancy in findings between its two studies.

Reasoning concludes from its studies that "there is a correlation between code inspection/peer review and the resulting defect density".

Which is what common sense would have suggested in the first place.

Reasoning studies were derived from its automated software inspection services and used a "combination of proprietary technology and repeatable process". The company said that the results from its code inspection services are "objective and comparable across software applications, development methodologies, and coding styles".

The results of Reasoning's Apache V2.1 inspection can be obtained here. ®

Related Stories

Open and closed security are roughly equivalent
.NET 'more secure' than WebSphere
All bugs are created equal
Want to know the ten most critical web app vulnerabilities?

Sponsored: Customer Identity and Access Management