Languages don't breed bugs, PEOPLE breed bugs, say boffins

You say C++, I say Python, you say JavaScript, I say Erlang ...

If you want to spark a religious war, express an unshakeable preference for a programming language, and by preference, make your favourite something relatively obscure, like Erlang. It turns out, according to a study by a bunch of UC Davis boffins, the differences in code quality between languages are pretty small.

To be presented at the ACM's Foundations of Software Engineering (FSE) symposium in Hong Kong in mid-November, the paper, A Large Scale Study of Programming Languages and Code Quality in Github, even analysing 729 projects, 80 million lines of code, 1.5 million commits from 29,000 or so authors in 17 languages only found “modest” effects coming from the language software's written in.

While strong typing is a little better than weak typing, and in functional languages static typing is a little better than dynamic typing, and managed memory is better than unmanaged, the authors (Baishakhi Ray, Daryl Posnett, Vladimir Filkov, and Premkumar Devanbu of the UC Davis department of comp. sci) find that “modest effects arising from language design are overwhelmingly dominated by the process factors such as project size, team size, and commit size”.

Not only that, they suspect that even personality type (linked to a preference for different types of language) might be more important than programming language.

The languages ranged from C and C++ through to Ruby, JavaScript, Python, Perl and even Erlang, as shown in the screen-grab from the paper, below.

“Generic” programming errors, the researchers found, overwhelmingly dominate bugs – things like “incorrect error handling, type errors, typos, compilation errors, incorrect control-flow, and data initialisation errors”. Those accounted for more than 88 per cent of the bugs the researchers identified. The rest were made up of memory handling errors, concurrency bugs, and algorithmic mistakes.

C, C++, JavaScript, Objective-C, Php, and Python had slightly higher error rates. “The languages Clojure, Haskell, Ruby, Scala, and TypeScript, all have negative coefficients implying that these languages are less likely than the average to result in defect fixing commits.”

However, they state that while the effects they observed were large enough to be statistically significant, the effects are “quite small”. At best, they conclude, functional languages might be “better” than procedural languages.

Oh, and Erlang, which Vulture South mentioned at the start in a humours vein? It showed no bias to being “better” or “worse”.

The paper is here. ®


Biting the hand that feeds IT © 1998–2017