The art of software testing
A look at the 2nd edition of a classic
Book review I've had a soft spot for Myers' book ever since it first gave me an insight into the economics of testing some 25 years ago. Its essential thesis is that you can't test everything, in practice; so you have to allocate the limited resources you always have to maximise the defects found per euro. There's no point in wasting resources running a test unless it finds a defect - yet how many programmers still feel good about running a test suite which doesn't turn up any defects at all? What chance that the program being tested really is defect free?
I also remember the response of my VP when I suggested buying a copy for our programmers. "Be realistic, David," he said, more or less. "These guys are flat out firefighting production bugs at 3am at the same time as they're getting software out of the door as fast as possible, we can't possibly expect them to read books on testing too".
Myers starts with a self-assessment test, which hammers home the difficulty of exhaustive testing. He then discusses the psychology and economics of testing and the importance of non-computer-based testing (inspections, walkthroughs and reviews). There are chapters on test-case design, unit testing, higher-order testing (Function, System, Acceptance testing etc.) and debugging.
This book was originally published in 1979 (and has remained in print since then, a tribute to its quality). The 2nd Edition was revised in 2004 and the rather outdated chapter on test tools replaced with chapters on eXtreme Testing and testing Internet applications. There's a practical example of eXtreme testing and a paragraph on the JUnit tool, which is used in the example.
The authors do say that much of the discussion in the earlier chapters applies equally to testing internet applications, but they make a reasonable case for covering them specifically (users are more critical of the user experience now, as the competition is merely a click away; and you have much less control over the operational environment generally). It is hard to compare old and new in detail when only paper is available but there are sufficient differences in the text of the earlier chapters, even when the meaning of the content hasn't changed, to give some confidence in the revision. The new chapters are more code-oriented than the original but the revised edition maintains a reasonably coherent feel.
This book is still a useful guide to what lies behind any software testing program: how best to allocate resources to testing, what you are trying to achieve and how you decide that you've finished. It has been bought up-to-date with eXtreme Programming etc. but it doesn't specifically tell you how to exploit the latest automated testing suites in detail.
However, their vendors will do that well enough and Myers' book gives you context, with which to uphold your end of the conversation. My only concern is that it is a bit expensive, even at discount, and in my experience, managers are loath to buy their teams books on testing. Nevertheless, buying test-automation shelfware and putting defect-ridden software into production isn't exactly cheap either.
The Art of Software Testing
Verdict: This revised edition still isn't as "modern" as it could be, but it remains an excellent guide to the fundamentals of software testing - without which test automation tools won't help you much. Should be on every programmers bookshelf, and it might be a good idea if they read it too.
Author: Myers, Glenford J; revised and updated by Sandler, Corey; Badgett, Tom; Thomas, Todd M
List Price: £57.50
Reg price: £46.00