Compuware's testing roundtable

Software quality is back on the board's agenda

Providing a secure and efficient Helpdesk

I’ve just been representing Reg Developer as one of only three journalists at an interesting roundtable discussion devoted to “Software Quality, Best Practice and Governance” (which seems to mean Testing, in the widest sense).

Among those present were Sarah Saltzman of Compuware, the sponsor of the roundtable event; Teresa Jones of Butler Group; Brian Wells, chair of something called the TMMi foundation (of which more later); Geoff Thompson, of Experimentus; and, representing the businesses using this stuff, Philip Griffiths, Global Applications Architect at Heinz. Most noticeable, by the way, was Sarah’s reluctance to talk about Compuware’s products too much – well done!

The discussion started off conventionally – but got more controversial later. The importance of early defect removal was stressed and the need for business resources to be made available, so that testers could relate the evolving system to the needs of the business as well as (or instead of) what the IT group said it was going to deliver. Clearly, it helps if the testers contribute to the project from inception; rather than being brought in at the end, as a "barrier to going live".

But I've heard all this before, at various times over the last 30 years or so. We even touched on the fact that if your project is behind schedule and testing is being squeezed, shipping in "testing mercenaries" on short term contracts probably puts you further behind schedule (someone on the team has to stop what they're doing and get these people up to speed). But this is one of Fred Brook's insights from 30 years or so ago. Why is it still relevant? And, since we really do know a lot about managing software quality why do we still accept bugs in software that is only marginally fit for purpose, sometimes just because it is popular (the Microsoft syndrome)?

And here’s where the discussion turned interesting. You won’t get blamed for buying popular (or well-marketed) software even if it isn't high quality, and bugs in features you don’t use don’t matter much, so perhaps we then come to accept bugs in features that do (after all, if popular software contains bugs, perhaps they don’t matter). Then, sometimes the problems are not in the software but the spec it was written to, so you can’t safely blame anyone (“the user is always right”), even though using software which automates the wrong process is seriously expensive and disruptive.

Once again, it’s the people/culture/change issues that bite. Phillip Griffiths pointed out that getting the resources for managing quality out of a management often preoccupied with short-term stock market performance isn’t easy (in the long term, quality is free, but only if you are prepared to invest in it up front in the shorter term). And people often have a vested interest in the status quo.

“Hero management” – by people who get a lot of fun (and, probably overtime and bonuses) out of waiting for things to go wrong and then fire-fighting with flair and enthusiasm - is rife. And it’s a very expensive approach to managing quality. It was even noted that a profitable testing market is evolving – full of people who’ve done a course on testing but who have little experience of “defect risk management” and who "don't know what they don't know" – but are paid high rates to help with last minute quality panics. And, often a system will go live even though people at the sharp end know it can’t work, because no-one is brave enough to tell management the bad news; and people believe that they can manage their way out of the mess anyway – hero management again.

Griffiths sees a need for a business model fundamentally based on “fitness for purpose” quality. The CEO must define a way of working that is process-oriented, not technology-focussed and must support the quality champions in middle management actually managing change. And s/he must allocate business resources to testing and analysis.

It all comes down to organisational maturity and process improvement, it seems to me. Without this, you can even “get it right” in response (usually) to some crisis – and then slip back into old ways. Geoff Thompson described a firm which had done just that. In a situation of extreme pressure with no time for rework, quality was built in from the start and the developers/testers were given the time they needed – on condition that that was all they got – and, on delivery, the product "just worked" without any problems. But now that things are less stressed, techniques like “pair testing” (with the testers embedded in development alongside programmers), that helped achieve this, are less popular.

Which is where TMMi comes in. It’s based on the TMM (Test Maturity Model) process model, from around 2003, which adds testing to the Capability Maturity Management model (CMM; now Capability Maturity Management Integration, CMMI); which doesn’t go into much detail on actually achieving demonstrable quality. According to Brian Wells, testing is what demonstrates Quality and it covers all forms of defect removal

Now, the TMMi foundation is taking the text book TMM model and "refactoring" it (simplifying it , adding stuff, moving stuff around - training is moving from level 3 to level 2, for example). The aim of this independent foundation is to produce a "generic but detailed enough" public domain reference model for the testing process – which (amongst other things) should help developers choose tools without the fear that their tool choice is being driven by a vendor agenda embodied in a proprietary process.

The TMMi model is non-prescriptive – it won’t mandate an independent test team, for example, but suggest that there is a need for independence in at least part of the testing process – and it is research driven, not commercial. The first draft of the new TMMi model should be available for review in 1Q 2007; delivery of version 1 is expected at the end of 2007 and I hope to look at it in more detail in Reg Developer on due course.

And Compuware? Well, it is looking at mapping TMMI onto the requirements-driven “risk based testing” process models it is developing for itself (see, for example, the Risks Are For The Weekend Executive Guide, at the bottom of the page, registration needed). That is something else we may look at in Reg Developer – and perhaps we’ll look at some of Sarah’s testing and requirements management tools too.

David Norfolk is the author of IT Governance, published by Thorogood. More details here.

Internet Security Threat Report 2014

More from The Register

next story
UNIX greybeards threaten Debian fork over systemd plan
'Veteran Unix Admins' fear desktop emphasis is betraying open source
Netscape Navigator - the browser that started it all - turns 20
It was 20 years ago today, Marc Andreeesen taught the band to play
Redmond top man Satya Nadella: 'Microsoft LOVES Linux'
Open-source 'love' fairly runneth over at cloud event
Chrome 38's new HTML tag support makes fatties FIT and SKINNIER
First browser to protect networks' bandwith using official spec
Admins! Never mind POODLE, there're NEW OpenSSL bugs to splat
Four new patches for open-source crypto libraries
prev story


Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
Why and how to choose the right cloud vendor
The benefits of cloud-based storage in your processes. Eliminate onsite, disk-based backup and archiving in favor of cloud-based data protection.
Three 1TB solid state scorchers up for grabs
Big SSDs can be expensive but think big and think free because you could be the lucky winner of one of three 1TB Samsung SSD 840 EVO drives that we’re giving away worth over £300 apiece.
Reg Reader Research: SaaS based Email and Office Productivity Tools
Read this Reg reader report which provides advice and guidance for SMBs towards the use of SaaS based email and Office productivity tools.
Security for virtualized datacentres
Legacy security solutions are inefficient due to the architectural differences between physical and virtual environments.