Compuware's testing roundtable

Software quality is back on the board's agenda

Remote control for virtualized desktops

I’ve just been representing Reg Developer as one of only three journalists at an interesting roundtable discussion devoted to “Software Quality, Best Practice and Governance” (which seems to mean Testing, in the widest sense).

Among those present were Sarah Saltzman of Compuware, the sponsor of the roundtable event; Teresa Jones of Butler Group; Brian Wells, chair of something called the TMMi foundation (of which more later); Geoff Thompson, of Experimentus; and, representing the businesses using this stuff, Philip Griffiths, Global Applications Architect at Heinz. Most noticeable, by the way, was Sarah’s reluctance to talk about Compuware’s products too much – well done!

The discussion started off conventionally – but got more controversial later. The importance of early defect removal was stressed and the need for business resources to be made available, so that testers could relate the evolving system to the needs of the business as well as (or instead of) what the IT group said it was going to deliver. Clearly, it helps if the testers contribute to the project from inception; rather than being brought in at the end, as a "barrier to going live".

But I've heard all this before, at various times over the last 30 years or so. We even touched on the fact that if your project is behind schedule and testing is being squeezed, shipping in "testing mercenaries" on short term contracts probably puts you further behind schedule (someone on the team has to stop what they're doing and get these people up to speed). But this is one of Fred Brook's insights from 30 years or so ago. Why is it still relevant? And, since we really do know a lot about managing software quality why do we still accept bugs in software that is only marginally fit for purpose, sometimes just because it is popular (the Microsoft syndrome)?

And here’s where the discussion turned interesting. You won’t get blamed for buying popular (or well-marketed) software even if it isn't high quality, and bugs in features you don’t use don’t matter much, so perhaps we then come to accept bugs in features that do (after all, if popular software contains bugs, perhaps they don’t matter). Then, sometimes the problems are not in the software but the spec it was written to, so you can’t safely blame anyone (“the user is always right”), even though using software which automates the wrong process is seriously expensive and disruptive.

Once again, it’s the people/culture/change issues that bite. Phillip Griffiths pointed out that getting the resources for managing quality out of a management often preoccupied with short-term stock market performance isn’t easy (in the long term, quality is free, but only if you are prepared to invest in it up front in the shorter term). And people often have a vested interest in the status quo.

“Hero management” – by people who get a lot of fun (and, probably overtime and bonuses) out of waiting for things to go wrong and then fire-fighting with flair and enthusiasm - is rife. And it’s a very expensive approach to managing quality. It was even noted that a profitable testing market is evolving – full of people who’ve done a course on testing but who have little experience of “defect risk management” and who "don't know what they don't know" – but are paid high rates to help with last minute quality panics. And, often a system will go live even though people at the sharp end know it can’t work, because no-one is brave enough to tell management the bad news; and people believe that they can manage their way out of the mess anyway – hero management again.

Griffiths sees a need for a business model fundamentally based on “fitness for purpose” quality. The CEO must define a way of working that is process-oriented, not technology-focussed and must support the quality champions in middle management actually managing change. And s/he must allocate business resources to testing and analysis.

It all comes down to organisational maturity and process improvement, it seems to me. Without this, you can even “get it right” in response (usually) to some crisis – and then slip back into old ways. Geoff Thompson described a firm which had done just that. In a situation of extreme pressure with no time for rework, quality was built in from the start and the developers/testers were given the time they needed – on condition that that was all they got – and, on delivery, the product "just worked" without any problems. But now that things are less stressed, techniques like “pair testing” (with the testers embedded in development alongside programmers), that helped achieve this, are less popular.

Which is where TMMi comes in. It’s based on the TMM (Test Maturity Model) process model, from around 2003, which adds testing to the Capability Maturity Management model (CMM; now Capability Maturity Management Integration, CMMI); which doesn’t go into much detail on actually achieving demonstrable quality. According to Brian Wells, testing is what demonstrates Quality and it covers all forms of defect removal

Now, the TMMi foundation is taking the text book TMM model and "refactoring" it (simplifying it , adding stuff, moving stuff around - training is moving from level 3 to level 2, for example). The aim of this independent foundation is to produce a "generic but detailed enough" public domain reference model for the testing process – which (amongst other things) should help developers choose tools without the fear that their tool choice is being driven by a vendor agenda embodied in a proprietary process.

The TMMi model is non-prescriptive – it won’t mandate an independent test team, for example, but suggest that there is a need for independence in at least part of the testing process – and it is research driven, not commercial. The first draft of the new TMMi model should be available for review in 1Q 2007; delivery of version 1 is expected at the end of 2007 and I hope to look at it in more detail in Reg Developer on due course.

And Compuware? Well, it is looking at mapping TMMI onto the requirements-driven “risk based testing” process models it is developing for itself (see, for example, the Risks Are For The Weekend Executive Guide, at the bottom of the page, registration needed). That is something else we may look at in Reg Developer – and perhaps we’ll look at some of Sarah’s testing and requirements management tools too.

David Norfolk is the author of IT Governance, published by Thorogood. More details here.

Remote control for virtualized desktops

More from The Register

next story
PEAK APPLE: iOS 8 is least popular Cupertino mobile OS in all of HUMAN HISTORY
'Nerd release' finally staggers past 50 per cent adoption
Microsoft to bake Skype into IE, without plugins
Redmond thinks the Object Real-Time Communications API for WebRTC is ready to roll
Microsoft promises Windows 10 will mean two-factor auth for all
Sneak peek at security features Redmond's baking into new OS
Mozilla: Spidermonkey ATE Apple's JavaScriptCore, THRASHED Google V8
Moz man claims the win on rivals' own benchmarks
FTDI yanks chip-bricking driver from Windows Update, vows to fight on
Next driver to battle fake chips with 'non-invasive' methods
DEATH by PowerPoint: Microsoft warns of 0-day attack hidden in slides
Might put out patch in update, might chuck it out sooner
Ubuntu 14.10 tries pulling a Steve Ballmer on cloudy offerings
Oi, Windows, centOS and openSUSE – behave, we're all friends here
Was ist das? Eine neue Suse Linux Enterprise? Ausgezeichnet!
Version 12 first major-number Suse release since 2009
prev story


Cloud and hybrid-cloud data protection for VMware
Learn how quick and easy it is to configure backups and perform restores for VMware environments.
Forging a new future with identity relationship management
Learn about ForgeRock's next generation IRM platform and how it is designed to empower CEOS's and enterprises to engage with consumers.
High Performance for All
While HPC is not new, it has traditionally been seen as a specialist area – is it now geared up to meet more mainstream requirements?
New hybrid storage solutions
Tackling data challenges through emerging hybrid storage solutions that enable optimum database performance whilst managing costs and increasingly large data stores.
Security and trust: The backbone of doing business over the internet
Explores the current state of website security and the contributions Symantec is making to help organizations protect critical data and build trust with customers.