CMMI, practically speaking

London was recently host to a conference showcasing CMMI process improvement. Primarily sponsored by the consultancy Lamri, the conference was held on 19-20 March.

To some extent, it's a marketing exercise, but the presence of representatives from the Software Engineering Institute (SEI) at Carnegie Mellon and truly independent consultants such as Marilyn Bush and others, means it provides real information content.

However, Andrew Griffiths (MD of Lamri)gave a keynote presentation entitled "Buying the right to Change" in which he pointed out that the key skills for transforming an organisation (which is the essence of CMMI process improvement) are, in essence, sales and marketing skills – and he really does have a point there...

This conference is also an excellent opportunity for networking. Over the conference dinner, for example, I had the opportunity to talk about pressures for CMMI adoption with a "process person" from an aerospace firm that's revisiting CMMI because it's becoming a "filter" in procurement. In his field, only firms appraised at CMMI Maturity Level 3 may be in with a chance soon.

Of course, this situation raises the spectre of "check-box compliance" with the letter rather than the spirit of CMMI, but my dinner companion agreed that this sort of approach has all the overheads of CMMI adoption but delivers few of the strategic benefits (apart from, possibly, the tactical benefit of staying in business for the time being).

Jay Douglass from the SEI reported on what is new in CMMI 1.2 and then gave us an introduction to the latest CMMI models: CMMI for Acquisition and CMMI for Services (CMMI-ACQ and CMMI-SVC respectively). Douglass listed the three main drivers for v 1.2 as:

1. Reducing the complexity of CMMI in use
2. Increasing coverage
3. Increasing confidence in appraisals

Points one and two are straightforward improvements based on practical experience with CMMI, but the third is probably more significant, to my mind (and addresses the "check box" issue which concerned my dinner companion). It used to be possible to "cheat" with CMMI for marketing purposes; to appraise one part of an organisation and claim a maturity level for the whole of it; to select lead appraisers with little real experience of maturity level four and five organisations; and to base maturity claims on wildly out of date appraisals.

Since the appraisal process is central to CMMI, the SEI has moved to address any such issues: appraisal disclosure has been tightened up, special training is mandated for high-maturity appraisers, and appraisals now have a "shelf life" (three years). The SEI has also instigated appraisal audits for some maturity level four and five appraisals (unusually rapid progress through the maturity levels, for example, might attract audit). CMMI v 1.2 finally replaces v 1.1 around August 2007.

CMMI-ACQ helps deal with the possible maturity mismatch between organisations building software and the organisations acquiring it from them; and CMMI-SVC deals with IT service delivery. Both initiatives are based on existing work in the CMMI community and CMMI-ACQ is just about complete and should be delivered shortly (June 2007).

CMMI-SVC will then be delivered (it's currently at "final comment" stage and scheduled for March 2008 delivery; but may well arrive sooner). Douglass wasn't really able to talk about CMMI-SVC in detail, but I remain to be convinced as to how well a project-based framework such as CMMI will support and complement ITIL, which is task-based, and which I see as the most practical approach to service delivery available today. If developers want to design systems for operational manageability, resilience, and service delivery (as they should), it's ITIL that provides a guide to the requirements.

Marilyn Bush represents the CMMI community in general rather than the SEI itself, although she's worked for the SEI. She is an extremely experienced CMMI practitioner and mentor (and one of that select band of IT people who've worked in the space industry and can be called a "rocket scientist" with some credibility).

She gave a presentation on the "Good, Bad and the Ugly" in the latest incarnation of CMMI. It's healthy that practitioners can constructively criticise the framework - and healthy that the CMMI community tolerates this. Despite the fears of some Agile practitioners, most CMMI process gurus certainly don't underestimate the people aspects of CMMI process improvement and this extends to encouraging debate about CMMI itself.

Her concerns with CMMI include:

• It sometimes oversells the "continuous representation" of CMMI, which allows you to pick and choose what high maturity processes you adopt – but only up to a point, as process areas are often dependent on each other.
• It hides the importance of active involvement by senior management – more than just "sponsorship" is needed.
• It implicitly assumes the importance of early defect removal without explicitly stating it.
• It over-emphasises testing as opposed to early defect removal; you should be putting a high-quality product into the testing process.
• The (misuse) of appraisal as an audit process, with consequent production of documents solely for appraisal.