Why Agile is like flossing and regular sex
The difference between doing it and saying you're doing it
After roughly 20 years, agile software development has wheedled its way into most every developer's mind as The Way Good Software Is Done. Like flossing, while we can all agree agile is a good idea, we're not quite up to snuff on keeping all our teeth in our heads, so to speak.
A recent Gartner survey [registration required] had 37 per cent of respondents saying they were doing agile, while 45 per cent preferred to float along with the traditional "waterfall" approach (the remaining said they were doing "lean," "iterative," or the always delightful "other"). While this isn't world domination, a 2015 report put waterfall at 56 per cent.
Survey data like this can be dicey, and it's best to treat them more like a wet finger in the wind than as rigorous science. That said, the wind seems to be blowing in agile's direction.
"I think from a tactics perspective, Agile is increasingly a 'solved problem'," said Forrester's Jeffrey Hammond when asked about agile adoption in the industry.
"We know many practices that work, and that have been well proven in the field," he added.
Proven as those techniques may be, once again, loose meatware is catching in the gears of progress. As Jeffrey adds, "from an adoption standpoint, Agile is a 'work in progress' mainly because Agile is as much about cultural transformation as it is tactics." Cultural transformation: it'll get you every time.
Indeed, looking back at that 2016 survey, you see that while easier practices like unit testing are widely practiced, onerous practices like continuous delivery and pair programming are mostly ignored by the buffet agilists. Agile is taking its time along the innovation curve, but one gets the feeling that the down-slope folks are methodically being routed, if only through the slow-but-steady siege tactic of retirement.
Beyond The War of the Story Cards
Early on in agile, there were some vicious battles around defining The One True Agile, especially as Scrum rose in popularity. For the most part, these battles were case studies of the narcissism of small differences, though mentioning "agile in the large" practices like SAFe can still be relied on to pop a true believing agilista's neck veins.
Several schools have descended from agile, primarily in the form of "lean" and "DevOps." In practice, these schools should be thought of as types of agile – at most, extensions – rather than so philosophically different as to be called distinct. They're nothing to start a holy war over.
"Lean," as its name would imply, comes from Lean Manufacturing, the continuous learning and "waste" removal process invented and perfected by Toyota. When you put lean Software Development practices in front of Lean Manufacturing people, they react similarly to how I imagine the French would react when presented a baguette à la Piggly Wiggly. The same idea is there, but it's been transmogrified by a new set of hands. In practice, lean software development focuses on putting a small batch approach in place, releasing software as frequently as possible to limit work in progress, and using an end-to-end mindset to discover and eliminate waste.
The idea of "waste" is perhaps the most intriguing and novel part of lean software development: unless an activity adds value for the customer, it's eliminated (más o menos). Once you start asking: "Will this help the customer?" many of the tasks in the so-called LSDC melt away like chicken fat under high heat. A notable variant of lean is the Lean Startup method, which seeks to discover the market/product fit for any given piece of software. This is the school of "pivots" where you continually evolve your software, observing how it's used until you figure out something that people will pay for, or at least use.
Enter the two pizza team
El Reg commenters' favorite, "DevOps," is the last descendant of agile, building from lean along with a dollop of its own special sauce. A canon DevOps isn't a fully "solved problem" yet, but we're getting close. Originally, what became known as DevOps was solving for two things: enabling frequent deployment of software (many times a day) and maximum uptime. For most people, the idea of deploying software daily is the gut-proven opposite of "maximum uptime." However, the disciplined, small batch mentality practices of DevOps have been showing that the two can be done in tandem.
One of the key components DevOps adds to agile is the idea of a small, cross-functional team rather than "silo'ed" teams. Agile-minded developers land-grabbed QA early on, but mostly stopped there. In contrast, DevOps looks to gobble all the roles, from developers, to operations, to designers, product managers, and whatever other role is needed to make sure you can ship useful features frequently and keep the stuff up and running.
Though there are numerous roles, the teams are small. As Ben Terrett, former head of the UK Government Digital Service, put it: "[T]he best way to do this stuff is to get a multi-disciplinary team of people in house – designer, user researcher, developer, content person – you're talking a team of about twelve people." In Amazon terms: no team should be so large that it needs more than two pizzas for dinner.
Admit it: you have no idea what's going on
Recently, I've been asked several times to address when waterfall is a good choice. The answer to that question is good insight into agile's trick. Waterfall is fantastic if you know exactly what you want to build, up front. What a wonderful project that'd be to work on!
As most of the people who develop software find, however, very few people know exactly what needs to be built ahead of time, least of all the actual users and customers.
Agile, instead, builds its thinking and processes around the assumption that we can't know what the software should be until we start deploying it to actual users. Doing so isn't easy, and while adoption has been slower than you'd expect, as was said long ago: if you are made to wait, it is to serve you better, and to please you. ®