My company announced internally that it was adopting an Agile methodolgy after having gone through several stages of CMMI and I got yet more forms to fill out for every state of the development cycle.

Sometimes managers hear great buzzwords but don't get beyond the first paragraph.

"Keep It Simple S****"

When the projects becomes too big, no matter the methodology, it will fail.

You need to keep the software projects small; that is the only way to make it work.

Instead of having a 500 people project, create 10 projects of 20 people and a tree of say 1+4 integration projects each with 2 or 3 people each.

You will get the project done faster and with less people.

And if any sub-project fails, you can replace the team of just that sub-project and still manage to get to an end.

KISS

"But what happens when there are more than 30 people on the project? Is it still possible to be agile when there are 50, 100 or 500 developers, analysts and managers involved? Is there a cut-off point where agile principles and methods start to become more trouble than they are worth? "

If you've read Kent Brock's book you would know that Agile is predicated on good communication and no barriers between team members, and small teams. If the task can be broken down into small enough pieces then it could still use agile methods, but you're going to keep hitting the buffers of each small team needing to spend a lot of time making sure it isn't doing something that will destroy stuff for the other small teams. Cockburn talks an awful lot about good sharing of information too. Very necessary and hard to do well even if the team only has 5 or 6 people in it.

You can be more agile than the "200 consultant waterfall it'll be ready in 2 years' time" methodology beloved of the big management consultancy firms (and we all know they lie anyway). You can apply the principles, but as soon as you need a lot of communication and compromise to keep things alive it will start to become difficult. Not impossible, but difficult. And you're much more likely to have Dilbert-zone management somewhere completely screwing things up by not understanding what they're supposed to be doing, which is keep out of the way and protect the team from trivial stuff.

You can call it agile even when its a mammoth floundering in a tar pit, but that doesn't make it so. Agile is left best to small projects imoh

Having just completed the first phase of an Agile project we were in a unique position to rate our progress against a rival team working on a system with exactly the same functionality as ours.

A team of three and a half people completed the full system for the September deadline where as our opponents with a team of 15 people were hoping to go live at the end of October.

It can't be that the three of us who were full-time on the project were faster and brighter than 5 each of our opponents (although it would be nice to think it was.)

It has to be that the speed and flexibility of regular scrums, 2 week sprints, and constant UAT really accelerated the development - it certainly felt like it did. There was also the good feeling that when the final UAT sign-off came you'd not have that call saying "this isn't what we wanted".

The key part was total buy-in from all the parties concerned, us in engineering were keen to give a proper agile project a fair crack, the junior members of the business team were excited by the possibility of the hand's on approach, and their seniors were nervous, but trusted everyone to do their best in a very short time scale.

But will it scale... For a team of 500 there's no chance, the sheer amount of "work stuff" that mires a large project will bog a team of 500 - but if you're controlling a workforce of that size you're certainly not thinking in an Agile way. It would need a strong re-grouping exercise to identify the key points and responsibilities. The reality of the project was that we were working in an Agile way while the people delivering the services we were using could have been working in a completely structured manner.

So I'd suggest that you don't have to attempt to scale the agile approach, as you are best served in some areas (such as the foundation APIs and critical services) to maintain a solid reliable and quality tested platform, but in other areas notably those which are closest to the business and the users Agility will pay dividends and the entire project will benefit from the positive results it brings.

I'm currently working on an Agile project for a well known company with probably about 30-40 people involved directly (i.e. developers, testers, BAs, project managers, product owners, etc). There were a lot of problems getting everything into place, and things aren't perfect, but it does run pretty smoothly now.

I'm not sure how this project could have been done without an iterative approach (we're using 2 weeks), as the business is constantly changing and refining what they want, and the techies are gaining a deeper understanding of the technical issues, including dependencies between some complex systems, not all of which can be changed.

The thing is, Agile can actually make it *easier* to work with bigger teams, because it helps you break things down into rational chunks. Altho admittedly getting to that point can be painful.

I think companies in certain industries who've decided not to go for Agile because of the short term pain of putting it in place might live to regret it in the long run.

Thing is, what happens when the subprojects become interrelated with or dependent on each other? And there is such a thing as necessary complexity--something which cannot be simplified any further without knocking out something essential to the project.

It works fine - its when you have more that 2 managers involved in a project that timescales go exponential.

Too many oxymorons to start here.

A disturbing aspect of agile development is its apparent spread towards end users, with work-in-progress routinely being shipped to hapless buyers of everything hitech. Agile is something that should have been kept secret from the marketing depts and PHB's everywhere. Speaking as a consumer being able to get new toys sooner really doesn't make up for the bloody things not working on delivery and sitting through months (sometimes years) of broken iterations.

Its a dangerous tool in the wrong hands and the lack of scalability is a blessing if it slows down its misuse by making real customers part of the dev team.

Good team - it will work usually, irrespective of methodology chosen (team will bend system to fit if required)

bad team - well creek sans paddle

It's not that hard to scale, providing you know what you are doing. I've done it again and again with multiple enterprise customers.

It's really all about sensible problem decomposition.

What Agile/Scrum/Whatever *won't* do is save you if you've inappropriately offshored/outsourced. If, for instance, you've stuck all your developers in India, your testers in Croatia and all your customers in the UK, you're going to get massive problems no matter what technique you use. "Agile" (Let's say Scrum for example) will point out that problem, loudly and clearly.

Whether you do something with that is up to you.

One last thing - Scaling is the last thing you should do in a project . Throwing people at a problem can make the project harder and slower to deliver, not easier/quicker. (We've all read things like "The mythical man month".) That's probably where some of this "Agile doesn't scale" nonsense comes from - In the past scaling was one of the first things you used to think of, now we understand that it should be one of the last.

A good book on this is Ken Schwaber's "The Enterprise and Scrum".

The term "Agile" has come to mean a wide range of different things. A lot of the practices commonly associated with agile development are intrinsically small scale. Large teams need more process (or ceremony as Alistair Cockburn puts it).

The basic principles, however, definitely scale. In particular, iterative and incremental development in which you always work on the highest value outstanding task and finish that task in its entirety (i.e. fully test it and fully integrate it with the rest of the software) before moving on to the next, is appropriate for a team of *any* size.

"moving the deadline up 3 weeks"

At least around here.

.......the big 5 will, and charge you an arm and a leg along with a huge change request process and a delivery date that is less believable than the story lines on East Enders.

Split your teams up, and run many small projects, each team can manage their dependencies as customers on the other teams.

Yes you still have communication problems, but they are constrained to what is important to each team, instead of 200 people trying to report ALL their problems at the SAME TIME

I would like to express my pleasure at seeing Joe Satriani name-checked within the article.

That is all.

To expect agile practices to scale is a little unrealistic. The greater the number of stakeholders, the more risk and expectation management is required, the more effort on communication is required, etc. 'Classic' prescriptive approaches become necessary to ensure alignment of the stakeholder community.

Agile development takes the adaptive approach: timebox development, get frequent releases and feedback, and release incremental updates. Few larger scale problems can be successfully decomposed to allow small teams the necessary independence to develop adaptively (where by definition one is not exactly sure what the final deliverable will be, but the focus is on ensuring it is fit-for-purpose).

So where's the cross-over? Well it's probably where senior management are sufficiently sanguine and trusting of the development team to do the right thing without imposing onerous reporting and task management. Personally, I not seen it successfully scale beyond 10 without starting to take more hybrid approaches (e.g. use Agile approaches for prototypes of key functionality and to mitigate largest risks. Use more traditional Prescriptive approaches for work to be outsourced to external development resources, ensure full test coverage, etc.)

Software developers need to be treated not like engineers but like authors, the software tells a story and the user interacts with the story.

Oddly enough, authors tends to get paid more than developers, though I would say it is probably harder to code something with the same number of words than write it.

Still, add more developers onto the project and watch the time line creep forwards, it was all written about in the Mythical Man Month, and yes it is very very true. The reason you have more developers on a project than one is generally political. It would be great to be proven wrong, but so far I cannot think of one project that worked better and was produced faster by the addition of people.

Try writing a book with many people, it won't be that good a read, and the ideas will tend to meander, same with software.

If you take ten mediocre developers, and 1 good dev, put them on the same project and pay the 1 good dev 9 times more than each of the mediocre devs, I know where I would put my money on who would complete first. If you also offered a % cut on the action of the software to both teams then the odds would move even further to the favourite.

I have seen Siamese developers out there; guys who will only work together - and hey it might work, but they have to think in the same way or really compliment each other's style.

Graphic designer and developer tends to work well. You might be able to swing a db admin and a developer combo, but again the developers tend to be better at producing the procedural SQL where I would expect the db guy to shine.

Another approach is the apprentice, sort of works, but a bit weird.

In the end the best approach is to hire a dev and give them the responsibility on the project and the cheque or bacs to match. If you need to be sure a project will be developed then take an elite team of people and start them off at different ends of the project, there you will normally have a kerfuffle at the join up point but if the mythical bus does appear then at least you can finish.

"agile development" is great when used by the developers. It has, as usual, come to mean something completely different when the term is used by the lying, scumbag marketing and sales weasels that infest the business.

So before someone says "this agile project worked" or "that one failed", I like to find out if the person defining the methodology is a technical person who knows what "agile development" actually means, or a salesrat manager who heard the buzzword once while fiddling little boys and now thinks they know everything about it.

Scott Ambler did a keynote address on this very topic, last March at SD West. Short answer: Yes, agile development scales up to enterprise projects, and his data were persuasive. Working for IBM, he's in a position to know about large.

I worked on internal software development at a major airline for over 10 years, and we were able to handle fairly complex projects such as complete application subsystem rewrites with only a few people ... and did so with a low defect rate and very fast project turnaround times.

The airline's software development group (at least in Flight Ops) was organized into small teams of subject matter experts, most of them with very deep knowledge of the rather specialized airline applications that were being used internally.

I think that is the key -- the more bodies you include in the process, the more overhead you tend to introduce.

By doing software development using a few expert programmers, especially of those programmers are doing applications support as well as development, you end up with people who are intimately familiar with the software. Up-front analysis is faster, estimates are more accurate, and the integration of new code is easier because the people doing the work are often the same people who wrote the code which already existed.

Quality might also be enhanced because those writing the code already know the area and also have a vested interest in having it work correctly -- if the code breaks, *they* are the ones getting a phone call at three in the morning!

Also, because one expert programmer can often work very quickly in their areas of expertise, it rarely becomes necessary to have more than two or three people involved in small- or medium-scale projects.

Most projects tend to have a single business analyst driving the requirements from the end user side, and a single programmer handling the technical side of things, with both parties coordinating testing and end-user acceptance.

I've also worked in software development writing software for external customer use, and the process there was far more like a traditional waterfall with JAD sessions, a formal SQA period before release, etc., but that product had a release cycle between 6 and 18 months so they could afford to take the time.

When doing development in-house, often a fix is needed in days, or hours. You don't have the TIME to follow a large complicated process. Especially in the airline industry, which is where I've spent all of my 20 years as a designer and programmer/analyst. If a plane is taking flight delays because of a hitherto unknown bug in your software, you apply a fix now and ask permission to do so after the fact. :-)

Simple !! Just fire 90% of them including all but one manager !! Any one who did the feasibility and analysis studies and still agree to a team of 500+ should be re-investigates to see if he was ex-Enron !!

I agree with a lot of the comments above that a small, compact design team overseeing several development teams is the best way forward for speed of delivery and agility. However, the main issue of agility is that the original analysis and design *MUST* have that built-in from day one !! Rewriting hardcoded shit is the anti-thesis of agility. Something that I am all too familiar with, unfortunately !!

Like the old military maxim - It's better to have it and not use it than to need it and not have it !!

One other issue of agility is what a Russian general said - "Perfection" is the enemy of "good enough" !!

It all depends on what you call Agile. IBM claim to have gone agile but their version of agile appears nothing like my vision of agile (from the perspective of a 5 person team). I wet myself with laughter every time I hear one of these IBM guys talk about how agile they are!

By inshoring, to competent developers, you can achieve a 14:1 ratio from your crappy outsource staff. Thus your 200 person project becomes a 15 person project - most definitely within this size of an easy "professional" (as i've taken to calling 'agile') project.

(14:1 recently achieved at a big investment bank, by inshoring from big vendors - count the cost savings!)

Any project with 500 developers is going to fail, no matter what methodology is used. If you really have that much work, split it up into a number of smaller projects, and run a programme to synchronize the projects (although in my experience, programmes always fail anyway).

A few years ago I worked on a project that started with 15 developers using a traditional waterfall approach. It started getting late, so the pointy-haired boss decided to use the Mongolian Hordes technique, and we ended up with 27 developers. Of course, the project just got later and later, and we ended up with loads of defects because of people overwriting each others' code. The project ended up four months late against an original timeframe of 12 months, and the customer was not happy at all. Communication within the team was terrible - there were 2 or 3 developers that I never even spoke to at all, and I was one of the 3 technical leads on the project!

The following year, I ran a project of about the same size, only this time we did the whole Agile thing - release early and often, software over documentation, involve the users etc. With only 6 developers we produced the same amount of code as the other project in just 9 months, and were only 4 weeks late going live. We had a much lower defect rate, and a very satisfied customer. The code was also a lot better - we were able to do a substantial refactor after we went live without breaking everything. The key thing was that we had a project manager who trusted the developers just to get on and do the project without trying to micromanage. She still got nervous when we wouldn't give estimates of how much work was left to do, though :-)

Both of those projects were when I was working for IBM Global Services, so yes IBM can do agile. Sometimes.

The trick is not to have large scale projects.

The best manager of large scale projects I ever came across did it by identifing

the most important parts of the project, then identifing his most talelted resources

and only four of five of them and letting them loose. In the meantime he

managed the other 100 people while they muddled through hte non core

pieces of the project.

I did work for a company that brought in Agile just after the project had started. There was a lot differing views on how to work to a methology that either a group of people did'nt know or had no formal introduction or training. This led to some confusion, but after a short period of time it seem to work.

There however a down side to this, as someone else mentioned COMMUNICATION and ACCOUNTABILITY, knowing the true progress of a small project and whether they (the project managers) were sticking to the iterations. Some project managers where slipping work from iteration 1 into iteration 2 and so forth.

This led to the work load increasing to meet the 6 week iterations deadline. This gave a false idea of how the projects were progressing. It was when the company brought in two audit companies to do an audit of the project did they find what was truelly going on. When they had finished and produced their report the project was canned.

I would like to say that i learn a lot from the company regarding Agile and since moving on to pastures new. i have found the company i'm working for now adopting the agile principle, but on a smaller scale . Hopefully they won't fall into the same black hole

Regards

Bob

... just wait until the 'experts' have left the company and there's no documentation or system/subsystem overview !!

(and I include doxygen content in that ...)

I wish people would stop laying blame of failure at the feet of Agile.

If your project fails its normally because ...

a) too little effort / budget / talent invested.

b) too many 'code monkeys' and not enough developers.

c) management lost the plot.

d) it was doomed from the start !

or all of the above !!

What I have found is that senior management (i.e. ex-developers put out to grass when we stopped coding in machine code) think things like ......

"oooh Agile - that sounds like a magic fix for all our past failures ! let's do that !!"

So they turn everything on its head all at once, praying on the 'magic' to happen.

It normally does to start with ... i.e on the first small scale, well controlled and with a talent rich team.

So they add more projects .... which doesn't go quite as smoothly .... so they throw more people at it ...... which increases the budget but normally doesn't actually make things faster or help the project succeed !

Sooooooo ...... it's obvious to them upstairs ... the problem is the lack of control in Agile!

...... so they come up with a bunch of forms and procedures (or hoops and red tape) that they then throw at the project teams.

Surly all these controls will help get Agile back under control they think ............

eerrrrrr NO ! .......

Its not Agile that went wrong, its the idiotic, prehistoric response by senior management at a perceved 'lack of contol' in the Agile process thats the root cause of 90% of big Agile project failures.

Or at least, that's how it looks to me - could be wrong ! :)

Please take a look at the recent QSM Associates report (available on the Rally Software Development web site) on the BMC Software experience. We had 90-95 full time employees organized in twelve Scrum teams coordinated by Scrum of Scrums. It worked extremely well - we achieved exceptional results in terms of both productivity and time-to-market. Had we needed, we could have easily scaled to the level of a few hundred Scrummers without sacrificing effectiveness or efficiency.

The “secret sauce” BMC used to achieve great results is captured in various articles Melody Locke, Paul Beavers and I published in the Agile 2006, 2007 and 2008 conference. It is composed of four simple ingredients: intentionality, know-how, flexibility and patience. You own intentionality, flexibility and patience. If you do not possess the know-how, bring in tried and true Agile consultants. The Rally consultants we worked with made us successful by being true partners.

As Jim Highsmith astutely pointed out, you use Agile to cope with uncertainty, structure to cope with scope/complexity. Agile will scale for you as long as your architecture and organizational design are suitable for large scope. Conversely, poor technology, architecture and structural choices will often manifest themselves as process problems. For example, no matter what process you use you can expect a lot of process headaches if you opted for tight-coupling when loose-coupling was the right choice.

Israel Gat

BMC Software

The question isn't answerable - there is not yet anything to respond to. The story proposes no reason why agile won't scale, nor even defines specific processes or practices that won't scale. The story reflects a sentiment that some hold, but these sentiments lack an argument that can be discussed.

Arguably, a waterfall project is more likely to fail (and massively) at scale, for most products than an agile project. Design first, heavily planned projects will often be massively wrong in their assumptions up front and unable to adapt. These non-agile projects are frequently under-delivered,over budget, and late ... we simply tolerate that conventional projects usually fail. My city is littered remnants of (non-agile) companies that almost product glorious products that evaporated like the vaporware they were.

Most of what "agile" methodologies attempt to address are the failures of conventional methodologies to scale and adapt. An agile methodology places delivering value to a customer first, delivering some unit of value rapidly, and mechanisms that self correct in the face of uncertainty.

We could talk about certain projects being unsuited to particular methodologies. For example, I wouldn't want to fly a spaceship to the moon on strict iterative program. Get half way there and then start working on the reentry shield? No. However, very very few of us are flying to the moon, and the methods used to get there are a poor example for the rest to follow.