Dec 13, 2007

Agile Scope Creep and When is "Done" Done?

So, you decide to take this shiny new agile development methodology that everyone is talking about and use it on a new project you've just gotten sign off for.  You get commitments from your customer (I refer to either internal or external customers here) to have a high degree of involvement and you go and assemble a good team, many of whom have worked on agile projects before.  In terms of skill they can write unit tests, do just in time design, document and test with FitNesse, understand Continuous Integration, Subversion, Trac and everything else you want to use in your development process.

You and the team then sit with the customer for a few days and break out the project into various functional and non-functional requirements.  You come up with a list of items and some approximate sizing of the work involved.  You  group the functional requirements into themes to try to align with iterations and sprints.  Everything is discussed to enough detail to be useful for estimating the project and getting a rough plan of attack put together.

The customer is very aware that the estimates are just that - estimates, and that the project velocity won't really be known until 1 or 2 iterations have passed.  With this understanding you get the project moving and commence work.

Time passes, the first few iterations get completed, and a project velocity is determined.  From there an end date is projected and the customer realises that there's more work than budget so they drop some of the lower priority requirements.  The end result is a backlog with an estimated further 6 months to completion.  You're happy, the team is happy and the customer is happy.

In fact the customer is extremely happy.  They've never had this level of service or involvement before.  They're loving it so much that they regularly help the team in finding bugs and point out areas where they may not have explained things very well in terms of getting the functionality just right.  They happily accept that what was delivered is what they asked for, but now that they look at it they realise a few small changes can improve it no end.  You listen to what they say and decide that in the spirit of providing great customer service and wanting to give the customer the best software you can that you can take on their little corrections and alterations as bonus tasks for the next iteration.

Time passes and you reassess the velocity.  You notice that it has dropped by 20%.  That's weird.  The team hasn't changed and if anything seem to be getting more done now than they were before, and yet they're getting through fewer stories and delivering fewer requirements.  The customer isn't quite as happy as they were now that you've informed them of the drop in velocity and a resulting delay in the projects end date.  What went wrong?  Why the decline?

 

Well, as you may have guessed from the title of the post the problem comes from a misunderstanding of how to manage scope in an agile project.  In a waterfall project each of the little changes would have been written up as a change request, designed, documented, estimated, costed, signed off in both triplicate and in blood and then worked on an delivered as a second phase only after all the other agreed to work was completed.  That's because under waterfall models, an item is treated as done once the customer signs off on a spec.  In waterfall done doesn't mean working delivered software, but rather done means the design and scope is locked down and is as unchangeable as the mountains.  Because agile is all about being responsive and doing away with this kind of silliness it's a common mistake (especially for those just starting out with agile) to think that because we want to be responsive and collaborative that if the customer asks for a change, especially for a small one, we can just slip it in with the other tasks in the next iteration.

This is the agile version of "Death by a Thousand Cuts".

 

In agile, the definition of done is Done! meaning delivered working software requiring no other changes.  Not done meaning mostly done but for a few small things.  Not done meaning it kind of works but we'll have to come back later and clean it up.  Done means Done! Finished! Over! Wrapped Up! Complete! Tie a Bow Around It and Put It Under The Tree! Done!  The story/requirement/backlog item as we know it is done.  There will be no more work on it, there are no known bugs to fix and we do not have any other clean up tasks or refactorings to do for it.  We are DONE! :-)

Now, that doesn't mean the customer can't have their alterations, amendments and ideas as well.  It's just means that the way we handle those requests needs to change.  Instead of trying to jam them into the next iteration as extra tasks (not tied to a backlog item), they need to be treated as new requirements.  When the customer makes a suggestion, grab a pen and jot it down as a story or work item and add it to the product backlog so it can be estimated and prioritised.  Near the end of the iteration the team can provide a quick estimate for the new items and you can then go back to the customer and show them just how much effort is involved in delivering all of their extra suggestions.

The customer then has a choice.  Take on the extra work and expand the projects cost and time budgets or reprioritise the backlog and drop some existing requirements.  This way the fixed time and budget for the project can be respected, and scope creep doesn't get out of control.

It can be a great way to assist customers in managing themselves and helps massively in terms of managing project scope - especially if you have a customer that vacillates over what they really want and flitter back and forward between design ideas.  It may also help them see, possibly for the first time, just what sort of impact a lot of minor revisions can have on a project.

3 comments:

  1. You have hit it on the head. I am going through the a similar scenario you decribed.
    We have some new stakeholder (from a different division of the organization) and they now want to add additional features to the already aggreed upon features.
    I promptly told them to add it at the bottom of the Product Backlog, prioritize them and let us know if something else can be dropped.
    Cheers.
    N Rao
    Scrum Master

    ReplyDelete
  2. Given the nature of Agile, how can you ever draw a line in the sand and say "this far and no farther" for the iterations and features? There's no pre-agreed feature set to serve as an objective reference; if there were, you'd be back in waterfall territory.

    ReplyDelete
  3. Great Article. I recently published an online presentation that looks deeper into the Math behind scope creep in Agile methodologies; it agrees with several of your conclusions: http://www.slideshare.net/adidierk/software-scope-creep-and-the-golden-ratio-a-limit-to-effective-agility

    ReplyDelete