Jan

26

Challenges in Iterative Development

Posted by Keith McMillan

January 26, 2008 | 3 Comments

I was talking with a potential client this past week, and one of the things they asked was about the challenges they would face moving to an iterative style of development. I thought sharing my experience with a wider audience might be a good idea. Many development organizations want to move to an iterative style of development. The current darling child in the field is Scrum, XP, or another form of Agile development, but what does it mean, and why would we want to do it?

Iterative development breaks down the development process into several smaller chunks, typically called iterations or sprints. The size of the chunks, and what you focus on, differs based on what form of iterative development you choose. In most forms of iterative development, each iteration results in some subset of completed functionality. The reasons for breaking down large projects into iterations are many, but the main ones are that iterative development is easier to manage, develops results more rapidly, allows you to adapt to changes in requirements, incorporate feedback from the customer, and accommodate other changes more easily.

Large software development efforts are hard. I believe this has to do with chaos theory, frankly. As systems become more complicated, changes to one area can ripple and have unintended consequences in another area.

Software development has evolved mechanisms to deal with this, primarily centered on compartmentalizing changes, with the idea that we can try to restrict changes to within a compartment. This compartmentalization has been one of the main driving forces in development methodologies for a while. This is where subsystems come from in classic procedural programming, where object-oriented programming evolved from, and is behind the oft-heard mantra to “program to the interface” rather than the implementation. Contemporary languages, such as C++, Java and C# have native idioms to support formalizing the notion of an interface.

But this only addresses the design of software, trying to restrict the ripple of change when software is changed. It doesn’t consider other changes, those external to the software, and trying to reduce their impact. In a development effort of any significant size and duration, the likelihood that external forces will cause a change are very high. Requirements change can be driven by change in the structure of the business, by unforeseen market opportunities, or by regulatory changes, just to name a few. Not anticipating that external factors will force a change within the lifespan of a software development project is naive.

Additionally, with a project of any size, there are significant unknowns that won’t be resolved until they are worked on. Software engineering differs from our sibling engineering disciplines, because in software engineering, all our efforts are significantly unique. While architects can rely on designs for other buildings, and in some cases, simply reuse them, every new software engineering project is a one-off (otherwise, why are you doing it?). It’s impossible to know in detail, when you start out to plan a software engineering project, what the project will really look like in the end.

Iterative development goes some way towards addressing both of these issues. By breaking down a large project into several smaller ones (which is in essence what an iteration is) allows us to more easily accommodate changes, as now there’s a built-in mechanism to address the impact (a subsequent iteration). It also allows us to make a large project more easily manageable, by making it into several smaller ones. We can then focus on the needs of the next iteration, rather than on the project as a whole. We can now get feedback from the users more rapidly, and even decide to do partial deployment if the business sees advantage in doing so.

It’s an imperfect solution, frankly. Without careful consideration, and a good deal of insight, it’s entirely likely that you’ll not focus on a task until a late iteration that, in order to be most efficient, should have been addressed earlier. I’m thinking here of the tasks that result in significant change to work that’s already been completed. To be fair, this type of thing can also happen because of external changes forcing change to work that’s already been done. It’s an imperfect world, and this is the best we know how to do for now. And it’s certainly better than trying to eat the elephant in one bite.

Development teams moving to an iterative style of development face a number of challenges internally, and these are well documented. Less well known are the impacts on external entities. Breaking your development down into chunks have an impact on three main communities external to the development team: the business stakeholders, the quality assurance team, and the database team. I’m going to be devoting a subsequent post to each of these three groups.

Are you using iterative development?

  • Iterative what now? (57%, 4 Votes)
  • All our projects are iterative (29%, 2 Votes)
  • Rolling out to most projects (14%, 1 Votes)
  • Limited trials (0%, 0 Votes)
  • Considering it (0%, 0 Votes)
  • Iterative and waterfall are both used (0%, 0 Votes)

Total Voters: 7

Loading ... Loading ...

Comments

RSS feed | Trackback URI

3 Comments »

Pingback by Iterative Development Challenges Business Stakeholders : Adept Technologies
2008-01-28 09:04:41

[…] caused by an iterative style of development, and how you can address them. The previous post, Challenges in Iterative Development, gave an overview of this style of development, and some reasons why you’d want to adopt it. […]

Reply to this comment
 
Pingback by Iterative Development Challenges the Data Team : Adept Technologies
2008-01-29 19:59:30

[…] by an iterative style of development, and how you can address them. The first post in this series, Challenges in Iterative Development, gave an overview of this style of development, and some reasons why you’d want to adopt it. The […]

Reply to this comment
 
Pingback by Iterative Development Challenges the Quality Team | Adept Technologies
2008-01-31 16:07:48

[…] by an iterative style of development, and how you can address them. The first post in this series, Challenges in Iterative Development, gave an overview of this style of development, and some reasons why you’d want to adopt it. The […]

Reply to this comment
 
Name (required)
E-mail (required - never shown publicly)
URI
Your Comment (smaller size | larger size)
You may use <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> in your comment.

Blogroll