Iterative Development Challenges the Quality Team

Posted by Keith McMillan

January 31, 2008 | Leave a Comment

Software engineering teams everywhere are moving towards, or have already adopted, an iterative style of development, because it creates a number of desirable effects: better time to market, earlier feedback, easier to manage, to name a few.

But what does this mean to the groups they work with? In this, the last of 4 posts, I look at the challenges to the QA team caused by an iterative style of development, and how you can address them. The first post in this series, Challenges in Iterative Development, gave an overview of this style of development, and some reasons why you’d want to adopt it. The second, Iterative Development Challenges Business Stakeholders, reviewed the challenges faced by the customers of an organization moving to an iterative style. The third, Iterative Development Challenges the Data Team, dealt with challenges in the database organization and how you can address them.

Many large enterprises have established separate testing or quality assurance teams to test their software products, just as they have established separate data teams. Some, such as those in the medical devices industry, are required by law to do so. As with the business stakeholders and the data teams, these quality organizations are impacted by the development teams moving to an iterative style of development. In what ways?

In a waterfall style of development, the quality teams receive a complete product to test, one in which all functionality is completed and expected to be working. Iterative development changes this interaction, and the quality team can and typically are asked to test a release every few weeks. These releases have incomplete functionality, because not all the requirements have been addressed by iterations before the final one.

To add to the complication, each iteration can result in changes to functionality that was already tested. These changes can be intentional and desired, due to change in requirements, or unintentional breakage that should be caught and repaired. The quality team is expected detect these unintentional breakages.

So the quality team really gets it coming and going when development moves to an iterative style: they’re asked to test a release every few weeks, to understand what functionality they should be testing vs. what hasn’t been worked on yet, and to detect regressions in functionality that used to work but that has been unintentionally broken. This is a considerable list of challenges.

Not All Bad News

As with the database team, it’s not all bad news for the quality team. If the development team is making use of use cases or stories, these requirements contain valuable information that document the desired functionality of the system. They are typically a starting point for writing test cases, and also form a good mechanism of communicating to the quality team what is and isn’t in scope for any particular iteration. Many styles of iterative development also place a greater emphasis on testing within the development organization itself. The practices of continuous integration and test-driven development are examples of such practices that yield a better product going to the quality team.

It’s worth pointing out that while development organizations may use continuous integration and their own testing tools such as CruiseControl and JUnit/NUnit, these are typically unit tests, rather than the system/acceptance/intended use testing that the quality team focuses on. The use of these types of tools can result in better quality going to the quality assurance team, but can’t replace the testing provided by the quality team.

How to Survive

In order to make the transition to iterative development successful when interacting with a quality team, particularly one where this team is expected to test every iteration, you need tools. The degree of sophistication required in the tools varies based on the specifics of the project, sometimes purpose-built software is required to be most efficient.

The first tool you need is a good mechanism to communicate what the quality team is expected to test. The best tool for this are good requirements in the form of use cases or stories. From these requirements, the quality team can formulate their test cases. The suite of test cases grows with each iteration of the software development process. Requirements can be captured in a purpose-built requirements capture tool such as IBM’s Requisite Pro, an online tool like Accept 360, on a Wiki (I’m a fan of Atlassian’s Confluence Wiki for ease of use, but there are plenty available for free), or in a simple word processing program.

As the suite of test cases grows, you’ll discover that the second tool you need is a way to manage the suite of test cases: a tracking mechanism to determine which test cases are to be run for any given iteration, and the results of the run. There are a myriad of these types of tools available, such as Rational Test Manager, Borland SilkCentral Test Manager, and a raft of open source alternatives of varying capabilities.

The third tool that will eventually need to come into the mix is an automated test execution tool. The iterative process adds tests to the test suite every iteration, and ideally we should re-run the entire regression suite with every iteration to detect breakages. Therefore, the amount of work increases every iteration. This can be significantly more work for an iterative style of development than for waterfall. The good news is that re-running the regression tests for unchanged functionality can be automated, and should be for any development effort of significant size. The use of automated testing tools allows the quality team to focus on building new test cases and verifying new functionality with every iteration, rather than re-testing unchanged functions. Tools such as HP Mercury WinRunner are good for these sorts of automated regression test suites.

With the adoption of these tools and techniques, the quality team can overcome the challenges presented them by a move to an iterative style of development.

This series has looked at the impact of iterative development on parts of the business other than the development team. The impact to organizations other than the development team is sometimes poorly understood, and hopefully this series has helped point out some of the challenges you will face as you move to iterative development. Some may ask whether a move to iterative development is worth the risk posed by the challenges we’ve reviewed. I believe firmly that iterative development gives more than it takes away: it yields better products, faster, more predictably and with more flexibility than waterfall-style development. It’s my sincere hope that a more complete understanding of the challenges iterative development poses will help you be successful.

Are you using iterative development?

  • Iterative what now? (57%, 4 Votes)
  • All our projects are iterative (29%, 2 Votes)
  • Rolling out to most projects (14%, 1 Votes)
  • Limited trials (0%, 0 Votes)
  • Considering it (0%, 0 Votes)
  • Iterative and waterfall are both used (0%, 0 Votes)

Total Voters: 7

Loading ... Loading ...


RSS feed | Trackback URI

Comments »

No comments yet.

Name (required)
E-mail (required - never shown publicly)
Your Comment (smaller size | larger size)
You may use <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> in your comment.