Iterative Development Challenges the Data Team

Posted by Keith McMillan

January 29, 2008 | Leave a Comment

Software engineering teams everywhere are moving towards, or have already adopted, an iterative style of development, because it creates a number of desirable effects: better time to market, earlier feedback, easier to manage, to name a few.

But what does this mean to the groups they work with? In this, the third of 4 posts, I look at the challenges to the data team caused by an iterative style of development, and how you can address them. The first post in this series, Challenges in Iterative Development, gave an overview of this style of development, and some reasons why you’d want to adopt it. The second, Iterative Development Challenges Business Stakeholders, reviewed the challenges faced by the customers of an organization moving to an iterative style. This post covers the data team.

Changes to the Status Quo

An essential part of many large-scale development efforts is the database in which information is stored. Many large businesses have set up dedicated teams of resources to create, tune and otherwise manage these databases. These folks are typically called data architects, database designers, or database administrators. What does the move to an iterative development process mean to them, and to the work that they normally do?

The data team’s job is to understand the structure of the information that our applications store, to create database structures to support them, and to tune the database to peak performance. The structures they create are typically table spaces, tables, triggers, views (materialized and normal), stores procedures and indexes. With your typical object-oriented project, many of the the domain objects that the designers identify will find their way into the database as tables, and their attributes as columns.

With an iterative style of development, the designers frequently identify new objects, and add additional attributes to previously identified objects, with every iteration. This can be every few weeks. Understandably, this results in the primary source of friction with the data team. With waterfall development, they’re used to having the completed design of the system, then creating the database structures that will best perform the task at hand, tuning it to within an inch of it’s life.

Now, however, the designers of the system are not only turning over incomplete designs (one per iteration), the designers don’t even know what the completed design is until rather late in the development process. As we discussed in the first post in the series, one of the typical goals of many styles of iterative development is a working subset of functionality every iteration. This typically includes some database support. Many database organizations are made unhappy by this state of affairs. They view it as doing the same work over and over with subsequent iterations.

What to do

It’s not all bad news for the data team. The use cases or stories used by many iterative styles of development to capture the requirements give the data team much more information about the patterns of data access than they may typically received. Better still, if the designers produce UML interaction diagrams to show the design of the functionality, then it removes even more of the guesswork about how data will be retrieved by the system. This helps the data team create the appropriate structures for best performance, because it shows them the interactions that the system will need to support. So that’s an upside.

The first step in easing relations with the data team while moving to iterative development is to engage their leadership in the changes you’re making. Once you have management’s support, the next step is to bring a representative from the data team into the design process as a valuable part of the design team. Incorporate their input as the design goes forward, listen to their input, and they’ll be much less hostile to your adoption of incremental processes for developing your systems. Since the changes to the data team aren’t as traumatic as those to other groups, these simple measures should go a long way towards making your interactions with the data team work well.

So far, we’ve provided an overview of iterative development and why you’d want to do it, the challenges posed by interactions with the business stakeholders, and the data team. The next post in this series will review challenges faced by the quality assurance team.

Are you using iterative development?

  • Iterative what now? (57%, 4 Votes)
  • All our projects are iterative (29%, 2 Votes)
  • Rolling out to most projects (14%, 1 Votes)
  • Limited trials (0%, 0 Votes)
  • Considering it (0%, 0 Votes)
  • Iterative and waterfall are both used (0%, 0 Votes)

Total Voters: 7

Loading ... Loading ...


RSS feed | Trackback URI

Comments »

No comments yet.

Name (required)
E-mail (required - never shown publicly)
Your Comment (smaller size | larger size)
You may use <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> in your comment.