Aug

31

What’s Wrong with Test Driven Development?

Posted by Keith McMillan

August 31, 2009 | 3 Comments

I’ve recently been working on a project that is making a very determined try to use Test Driven Development (TDD). For those who are unfamiliar with the practice, rather than writing your unit tests after you write your code, you write the tests first.  I’ve known about the practice for years, but this is the first time I’ve worked on a team that’s really doing it, and I’m no less puzzled than I used to be. Well, we should probably talk about first things first.

The TDD mantra goes:

  1. Write a little test, just enough to fail.
  2. Run the test, and watch it fail. This proves that the test works.
  3. Write a little code, just enough to make the test pass. This can be as down and dirty as needed.
  4. Run the test and watch it pass (fix the code if your test doesn’t pass).
  5. Refactor your code, cleaning up any  messes you’ve created.
  6. Repeat until you can’t think of any more tests

Let’s take a look at each of these in turn. The write a little test part can be as simple as trying to create a new class that doesn’t exist, or calling a method that doesn’t exist. In Java, this could look as simple as:

public class MyTest {

@Test

public void testCreateMyClass() throws Exception {

new MyClass();

}

}

Now, if MyClass doesn’t exist, you have a test that won’t compile, but should, and voila, you have your broken test.  To fix this test, you’d simply create MyClass, and you’d be on to step 4 above, run the test and watch it pass, which it would.  At this point, you’re free to refactor (none needed here, I think), and write our next test.

Our test isn’t very interesting, since all it does is test that MyClass is a real class. The next test could test that there’s a method, say “foo()” defined in MyClass, and then say that foo does something interesting.

Working in this way has several advantages, for one if we agree that We Should Write Tests (we do agree that tests are a good thing?) then what does it matter if we write the tests first or second?  We’ve traditionally written them second, but if we write them first, then we’re pretty sure

  1. that we wrote tests, and
  2. that the code we wrote was testable, because we wrote the test first, right?

So we end up with a very robust suite of tests, which we constantly run. Our current code coverage is somewhere in the 98% range, and that’s including the UI which is difficult to test.  We also have very few defects reported by our testing folks, and I really do attribute this to our use of TDD.

TDD also provides you with a great safety net. When you’re working on a large software project, you frequently need to change the design as you add new features.  TDD allows you a lot more freedom to make major changes, because you can always re-run the test suite and be sure that everything that used to work still works when you’re finished.  It’s gotten to the point where if I run the suite when I finish a major change and something isn’t broken, I get suspicious about my test suite.  It’s tremendously helpful.

So here’s the puzzling part: why is it so hard to get developers to even try to use TDD?  Most everyone I’ve talked to in the last decade or so agrees that it seems like a good idea, and that we should be doing it, so why do we constantly find excuses to not begin?

There are some parts of the puzzle I think I understand, but I’ve not put the whole picture together to my own satisfaction yet.  I believe, for instance, that project managers sometimes give lip service to quality, but their first and only loyalty is to the schedule.  As exhibit A, I present one of the project managers from my current client, who when we mentioned that we should be testing more responded with “But hey, you guys are rock stars, right?”  and “Just do what you need to, but we need something in three weeks.”  Given the subliminal message here (quality is unimportant, and we can get it later if we need it) is it any surprise that practitioners are hesitant to say “we’re going to write the tests first?”

I also believe that there’s a belief that writing the tests first somehow slows practitioners down. This has always felt intuitively right, in the same way apparently that looking at a jumbo jet, I say “That thing is going to slow and is too big to fly.”  IBM and Microsoft have both run studies, and published the results, indicating that while TDD sometimes had minimal impacts on overall productivity, these were offset by maintenance savings. Indeed some teams even went faster with a TDD approach.  I also think that the test-first approach makes code that’s testable the first time, rather than writing some code and only then discovering you’ll have to change (i.e redesign)  it if you really want to test it thoroughly.

I also think that some practitioners have drunk the Kool-Aid, and believe that they’re rock stars, and they don’t need to test.  These folks, thankfully, seem to be a minority.

Then there are the architects (I’m an architect, but not one of these) who still believe in the Big Design Up Front, and they can’t conscience letting the tests drive the design, since this means that the design is no longer their sole domain, but one that’s at least partially shared with the folks writing the tests.

The final piece of the puzzle I think I understand is one of the Great Virtues: Laziness.  I think we get lazy around unit tests, and believe that things are “good enough.”  This I think is the truest of the reasons why TDD seems to be hard. It takes determination to always write your tests first, even when you know what you’re going to do, and you know how you’re going to do it.

There’s also a tendency to give unit tests short shrift. We’d much rather just fix the code than fix the code and the tests, and let’s face it, when we break the code, we will break the tests.

So to sum up, I think that TDD is a good idea for all the reasons I’ve covered:

It requires a little additional investment: your unit tests must be kept just as clean as your code. So why is it so hard to get people to even try it?


Comments

RSS feed | Trackback URI

3 Comments »

Comment by Daniel Markham
2009-09-01 14:25:52

So:

TDD is great, but it confuses you. It confuses you because it is so great you do not understand why everybody would be doing it.

Did I get that right? Sounds like we moved off into marketing land there, dude.

How about “TDD is great because on this project I’m on we’re delivering more code faster with less defects than we would have done otherwise”

Then build a supporting case for that.

Sorry for busting your chops, Keith! I just wrote you a glowing review on LinkedIn, and had to let out my crankiness somewhere(grin) Somehow your article set off my BS detector, but it’s been needing some work lately, so hard to tell if the alarm was real or not. I might need to change the batteries.

Reply to this comment
Comment by Keith McMillan
2009-09-09 08:00:45

Let’s try that again.

TDD does yield very good things, and I find the arguments against it to be generally pretty weak.

At the same time, I don’t see a lot of (very smart) people adopting TDD, and even those that do seem to want to abandon it, even when they will admit they should probably be doing it.

Even when teams are encouraged (or even pushed) to adopt TDD, it seems to be difficult to stick with. I put myself in the the same camp, frankly: I see the positive results, yet I continually fight the urge to cut corners.

Given that it’s full of goodness, as it were, what’s the problem (or problems) preventing more widespread adoption? Why do I have to convince myself that I need to keep doing TDD, given that there are so many benefits, and I can’t put my finger on many shortcomings? I’m not the only one, clearly, so the post was an attempt to work through some of that, and engage a bit of group-think.

Reply to this comment
 
 
Comment by Mediocre-Ninja.blogSpot.com
2010-08-10 20:37:15

I’d rather call it Behaviour Driven Development. When you think how to write the tests, you think how the application behaves, and that’s the root of the business value.

Of course BDD has more benefits, for both end-users and the development team. But if some “TDD guy” insists on ‘test-first’ without understanding why it is neccesary, it would become a burden for the team 🙂

Reply to this comment
 
Name (required)
E-mail (required - never shown publicly)
URI
Your Comment (smaller size | larger size)
You may use <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> in your comment.

Blogroll