Sep

22

What to test?

Posted by Keith McMillan

September 22, 2011 | 2 Comments

A couple of years ago, I started working with TDD, and I must admit it’s changed the way I like to work in software. One of the side effects has been some angst on my part on exactly what needs testing…It’s pretty straightforward if you have a coding standard (like my current client) that says 100% of your code must be tested: you look to see if there are any exceptions to that rule, and if not, you write a test. But suppose you want to follow Kent Beck’s advice in Test Driven Development, and test what’s likely to break? There are some things that are complicated enough that you should obviously test them (computing the monthly payment on a mortgage over various time spans? Probably should test that..) and others that are less obvious.

I recall once I spent a day trying to run down a problem in some Java code. It was a simple setter method:

public class Foo {

int bar;

…

public setBar (int bar) {

bar = bar;

}

}

Spot the problem? I didn’t for a good solid day!  In the setter, bar = bar, what’s called the “formal parameter” is assigned to itself. It should really be “this.bar = bar”. That’s the sort of thing that gives me pause. That was a pretty easy mistake to make, and it took me a day to find it. Is that the sort of thing that’s “likely to break”?

As a result of this sort of thing, I tend towards “test it if you can” and try to find a way to make things more testable. How about you?


Comments

RSS feed | Trackback URI

2 Comments »

Comment by Jason Titus
2011-09-23 06:47:26

Keith,

I won’t argue with the validity of your post. As usual, you are talking about good stuff, and right on. I will say, however, that I feel not enough people rely on the compiler and static analysis tools to assist. The goal of your unit tests is to insure your production software is doing the right thing. In my mind, that is also the goal of the compiler and static analysis tools. Both of which people seem to discount. In fact, I’ve seen that very issue you identify highlighted in Eclipse when configured correctly so you’d have a much harder time missing it. In this particular case I believe the tool that would identify the condition was CheckStyle. We’ve had a zero warnings policy and the first time you do something like this it will stick out on the warnings tab like a sore thumb. It can catch tons of simple things like single equals instead of double, poor null handling, etc.

If you are using a dynamic language, you have a lot more of this burden placed on your unit tests. That fact makes me conflicted as to whether I’m a fan of them or not.

Again, I am strongly in agreement that unit tests (and TDD) are invaluable to a development project. I’m just out to promote other tools that can assist as well.

One other thing I will say is that I expect you are not advocating a simple test specifically for the getter and setter in this case. In my opinion, unit tests are great, but if something like that isn’t exercised by a unit test that provides value in some other way, it is likely dead code (although it wouldn’t have required you to look at it for a day if it were dead code, obviously). TDD helps to keep code clean in that respect, in a way 100% unit test coverage metrics doesn’t.

Jason

Reply to this comment
Comment by Keith McMillan
2011-09-29 05:47:55

Hi Jason,

As usual, very insightful comments!

With respect to the static analysis tools, my example was a bit of a throw-away: years ago when I had the problem, the IDE in question (eclipse in fact) didn’t have that sophisticated a set of static analysis tools, so it didn’t highlight my error, but that’s not really the point. As you allude to, my point was sometimes simple things add value, and so the advice to “test what’s likely to break” is somewhat difficult to follow. It’s also difficult for a static analysis tool to say “that’s wrong” usually they’re limited to saying “that’s suspicious.” People have to do the thinking, and that’s where we have to use unit tests to capture the intention of the code. Static analysis will, I think, always be limited to flagging things for further review, until computers become sentient (I for one welcome our new computer overlords).

I’m also conflicted about your last point: advocating for a unit test for just a setter and getter. In fact I think I am in fact advocating for that. I could deduce the function of the test from other, higher level functions that use the property in question, but when they failed, it might take me quite a while to figure out why (and in fact, it did take me quite a while). If the “unit” that I’m testing is an entity, then I think I come down on the side of writing a test for the getter/setter pair, even if it’s tested elsewhere.

– Keith

Reply to this comment
 
 
Name (required)
E-mail (required - never shown publicly)
URI
Your Comment (smaller size | larger size)
You may use <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong> in your comment.

Blogroll