[]
TDD *is* about testing
[ tirsen ] 18:47, Monday, 26 January 2004
Sorry, Dan North but I have to disagree. I've always been a bit sceptical about TDD not being about testing but Martin Fowler latest blog entry got me thinking. So I might be a bit old-fashioned here but let's approach the subject from a pragmatic view-point.

I see two fundamentally different types of testing in an XP/TDD/Agile project. Acceptance testing and Unit testing. There's no point in describing acceptance tests as driving the design but unit testing is commonly viewed as doing just that. What you are saying Dan, is that to a very large extent TDD is about driving the design and very little about producing a high-quality unit-test harness. Let's see what I prioritize when it comes to acceptance and unit-tests (both can be used to drive development) and why these two testing approaches complement each other so nicely.

Acceptance tests are about (in order of priority):

  1. Capturing, documenting and proving customer requirements This is sort of the holy grail of XP, but we actually have a couple of projects were customer requirements are primarily captured as acceptance tests (never actually written solely by the customer of course, that is (j)utopia, but definitely written in a form that can easily be read and verified by the customer, more on this some other time).
  2. Ensuring that all components in the system play along nicely Acceptance tests should therefor as much as possible be integration tests, that is as little part as is practically possible should be stubbed out. If you rely on some systems that are in internal control, run directly against them, if you rely on external systems consider stubbing them out if they are volatile. This also implies that acceptance tests are quite slow, many suites can keep on going for many, many hours (I've seen suites that take as long as 20 hours).
  3. Insuring against regressions Many agile projects try to release often, the acceptance tests can be used to insure the customer from broken business-critical behaviour.
  4. Provide a way for the customer to monitor progress Reports on what stories (or use-cases) have been succesfully tested is the surest and most truthful way for a customer to understand what progress a project is making.

The point of unit-tests are (in level of priority):

  1. Run fast Agile projects are all about enhancing and shortening feedback loops, if something goes wrong you should know about it as soon as possible so that you can correct your behaviour. The acceptance tests are usually sloooow (unless you're really lucky) so they are virtually useless for providing feedback to a developer when he's checked in some bad code. A unit test suite should run with everything stubbed out and run in minutes. My rule of thumb is: unit tests < 1 min => you're at top productivity, unit tests ~ 5 mins => you're still agile, unit tests > 10 mins => you're no longer an agile project, unit tests > 30 mins => you're project is screwed, look for a new job. Okay, don't take this to literally, but you get my point. (This is also the reason why I keep bitching about not starting web servers in your unit tests.)
  2. Show what is broken Acceptance tests show that something is broken, unit tests should show what is broken. Ideally for each failing acceptance tests there should be a couple of failing unit tests. To be more precise, a failing acceptance test is symptom of an incomplete unit-test suite (thanks for that one Martin) and before the fix is introduced a new failing unit-test should be added to the unit-test suite.
  3. Give you courage to refactor Refactoring without unit tests is like the circus without a safety net. You can do it, but you it's really dangerous (and you have to be really good). Most people don't have that sort of courage (I sure know I don't!) so important refactorings never ends up being done leading to faster software rot.
  4. Ease communication with your peer while pair programming In my opinion you can't do pair programming if you're not doing TDD. The unit-test is a natural discussion point and the rule about just implementing enough for the test to pass makes discussions more to the point. This makes the pair less likely to disagree about subjective matters where communication usually start breaking down.
  5. Drive the design on unit level Here's what you are talking about Dan. This is actually essential for creating a high-quality test-suite that fulfills all of the above, but when I actually judge a test-suite this comes quite long down on my list. The unit-test acts as "yet another client" to the unit (eg class) you are trying to develop helping you think about the interfaces, collaborators and other types of structure before you actually start writing the actual code.
To be quite honest, the main reason why I write my tests before I actually write the code is because it's sooo boring writing tests afterwards. I mean, I usually know it works (via for example manual tests) so why should I spend all that time writing tests for it again?
Comments

A decent test suite is a _side effect_ of the TDD process.

While this is not a side effect to be neglected, (it is fantastic!), I still think about TDD as a _design process_.

TDD is try and fail experience. You (the code) learns (improves) at each small coding iteration (test execution, refinement, refactoring).

All great line dancers have fallen hundreds of times in the net. I'm sure it's possible to learn to walk on a line without a net (from say, 50 cm). But those who learn it that way will never be as good as those who can rely on the net when rehersing. Instead, they will always walk staccato and there will be no somersaults!

(OK, it's a bit far fetched, but it was your metaphor ;-))

Aslak

--Aslak Hellesoy, January 26, 2004 11:00 PM

The interesting thing is: To have a high quality unit-test (that is fulfills my criteria as above) is almost synonomous to have a high quality design in your actual production code!

TDD is a good way to get high quality unit-tests and therefor high-quality design.

See, we don't disagree, I just see it the other way around.

--Jon Tirsen, January 26, 2004 11:17 PM

Acceptance tests and integration tests are not the same thing. Acceptance tests demonstrate that the system meets requirements. They run end-to-end and produce results that make sense to the customer.

Integration tests are for the developers, not the customers. They test that the units "play nicely together", but don't all run end-to-end.

--Nat Pryce, January 27, 2004 12:15 PM

Did I implie that?

In most situations I don't see the use of special integration tests as necessary, acceptance tests can usually serve that purpose.

--Jon Tirsen, January 27, 2004 01:28 PM
Post a comment









Remember personal info?