TDD: Downsides

Being a fan of TDD I have heard and read a lot about the benefits of test-driving software and creating tests in general – as well as about the drawbacks. While the most widely mentioned issue with tests is false confidence that they might give developers, there are also other major problems associated with this popular approach to software development. Most of these come from misusing TDD and because I have mastered the skill of doing things wrong I would like to describe some of the potentially harmful results of test-driving programs.

One problem that I faced is that tests grab developer’s attention. Tests can be easy to write, are known as positive and rewarding activity and support one with some confidence, so in many cases it is pleasure to create them, especially when the tested components are small and well-designed. In case they are not, composing test cases becomes difficult and makes one feel that since the task is hard it must be important as well. Moreover, creating a test suite for a complex piece of software is a challenging task and we, programmers, do like challenges. On the other side, when we test-drive a component, tests allow us to almost discover it as though it is already created and we just keep unveiling it piece after piece. Sometimes it really looks like this – and it is amazing! In reality, however, we are the ones who create and discover the thing at the same time, hence the exploration might last as long as our imagination allows – that is infinitely long. What I want to say here is that it is extremely easy to fall into hacking tests for the sake of tests themselves or, more precisely, to test much more than it is required right at the moment. While tests are not hamburgers and one may believe there can’t be too much of them, our time and powers are limited, so when utilizing these resources we should do it carefully: our customers may feel good if the product is backed by an extensive test suite, but they will hardly pay for tests alone. Personally, I sometimes find myself writing new tests when taking the next step along development path requires me to make difficult design decisions and I feel afraid of doing this. In such circumstances the idea that the existing code is not tested well enough comes as a remedy that seemingly frees me from the responsibility for creating new pieces of the system. Whenever this happens, it is crucial to push yourself to advancing the product instead of cowardly trying to cover with tests something, who’s existence is justified only by the tests themselves.

Now to hamburgers. How do you think, can software really get fat, sorry, overweight? Every experienced developer shouts ‘Yes!’ now – I almost hear it. Like in case of people this may be caused by unhealthy food, our problem can come from bad code. Now, try to imagine how difficult it would be for you to move if you are, say, 150 or 200 kg person? Or rather, how do you think, will it be easy to learn to drive if you have difficulties with pulling your entire body into a car (closing the door is not absolutely required)? The same way if a piece of software is written badly – is ‘fat’ – it won’t be easy to add new features to it or even make the old ones work correctly. You will now ask me, what does this have to do with tests? Aren’t tests a kind of gym-training for programs? Well, while it may seem so, I believe they are more like healthy food. Eating proper things is good for your body and health, but eating too much of good things will lead to the same problems as fast food does. Tests serve to encode assumptions, to specify how your components depend on what. Making dependencies explicit is great, but explicit dependencies are still dependencies. This said, when writing tests you set up relations which make future changes difficult – no matter whether these are harmful or not. The above issue becomes very severe when you test implementation instead of interfaces, but even avoiding this won’t save you from pain related to altering an interface of some component, not to mention the changes in the design of the entire system. I have written a post about the problems caused by the decision to change a signature of a constructor and showed some recipes to make the task easier, although the situation described there is just a tiny, toy-like example of how even well-written test suite can resist the progress of a software system. Too many tests unfortunately can’t make your code more reliable and documented at no cost – they also make software inert, ‘fat’.

As though that’s not enough, there are also additional costs introduced by investment into testing. Particularly, when you attempt to create unit tests for a C# class and place them in a separate project (obviously, a good idea), you are forced to make the tested component public in the C# sense, so that tests can reach to it. That inevitably leads to expanding the interface, opening a larger than required part of your system to the world. Even if one can deal with this by means of extra tools and more complex build process, that's an additional portion of work, which I find hard to justify. Furthermore while the frameworks and tools that we use to create tests struggle hard to make the process easy and avoid influencing the production code in any way, sometimes they can fail, for example implicitly making certain design decisions more favorable than the other, better ones or simply making us write a bit more code than we need, so that tests do impact our product more than we want.

I don’t write this to restrain one from test-driving software. Instead these ideas lead me to a banal conclusion that when choosing where to allocate available resources we have to do this carefully. Tests give us the opportunity to write better code, refine interfaces, avoid many mistakes, better understand what we want to achieve and at the same time document our systems. Still even with all these benefits, there are risks associated with test-driving software and one should keep this in mind, so that no harm is done to a system by testing too much, introducing useless features or opening details of implementation to users.

CodeProject