Imperfect tests are better than no tests
Developers want to produce high-quality products, but are often asked to do this with fewer resources and less time than we would like. The balancing act between quality, cost and speed has never rested on the shoulders of developers as much as it does today. Companies assume developers are agile, and therefore can easily introduce CI and CD and shift left product development. Once this small task has been completed, development rates will accelerate and we will deliver features at 10x our current rate.
If only it were that simple
Behind every buzzwordy new approach is an assumption that the new approach can be implemented perfectly. However, given the pressures that development teams are under, the question we should start thinking about is: what is good enough to deliver a return? Let’s turn to our automated test suite, which is integral in accelerating our development processes. What are we trying to achieve? We want each build to be self-testing, i.e. to give us a clean bill of health (or report an issue). Reporting issues is the easy bit: if a test fails, the build is marked as broken and people can immediately look into it.
Getting a clean bill of health from the build passing is much harder. We are trying to prove the absence of bugs. Therefore we can consider the perfect test suite as one where, if the build passes, there are no bugs in the software. This is a lofty goal, however-as momentous as trying to prove that aliens don’t exist.
Having realized that it is impossible to create the perfect suite of tests, and being under pressure to deliver, often leads to teams simply not writing unit tests, and instead raising tickets saying, “We should write unit tests for feature x.” This is a compromise. But is it the best compromise?
Rather than rationalizing the decision not to write tests because of time shortages, how about asking yourself this: Do I have time to write one test? If the answer is yes, then write that one test. And keep going until you run out of time. It’s true that this won’t lead to a perfect test suite. But as Martin Fowler says: Imperfect tests are better than perfect tests that are never written.
All too often there is a tug-of-war between teams when tests fail. Teams ask: are the tests wrong, or is the code wrong? If we stop thinking about the tests failing, and instead think about tests reporting change in the product, then rather than looking for blame, we can consider whether the change is desirable or not.
Given a spare half an hour each day, we could all write a new test and gradually increase overall code quality. Remember: the test doesn’t have to be perfect; it just has to be good enough to highlight an issue when it fails.
