Acceptance Tests Don’t Replace Unit Tests
I read a post last week that was celebrating (among other things) one developers epiphany that unit tests were a waste of time. He focused on acceptance tests and that was all that mattered. His case was that his boss and customers only care about the end result and not the unit test suite. He further gave evidence that even with his unit tests, his functions broke.
The Big Misconception
There is seemingly a big misconception that automated acceptance tests and unit tests perform the same functions and have the same goals. I don’t see it that way. Automated acceptance tests and unit tests compliment each other but have very different goals. The following table illustrates how acceptance and unit tests relate on a few levels:
|Unit Tests||Acceptance Tests|
|Written By:||Developers||The business or Q/A|
|Written For:||Developers||The business|
|Ensures:||The code does what the
developer expects it to
|The application does what
the business expects it to
|Modular / decoupled code||The code solves the business problem|
|Tests:||Individual units||The system as a whole|
|Tells you:||Where the code is failing||The code is failing|
How They Compliment Each Other
Both automated acceptance tests and unit tests allow developers to implement change. Change is inevitable and we, as professional software developers, need to build software that can adapt to change. In fact, as professional developers, we need to initiate change. Refactoring to better design is / should be a common activity.
The Role of Automated Acceptance Tests
If you stand back and think about what we, in IT, are trying to achieve it is this:
- The business defines the requirements of the system
- The architects / developers build a system that meets those requirements
The challenge we have historically had is in unifying the requirements with the system actually being built. The requirements may be living in several places:
- A high-level requirements document written by a business user
- Technical specifications written by a business analyst
- Design documents written by an architect or developer
These documents can easily become out of sync with the software. How many times have you started a project with a requirements or specifications document as your guide. As development progresses, we learn more and the requirements naturally change. Unfortunately, the documents don’t. They quickly become obsolete. It requires a PM with obsessive compulsive tendencies to keep everything synchronized.
With automated acceptance tests, we can capture features and specifications of the system within the system. Best of all, we have acceptance criteria for those specifications directly within the system and they are executable as tests. The tooling today is getting better and better. Tools like Specflow are leaps and bounds over where we were years ago.
What Don’t Automated Acceptance Tests Do?
Referring back to the table above, you can see that the specifications are written by the business and the acceptance criteria are there to ensure that the code does what the business intends it to do. What they are not all that good at are telling you where the code is failing or providing fast feedback. They also don’t help ensure that the design is decoupled and maintainable. Those are things that unit and integration tests are good at.
The Role of Unit Tests
To my thinking, unit tests provide 4 services:
- Guides us toward writing decoupled, testable, maintainable code
- Proves that the subject under test does what it is designed to do
- Allows us to refactor code
- Allows us to adapt to change
Perhaps the biggest benefit of unit tests is that they guide us toward writing testable, decoupled code. I’m sure most of us have been forced to maintain highly coupled code where a change to one method has rippling effects. The kind of code where a seemingly simple change will take days or weeks and not minutes.
Numbers 2, 3 and 4 talk to the other main purpose of unit tests: to ensure that the code does what you, as the developer, expect it to. With adequate test coverage, you have the confidence to refactor code, knowing that you won’t introduce a regression.
While automated acceptance tests will (hopefully) tell you that you have introduced a regression, it is less likely that it will tell you where. Furthermore, automated acceptance tests normally take much longer run than unit tests. Unit tests are about fast feedback.
My Unit Tests Passed, but My Code Is Broken
What happens when all of your unit tests pass, but there is a bug in the application? One of two things is going on. The issue has to do with the integration between components and you are missing an integration test or your unit tests are not testing the correct thing. It is possible, even likely, to have high unit test coverage, yet not be testing the right behaviour. Testing is every bit an art form as software development. It takes time and thought to get it right.
Both are Tools in the Toolbox
I doubt very much that if you spoke with a good carpenter you would hear him/her say anything like “I used to use claw hammers, but now I only use sledge hammers. They get the nails in much quicker. Claw hammers just slow me down.” No. They keep both in their toolbox and use each for the job it is best at.