Demystifying The Code

Acceptance Tests Don’t Replace Unit Tests

I read a post last week that was celebrating (among other things) one developers epiphany that unit tests were a waste of time. He focused on acceptance tests and that was all that mattered. His case was that his boss and customers only care about the end result and not the unit test suite. He further gave evidence that even with his unit tests, his functions broke.

The Big Misconception

There is seemingly a big misconception that automated acceptance tests and unit tests perform the same functions and have the same goals. I don’t see it that way. Automated acceptance tests and unit tests compliment each other but have very different goals.  The following table illustrates how acceptance and unit tests relate on a few levels:

  Unit Tests Acceptance Tests
Written By: Developers The business or Q/A
Written For: Developers The business
Ensures: The code does what the
developer expects it to
The application does what
the business expects it to
  Modular / decoupled code The code solves the business problem
Tests: Individual units The system as a whole
Tells you: Where the code is failing The code is failing

How They Compliment Each Other

Both automated acceptance tests and unit tests allow developers to implement change.  Change is inevitable and we, as professional software developers, need to build software that can adapt to change.  In fact, as professional developers, we need to initiate change.  Refactoring to better design is / should be a common activity.

The Role of Automated Acceptance Tests

If you stand back and think about what we, in IT, are trying to achieve it is this:

  1. The business defines the requirements of the system
  2. The architects / developers build a system that meets those requirements

The challenge we have historically had is in unifying the requirements with the system actually being built.  The requirements may be living in several places:

  • A high-level requirements document written by a business user
  • Technical specifications written by a business analyst
  • Design documents written by an architect or developer
  • etc

These documents can easily become out of sync with the software. How many times have you started a project with a requirements or specifications document as your guide. As development progresses, we learn more and the requirements naturally change.  Unfortunately, the documents don’t.  They quickly become obsolete.  It requires a PM with obsessive compulsive tendencies to keep everything synchronized.

With automated acceptance tests, we can capture features and specifications of the system within the system.  Best of all, we have acceptance criteria for those specifications directly within the system and they are executable as tests.  The tooling today is getting better and better.  Tools like Specflow are leaps and bounds over where we were years ago.

What Don’t Automated Acceptance Tests Do?

Referring back to the table above, you can see that the specifications are written by the business and the acceptance criteria are there to ensure that the code does what the business intends it to do.  What they are not all that good at are telling you where the code is failing or providing fast feedback.  They also don’t help ensure that the design is decoupled and maintainable.  Those are things that unit and integration tests are good at.

The Role of Unit Tests

To my thinking, unit tests provide 4 services:

  1. Guides us toward writing decoupled, testable, maintainable code
  2. Proves that the subject under test does what it is designed to do
  3. Allows us to refactor code
  4. Allows us to adapt to change

Perhaps the biggest  benefit of unit tests is that they guide us toward writing testable, decoupled code.  I’m sure most of us have been forced to maintain highly coupled code where a change to one method has rippling effects.  The kind of code where a seemingly simple change will take days or weeks and not minutes.

Numbers 2, 3 and 4 talk to the other main purpose of unit tests: to ensure that the code does what you, as the developer, expect it to.  With adequate test coverage, you have the confidence to refactor code, knowing that you won’t introduce a regression. 

While automated acceptance tests will (hopefully) tell you that you have introduced a regression, it is less likely that it will tell you where.  Furthermore, automated acceptance tests normally take much longer run than unit tests.  Unit tests are about fast feedback.

My Unit Tests Passed, but My Code Is Broken

What happens when all of your unit tests pass, but there is a bug in the application?  One of two things is going on.  The issue has to do with the integration between components and you are missing an integration test or your unit tests are not testing the correct thing.  It is possible, even likely, to have high unit test coverage, yet not be testing the right behaviour.  Testing is every bit an art form as software development.  It takes time and thought to get it right.

Both are Tools in the Toolbox

I doubt very much that if you spoke with a good carpenter you would hear him/her say anything like “I used to use claw hammers, but now I only use sledge hammers.  They get the nails in much quicker.  Claw hammers just slow me down.”  No.  They keep both in their toolbox and use each for the job it is best at.

Comments

One Response to “Acceptance Tests Don’t Replace Unit Tests”
  1. Hamish Graham says:

    Nice post Rob! I think spending time clarifying the differences and advantages here is important as it can get confusing. I am currently on a project with the opposite problem I think, thousands of unit tests but no integration or acceptance tests.

Speak Your Mind

Tell us what you're thinking...
and oh, if you want a pic to show with your comment, go get a gravatar!

Demystifying The Code