Unit testing is dead? Hardly.

Despite what His Majesty, David Heinemeier Hansson may have said, unit testing is by no means dead. And, in fact, system testing is no more a complete testing strategy than 100% test coverage with unit tests. Let me explain.

Test Driven Development (TDD) is a philosophy that asserts testing is so important that the tests should be written first, to emphasize the design of the code. The idea is that by writing a failing test, and then writing code that passes that test, you end up with an overall better architecture.

Of course, TDD has some flaws: namely it’s a “write to the tests” design strategy that, while potentially encouraging better design, can potentially harm your overall design by removing “big picture thinking.”

The problem is, DHH has basically said TDD is worthless, and system testing, not unit testing, is the way to go. And yet, this philosophy has just as many blind spots as TDD does.

Socorro used an “integration test” methodology to accomplish the majority of it’s testing. And while this worked well to avoid regressions in many cases, it also had many holes. One such hole that frustrated me was the complete lack of tests for the stored procedures in the database: I would write a stored procedure, and a middleware to access the results, but because the stored procedure created data on a cron cycle, there was no way to validate it automatically. You can guess which parts of my code often broke on staging.

Of course, system testing has a place alongside unit testing. System testing ensures a high-level assessment of the overall application, and ensures that we are getting the right answer at a high level. It makes sure that all our components come together and do their jobs. Unit tests won’t catch failed or missing database migrations, or Javascript bugs.

Unit testing, on the other hand, provides an opportunity for us to get nitty-gritty and to test specific avenues of our code for accuracy and effectiveness. Unlike a system test, a well-written failed unit test points right to the source of the problem, making it easier to correct. A system test will tell you that the system is broken, but you have to hunt for the cause.

By all means, we should be writing more, not fewer, tests. And we should be testing whole systems as well as specific units of code. Testing is not an either-or situation whereby we can only do one or the other.

Brandon Savage is the author of Mastering Object Oriented PHP and Practical Design Patterns in PHP

gggeek (@gggeek) wrote at 4/23/2014 12:09 pm:

What I find surprising is that we’re still openly debating the value of tests, and the system vs. integration vs. acceptance vs. unit split, after many years of agile/tdd/scrumban…

I was having this very argument today with a colleague!

In my experience bad developers write bad tests, while good developers write good code even without tests.

Bad tests being defined (by me) as those which
– did not help the developer to clarify his mind and implement a solid api/architecture
– will get refactored with 75% probability or higher
– take more time maintaining than the time it would take to fix the bugs that they prevent

In the end, it seems that writing good tests is at least as hard as writing good code…

What if there was no spoon?

Ben wrote at 4/23/2014 3:14 pm:

What about behaviour driven development?

Brandon Savage (@brandonsavage) wrote at 4/23/2014 10:43 pm:

Same toolkit, different tool. :)

Marc (@http://www.mogosselin.com/mogosselin) wrote at 4/27/2014 3:45 pm:

High level tests that behave exactly like a browser are good but, like you said, have a lot of downsides too. Try to write a use cases where you need to make an order in different time zones and different hours of the day… Good luck. Way easier to do on a lower level (not necessarily only with “pure” unit tests).

Also, tests with Selenium (and similar tools) can be difficult to maintain, especially while you are developing an application. And, it’s even more difficult if the front-end team doesn’t know what the QA is doing. For example, if the QA writes code that is dependent on IDs and classes in the HTML, the tests can easily break. Yes, the “teams” should know what has an impact on the other team, but it’s not always that simple.

I found that in big teams, unit tests are easier to write and maintain because there is just one team involved (the devs) compared to high level tests. That being said, high level tests are useful too. You just need to target the right part of an application to test.

ks wrote at 5/4/2014 6:36 am:

“Why Most Unit Testing is Waste” by James Coplien is really interesting.

Copyright © 2023 by Brandon Savage. All rights reserved.