We’re Writing Too Many Tests

Creating and maintaining a test automation suite is a time-consuming endeavor, yet the value of those efforts is often questionable.

Photo by Collab Media on Unsplash

Creating and maintaining a test automation suite is a time-consuming endeavor, often rivaling the time spent developing the code changes they are there to support. Yet the value of those efforts is often questionable. It’s hard to judge the value of test automation, and it’s surprisingly easy to invest a lot of time for no, or even negative, return.

Now, I don’t want to suggest that we shouldn’t write tests. Done carefully, they are a vital part of the development process; they increase confidence, help us catch bugs earlier in the development process, and make future improvements easier. Instead, I propose that we be more careful about how we write tests and what tests we choose to write. We need to ensure the effort in creating tests is balanced against the value we get in return.

Last year, I deleted our team’s end-to-end test suite. This suite had all the same problems as the last two e2e test suites I deleted. It wasn’t finding any valuable regressions. It was burning time investigating and fixing failures that all turned out to be flakes and had become difficult to add new tests to. A few months later, we started…