One of the most important things you can do is move your tests up the abstraction layers and away from unit tests. For lack of a better term, to move to integration tests. End-to-end tests are often too far from the system to easily understand what's wrong when they break, and can overwhelm a development org. Integration tests (or whatever you want to call them) are often the sweet spot: not tied to a particular implementation, able to survive fairly significant system changes, but also easy enough to debug when they break.
If you are having to refactor 150 things each time you change your codebase then maybe you need to refactor your test suite first. Direct calls in tests to constructed/mocked objects is usually something you can just stuff into a private method so you only need to change it in one place.
Not quite sure I agree with the conclusion of the tiers of testing section either. If a test suite takes a long time but still covers something useful, then just deleting it because it takes too long makes no sense. Yes, if you have a 'fastTests' profile that doesn't run it that could temporarily convince you your changes are fine when they aren't. But the alternative is just never knowing your change is bad until it breaks production instead of just breaking your CI prior to that point.
show comments
vitonsky
No, thanks. I already spent time to write tests while implementing a features, now I have a lot of tests that proof the feature is works fine, and I no more fear to make changes, because tests keep me safe of regression bugs.
The typical problems of any code base with no tests is a regression bugs, rigid team (because they must keep in mind all cases when code may destroy everything), fear driven development (because even team with zero rotation factor don't actually remember all problems they've fixed).
show comments
dcminter
How about you fix the flakey tests?
The tests I'd delete are the ones that just test that the code is written in a particular way instead of testing the expected behaviour of thr code.
show comments
avg_dev
idk, i never thought
> “it is blasphemy to delete a test”,
was ever a thing. i still don't.
if a test is flaky, but it covers something useful, it should be made not flaky somehow. if tests are slow, but they are useful, then they should be optimized so they run faster. if tests cover some bit of functionality that the software no longer needs to provide, the functionality should be deleted from the code and the tests. if updating a small bit of code causes many tests to need to be adjusted, and that's a pain, and it happens frequently, then the tests should be refactored or adjusted.
> Confidence is the point of writing tests.
yes, agreed. but tests are code, too. just maintain the tests with the code in a sensible way. if there is something worth deleting, delete it; there is no gospel that says you can't. but tests provide value just like the author describes in the "fix after revert after fix after revert" counterexample. just remember they're code like anything else is all and treat them accordingly.
show comments
dgunay
I see a lot of debate about the definition and merits of unit, integration, and e2e testing here.
For my workplace, we have recognized a few issues with an overreliance on unit tests.
When you have important behaviors and invariants enforced by your database, you mock it out at your own peril. This literally caused a bug to slip through to prod this week. Unit tests just don't help here.
We use clean architecture. There are pockets of our codebase where, through deliberate or accidental deviations from our architecture, stuff like e.g. controllers with business logic in them exists. In some cases it is easier to just integration or e2e test this code instead of do the ugly refactoring to bring it into compliance. Doing the test first will even make the refactor easier.
Parts of our codebase are just big, pure functions. Arguments go in, return values come out. These are the ideal candidates for unit testing, and we do so extensively because they're cheap and fast.
I think what occurs to me as I write this is that if you live in an idyllic codebase which can express every single state transition in memory, without any dependency on external systems, sure, unit tests are great. For those of us who are not so lucky, e2e tests can be a lifeline and a way to maintain control with minimal mock-induced churn in the test suite.
bubblebeard
The author has a point. Obsolete tests serves no one, but deleting a test because it will randomly fail is an indication of an unstable process. Maybe there is a race condition, maybe your code has some dependency that is sporadically unavailable. Deleting such tests is just turning a blind eye to the problem. Unstable tests means you either didn’t write that test very well to begin with, or the process you are testing is itself unstable.
jampa
I work in an app where bugs are unacceptable due to the nature of the company's reputation. We've been having a lot of success with E2E, but getting there was NOT easy. Some tips:
- False negative results will make your devs hate the tests. People want to get things done and will start ignoring them if you unnecessarily break their workflow. In the CI, you should always retry on failure to avoid flaky false-negative tests.
- E2E Tests can fail suddenly. To avoid breaking people's workflow, we do a megabenchmark every day at 1 AM, and the test runs multiple times - even if it passes - so that we can measure flakiness. If a test fails in the benchmark, we remove it from the CI so we don't break other developers' workflows. The next day, we either fix the test or the bug.
- Claude Code SDK has been a blessing for E2E. Before, you couldn't run all the E2E in the PR's CI due to the time they all take. Now, we can send the branch to the Claude Code SDK to determine what E2E tests should run.
- Also, MCPs and Claude Code now write most of my E2E. I wrote a detailed Claude.md to let it run autonomously --writing, validating, and repeating -- while I do something else. It does in 3 to 4 shots. For the price of a cup of coffee, it saves me 30-60 minutes per test.
show comments
sitkack
What a poorly written article. I should delete my tests because they fail randomly. My tests don't fail randomly.
4ndrewl
> If your test is creating confidence in broken code with failing tests, it would be better for it to not exist.
The author never considers the other option of fixing the flaky tests. I find this odd.
teiferer
All seems like "fix tests" is the better advice.
Flaky test? Fix it! Make it rock solid! Slow test? Fix it! Make it fast! That can be hard (if it was easy, people would have already done it), but it's vastly more useful than deleting.
Even the mentioned overtesting requires a fix by focusing the tests on separate things. You could call that "deleting" but that's oversimplifying what's going on. Same with changed requirements.
egeozcan
At one of the companies I worked with when I was doing consulting, they could make the slow tests, which used to take around 3 hours, run much faster and in parallel by throwing engineering and hardware resources at the problem. First it was 30 minutes, then it was 10, then around 2-3 minutes.
I think it was one of the best investments that company made.
So my point is, don't delete slow tests, just make them fast.
nine_k
Un-clickbaiting the title: "Delete useless tests".
I once faced a suite of half-broken tests; so many were broken that engineers stopped caring if their changes broke another test. I suggested to separate a subset of still-working, useful tests, keep them always green, and make them passing a required check in CI/CD. Ignore the rest of the tests for CI/CD purposes. Gradually fix some of the flaky or out-of-sync tests if they are still useful, and promote them to the evergreen subset. Delete tests that are found to be beyond repair (like the article suggests).
This worked.
throwmeaway222
delete all mocked tests imo
mock exists
call exists
assert exists was called one time
so so so useless so that you can increase your coverage. just move to integration tests
simianwords
A good heuristic for a test is: how many times you are having to fix it when you change real code.
If every small change in the code base causes you to go back and fix the tests then your tests are bad. They should not get in the way so often. There should be a concept of “test maintenance overhead” that is weighted against the number of bugs it catches. You could also think of it as false positives vs true positives.
huflungdung
Contrarian blog post trying to challenge the status quo without understanding the implications in order to look like a visionary.
I say this in interviews just to look smart. And people think it’s revolutionary. Everyone loves an outspoken opposition -what do they know, where did they get this knowledge?!
Only a couple have ever pushed back and those that do are the companies that I want to work with.
A wild example, let’s delete a test that ensures a heart pump works at the correct duty cycle given its parameters. Now someone comes along and redefines milliseconds to microseconds for some unrelated component. The tests are all fine. Patient now has a 60000 bpm heart rate.
Stupid idea.
rustystump
At the end of the day, you need to have some kind of way to know if shit work or dont. This article feels a bit contrived to make an edgy point of “delete the test” which feels like it misses the real why behind testing.
efitz
I have had a weird thought lately about testing at runtime. My thought is just to log violations of expectations- i.e. log when the test would have failed.
This doesn’t prevent the bug from being introduced but can remove a huge amount of complexity for test cases that are hard to set up.
show comments
rotbart
So... clickbait title for an article that could have been called "Delete flakey tests"...but then and most of us would have just gone "yep" and not clicked.
gijoeyguerra
I've always deleted tests. I've never heard anyone say not to delete tests.
show comments
mirekrusin
We add .skip and QAs are taking over in the background to address those issues.
imiric
I'm a big believer in the utility of tests, and I do think the author has a point. There is a time and place when a test is not useful, and should be deleted.
However...
> If the future bug occurs, fix it and write a new test that doesn’t flake. Today, delete the tests.
How is this different from simply fixing the flaky test today?
Tests are code, and can also incur technical debt. The reason the test is flaky is likely because nobody is willing to take the time to address it properly. Sometimes it requires a refactoring of the SUT to allow making the test more reliable. Sometimes the test itself is too convoluted and difficult to change. All of this is chore work, and is often underappreciated. Nobody got promoted or celebrated for fixing something that is an issue a random percentage of times. After all, how do we know for sure that it's permanently fixed? Only time will tell.
But the flaky test might still deliver confidence and be valuable when it does run successfully. So deleting it would bring more uncertainty. That doesn't seem like a fair tradeoff for removing an annoyance. The better approach would be to deal with the annoyance.
> What if your tests are written so that a one line code change means updating 150 tests?
That might be a sign that the tests are too brittle, and too "whiteboxy". So fix the tests.
That said, there are situations when a change does require updating many tests. These are usually large refactors or major business logic changes. This doesn't mean that the tests are and won't be useful. It's just a side effect of the change. Tests are code, so fix the tests.
I've often heard negativity around unit tests, from programmers who strongly believe that more utility comes from integration tests (the inverted test pyramid, etc.). One of the primary reasons is this belief that unit tests slow you down because they need to be constantly updated. This is a harmful mentality, coming from a place of laziness.
Tests are code, and require maintenance just as well. Unit tests in particular are tightly coupled to the SUT, which makes them require maintenance more frequently. There should also be more unit tests than other types, adding more maintenance burden. But none of these are reasons to not write unit tests, and codebases without them are more difficult to change, and more susceptible to regressions.
> What if your tests take so long to run that you can’t run them all between merges, and you start skipping some?
That is an organizational problem. Label your tests by category (unit, integration, E2E), and provide quick ways to run a subset of them. During development, you can run the quick tests for a sanity check, while the more expensive tests run in CI.
There's also the problem of long test suites because the tests are inefficient.
Again: *fix the tests*.
> Even worse, what if your business requirements have changed, and now you have thousands of lines of tests failing because they test the wrong thing?
That is a general maintenance task. Would you say the same because you had to update a library that depended on the previous business logic? Would you simply delete the library because it would take a lot of effort to update it?
No?
Then *fix the tests*. :)
mattlondon
Don't delete flaky tests, fix them.
juped
I think all the listed reasons are good reasons to delete tests. I like to keep the test suite running in a single-digit number of seconds. (Sometimes a test you really need takes a while, and you can skip it by default and enable it on the CI test runner or whatever.)
Another one I really agree with is "What if your tests are written so that a one line code change means updating 150 tests?". If you update a test, basically, ever, it's probably a bad test and is better off not existing than being like that. It's meant to distinguish main code with errors from main code without errors; if it must be updated in tandem with main code, it's just a detector that anything changed, which is counterproductive. Of course you're changing things, that's why you fenced them with tests.
jessekv
I hate to admit it, but flaky tests almost always highlight weaknesses in my software architecture.
And fixing a flaky test usually involves making the actual code more robust.
codeulike
But before you delete the test, write a test that tests whether the test is deleted, and make sure that test is failing as expected. Then delete the test. Then run the other test that makes sure the test is deleted and it should now pass /s
One of the most important things you can do is move your tests up the abstraction layers and away from unit tests. For lack of a better term, to move to integration tests. End-to-end tests are often too far from the system to easily understand what's wrong when they break, and can overwhelm a development org. Integration tests (or whatever you want to call them) are often the sweet spot: not tied to a particular implementation, able to survive fairly significant system changes, but also easy enough to debug when they break.
https://grugbrain.dev/#grug-on-testing
If you are having to refactor 150 things each time you change your codebase then maybe you need to refactor your test suite first. Direct calls in tests to constructed/mocked objects is usually something you can just stuff into a private method so you only need to change it in one place.
Not quite sure I agree with the conclusion of the tiers of testing section either. If a test suite takes a long time but still covers something useful, then just deleting it because it takes too long makes no sense. Yes, if you have a 'fastTests' profile that doesn't run it that could temporarily convince you your changes are fine when they aren't. But the alternative is just never knowing your change is bad until it breaks production instead of just breaking your CI prior to that point.
No, thanks. I already spent time to write tests while implementing a features, now I have a lot of tests that proof the feature is works fine, and I no more fear to make changes, because tests keep me safe of regression bugs.
The typical problems of any code base with no tests is a regression bugs, rigid team (because they must keep in mind all cases when code may destroy everything), fear driven development (because even team with zero rotation factor don't actually remember all problems they've fixed).
How about you fix the flakey tests?
The tests I'd delete are the ones that just test that the code is written in a particular way instead of testing the expected behaviour of thr code.
idk, i never thought
> “it is blasphemy to delete a test”,
was ever a thing. i still don't.
if a test is flaky, but it covers something useful, it should be made not flaky somehow. if tests are slow, but they are useful, then they should be optimized so they run faster. if tests cover some bit of functionality that the software no longer needs to provide, the functionality should be deleted from the code and the tests. if updating a small bit of code causes many tests to need to be adjusted, and that's a pain, and it happens frequently, then the tests should be refactored or adjusted.
> Confidence is the point of writing tests.
yes, agreed. but tests are code, too. just maintain the tests with the code in a sensible way. if there is something worth deleting, delete it; there is no gospel that says you can't. but tests provide value just like the author describes in the "fix after revert after fix after revert" counterexample. just remember they're code like anything else is all and treat them accordingly.
I see a lot of debate about the definition and merits of unit, integration, and e2e testing here.
For my workplace, we have recognized a few issues with an overreliance on unit tests.
When you have important behaviors and invariants enforced by your database, you mock it out at your own peril. This literally caused a bug to slip through to prod this week. Unit tests just don't help here.
We use clean architecture. There are pockets of our codebase where, through deliberate or accidental deviations from our architecture, stuff like e.g. controllers with business logic in them exists. In some cases it is easier to just integration or e2e test this code instead of do the ugly refactoring to bring it into compliance. Doing the test first will even make the refactor easier.
Parts of our codebase are just big, pure functions. Arguments go in, return values come out. These are the ideal candidates for unit testing, and we do so extensively because they're cheap and fast.
I think what occurs to me as I write this is that if you live in an idyllic codebase which can express every single state transition in memory, without any dependency on external systems, sure, unit tests are great. For those of us who are not so lucky, e2e tests can be a lifeline and a way to maintain control with minimal mock-induced churn in the test suite.
The author has a point. Obsolete tests serves no one, but deleting a test because it will randomly fail is an indication of an unstable process. Maybe there is a race condition, maybe your code has some dependency that is sporadically unavailable. Deleting such tests is just turning a blind eye to the problem. Unstable tests means you either didn’t write that test very well to begin with, or the process you are testing is itself unstable.
I work in an app where bugs are unacceptable due to the nature of the company's reputation. We've been having a lot of success with E2E, but getting there was NOT easy. Some tips:
- False negative results will make your devs hate the tests. People want to get things done and will start ignoring them if you unnecessarily break their workflow. In the CI, you should always retry on failure to avoid flaky false-negative tests.
- E2E Tests can fail suddenly. To avoid breaking people's workflow, we do a megabenchmark every day at 1 AM, and the test runs multiple times - even if it passes - so that we can measure flakiness. If a test fails in the benchmark, we remove it from the CI so we don't break other developers' workflows. The next day, we either fix the test or the bug.
- Claude Code SDK has been a blessing for E2E. Before, you couldn't run all the E2E in the PR's CI due to the time they all take. Now, we can send the branch to the Claude Code SDK to determine what E2E tests should run.
- Also, MCPs and Claude Code now write most of my E2E. I wrote a detailed Claude.md to let it run autonomously --writing, validating, and repeating -- while I do something else. It does in 3 to 4 shots. For the price of a cup of coffee, it saves me 30-60 minutes per test.
What a poorly written article. I should delete my tests because they fail randomly. My tests don't fail randomly.
> If your test is creating confidence in broken code with failing tests, it would be better for it to not exist.
The author never considers the other option of fixing the flaky tests. I find this odd.
All seems like "fix tests" is the better advice.
Flaky test? Fix it! Make it rock solid! Slow test? Fix it! Make it fast! That can be hard (if it was easy, people would have already done it), but it's vastly more useful than deleting.
Even the mentioned overtesting requires a fix by focusing the tests on separate things. You could call that "deleting" but that's oversimplifying what's going on. Same with changed requirements.
At one of the companies I worked with when I was doing consulting, they could make the slow tests, which used to take around 3 hours, run much faster and in parallel by throwing engineering and hardware resources at the problem. First it was 30 minutes, then it was 10, then around 2-3 minutes.
I think it was one of the best investments that company made.
So my point is, don't delete slow tests, just make them fast.
Un-clickbaiting the title: "Delete useless tests".
I once faced a suite of half-broken tests; so many were broken that engineers stopped caring if their changes broke another test. I suggested to separate a subset of still-working, useful tests, keep them always green, and make them passing a required check in CI/CD. Ignore the rest of the tests for CI/CD purposes. Gradually fix some of the flaky or out-of-sync tests if they are still useful, and promote them to the evergreen subset. Delete tests that are found to be beyond repair (like the article suggests).
This worked.
delete all mocked tests imo
so so so useless so that you can increase your coverage. just move to integration testsA good heuristic for a test is: how many times you are having to fix it when you change real code.
If every small change in the code base causes you to go back and fix the tests then your tests are bad. They should not get in the way so often. There should be a concept of “test maintenance overhead” that is weighted against the number of bugs it catches. You could also think of it as false positives vs true positives.
Contrarian blog post trying to challenge the status quo without understanding the implications in order to look like a visionary.
I say this in interviews just to look smart. And people think it’s revolutionary. Everyone loves an outspoken opposition -what do they know, where did they get this knowledge?!
Only a couple have ever pushed back and those that do are the companies that I want to work with.
A wild example, let’s delete a test that ensures a heart pump works at the correct duty cycle given its parameters. Now someone comes along and redefines milliseconds to microseconds for some unrelated component. The tests are all fine. Patient now has a 60000 bpm heart rate.
Stupid idea.
At the end of the day, you need to have some kind of way to know if shit work or dont. This article feels a bit contrived to make an edgy point of “delete the test” which feels like it misses the real why behind testing.
I have had a weird thought lately about testing at runtime. My thought is just to log violations of expectations- i.e. log when the test would have failed.
This doesn’t prevent the bug from being introduced but can remove a huge amount of complexity for test cases that are hard to set up.
So... clickbait title for an article that could have been called "Delete flakey tests"...but then and most of us would have just gone "yep" and not clicked.
I've always deleted tests. I've never heard anyone say not to delete tests.
We add .skip and QAs are taking over in the background to address those issues.
I'm a big believer in the utility of tests, and I do think the author has a point. There is a time and place when a test is not useful, and should be deleted.
However...
> If the future bug occurs, fix it and write a new test that doesn’t flake. Today, delete the tests.
How is this different from simply fixing the flaky test today?
Tests are code, and can also incur technical debt. The reason the test is flaky is likely because nobody is willing to take the time to address it properly. Sometimes it requires a refactoring of the SUT to allow making the test more reliable. Sometimes the test itself is too convoluted and difficult to change. All of this is chore work, and is often underappreciated. Nobody got promoted or celebrated for fixing something that is an issue a random percentage of times. After all, how do we know for sure that it's permanently fixed? Only time will tell.
But the flaky test might still deliver confidence and be valuable when it does run successfully. So deleting it would bring more uncertainty. That doesn't seem like a fair tradeoff for removing an annoyance. The better approach would be to deal with the annoyance.
> What if your tests are written so that a one line code change means updating 150 tests?
That might be a sign that the tests are too brittle, and too "whiteboxy". So fix the tests.
That said, there are situations when a change does require updating many tests. These are usually large refactors or major business logic changes. This doesn't mean that the tests are and won't be useful. It's just a side effect of the change. Tests are code, so fix the tests.
I've often heard negativity around unit tests, from programmers who strongly believe that more utility comes from integration tests (the inverted test pyramid, etc.). One of the primary reasons is this belief that unit tests slow you down because they need to be constantly updated. This is a harmful mentality, coming from a place of laziness.
Tests are code, and require maintenance just as well. Unit tests in particular are tightly coupled to the SUT, which makes them require maintenance more frequently. There should also be more unit tests than other types, adding more maintenance burden. But none of these are reasons to not write unit tests, and codebases without them are more difficult to change, and more susceptible to regressions.
> What if your tests take so long to run that you can’t run them all between merges, and you start skipping some?
That is an organizational problem. Label your tests by category (unit, integration, E2E), and provide quick ways to run a subset of them. During development, you can run the quick tests for a sanity check, while the more expensive tests run in CI.
There's also the problem of long test suites because the tests are inefficient.
Again: *fix the tests*.
> Even worse, what if your business requirements have changed, and now you have thousands of lines of tests failing because they test the wrong thing?
That is a general maintenance task. Would you say the same because you had to update a library that depended on the previous business logic? Would you simply delete the library because it would take a lot of effort to update it?
No?
Then *fix the tests*. :)
Don't delete flaky tests, fix them.
I think all the listed reasons are good reasons to delete tests. I like to keep the test suite running in a single-digit number of seconds. (Sometimes a test you really need takes a while, and you can skip it by default and enable it on the CI test runner or whatever.)
Another one I really agree with is "What if your tests are written so that a one line code change means updating 150 tests?". If you update a test, basically, ever, it's probably a bad test and is better off not existing than being like that. It's meant to distinguish main code with errors from main code without errors; if it must be updated in tandem with main code, it's just a detector that anything changed, which is counterproductive. Of course you're changing things, that's why you fenced them with tests.
I hate to admit it, but flaky tests almost always highlight weaknesses in my software architecture.
And fixing a flaky test usually involves making the actual code more robust.
But before you delete the test, write a test that tests whether the test is deleted, and make sure that test is failing as expected. Then delete the test. Then run the other test that makes sure the test is deleted and it should now pass /s