The Cost of Test Driven Development

** Want to try TDD, but don’t know how to start? Watch our webinar “Intro to TDD“ **

Test Driven Development (TDD) is well known and practiced methodology that reduce the amount of defects introduced during the coding stage of the development process. It is hard to determine how much money using TDD saved for an organization.

After searching the web for research I found a paper on the subject titled: Realizing quality improvement through test driven development: results and experiences of four industrial teams.

The research used four projects in IBM and Microsoft each project two teams were chosen – one team developed using TDD while the other team didn’t.

image
[Graphs taken from: Benefit From Unit Testing in THE REAL WORLD read about it here and here]

Although Using TDD took more time (15%-35%) the defect density (defects per feature) decreased tremendously!

If for example IBM Driver team that used TDD took 20% more time to code a certain feature it was well worth – they had 39% less defects then the team that finished before them, and in Microsoft’s teams the benefits are even greater.

So What?

One can argue that the results doesn’t mean anything, even if a dev team has more defects those defects will be caught during QA and integration testing.

There is a known fact in software development – the longer it takes to find the bug the more it would cost to fix it.

image

Prevention defects at the coding stage saved the cost that fixing the same defect during the testing/support stage.

Another factor is the fact that defects found during testing really sends us back to the coding stage which means that although adding a feature without using TDD took less time initially in the long run when adding the QA –> Dev cycles it could take more.

An last during QA stage some more defects are introduced due to fixes of other defects (some claim that for every 3-10 bug fixes a new bug is added) – using TDD creates an integration test base that should prevent this from happening.

The Bottom Line

The research proved two points:

  1. Using TDD reduce the amount of bugs in the code significantly
  2. Using TDD takes more time then not using TDD

And I hope that after reading this post you understand that the amount of time invested in TDD actually well worth it due to the defect decrease and the money saved due to a quality increase of the code.

Claim your FREE trial of Typemock Isolator Complete - and be on your way to TDD nirvana. 

  • Peter Kofler

    I absolutely agree with you that TDD has advantages and I like to do it myself. But the graphs have a problem: They compare time with defects. But we need to compare time with time. So the question is how much time does fixing the defects that remain in non TDD project cost? Is it less or more than the overhead induced by TDD?

    Is there more data available on the parallel projects, because we would have to compare maintenance times too.

  • Olivier

    Fixing a bug in TDDis really easy. Fixing a bug without tests is really hard and incurs the risk of introducing new defects, as the post mentioned. It's already a stretch to compare different projects, but comparing the time to fix hypothetical defects, you might be asking too much. i too would love to see this data, but it would take experiments, not just observation of production projects.

  • Pingback: TDD :: การลงทุนมีความเสี่ยง แต่คุ้มพอที่จะลงทุน ?

TOP