Paying down test debt

Another aspect of my work recently has been paying down technical debt we built over the years.  An example of technical debt would be this:

1. Imagine we are building an application that can compute miles per gallon your car gets
2. We  create the algorithm to compute miles per gallon
1. We add tests to make sure it works
2. We ship it
3. Then we are a hit in the USA!  Yay!
4. But the rest of the world wants liters per 100 kilometers. 
5. We add that feature
1. As we add it, we realize we need to change our existing code that only knows about miles
2. We figure it will take a week to do this
3. During this week, we disable the tests that test the code for "miles"
4. We finish the liters per 100km code
5. We check in
6. We ship and the whole world is happy

But look back at step 5c.  The tests for miles (whatever they were) were disabled and we never turned them back on.  We call this "technical debt" or, in this case since we know it is test related, "test debt."  It happens when we take shortcuts like 5c - disabling a test.  I'll just point out a better practice would have been to ensure every bit of new code we wrote for the metric values should never have broken the MPG code, and the test should never have been disabled.  In the real world, the most likely reason to do this would be for speed - I simply want to test my new code quickly and don't want to run all the tests over the old code I am not changing, so I disable the old tests for right now and will re-enable them when I am done.  (Or so I say...)

So one other task I have taken on is identifying these tests that are in this state.  Fortunately, there are not many of them but every so often they slip through the process and wind up being disabled for far longer than what we anticipated.  Turning them back on is usually easy.  Every so often, an older test won't pass nowadays because so much code has changed while it was disabled.

What to do in those cases is a little trickier and I will cover that next.

Questions, comments, concerns and criticisms always welcome,

John