Friday, July 21, 2017

Paying down test debt, continued

Last week I mentioned that some old test automation breaks while it is disabled.

As an example, suppose I added a test to check for the 22 Franch regions being labelled properly back in 2014.  It works for a year, but then France announces it will consolidate its regions in January 2016.  While working on that change, I disable my test since I know it won't provide any value while the rest of the changes are in progess.

Then I forget to turn the test back on and don't notice that until after the change.

In this case, the fix is straightforward.  I change my test to account for the real world changes that happened while it was disabled.  In this case, I take out the list of the 22 regions and replace that list with the 14 new ones. 

This pattern - the code being tested changes while the test is disabled - is common.  In almost all cases, simply changing the test to account for the new expected behavior is all that needs to be done to enable the test.  So I typically make that change, enable the test, run it  a few thousand times and if it passes, leave it enabled as part of the build system moving forward.

Sometimes the tests are more complicated that I know how to fix.  In that case, I contact the team that owns the test and hand off the work to enable it to them.

All in all, this is a simple case to handle. 

There is also the case that the test is no longer valid.  Think of a test that validated Tableau worked on Windows Vista.  Vista is no longer around, so that test can simply be deleted.

Other factors can change as well, and I'll wrap this up next week.

Questions, comments, concerns and criticisms always welcome,

No comments:

Post a Comment