Last week I
mentioned that some old test automation breaks while it is disabled.
As an example,
suppose I added a test to check for the 22 Franch regions being labelled
properly back in 2014. It works for a
year, but then France
announces it will consolidate its regions in January 2016. While working on that change, I disable my
test since I know it won't provide any value while the rest of the changes are
in progess.
Then I forget to
turn the test back on and don't notice that until after the change.
In this case, the
fix is straightforward. I change my test
to account for the real world changes that happened while it was disabled. In this case, I take out the list of the 22
regions and replace that list with the 14 new ones.
This pattern - the
code being tested changes while the test is disabled - is common. In almost all cases, simply changing the test
to account for the new expected behavior is all that needs to be done to enable
the test. So I typically make that
change, enable the test, run it a few
thousand times and if it passes, leave it enabled as part of the build system
moving forward.
Sometimes the tests
are more complicated that I know how to fix.
In that case, I contact the team that owns the test and hand off the
work to enable it to them.
All in all, this is
a simple case to handle.
There is also the
case that the test is no longer valid.
Think of a test that validated Tableau worked on Windows Vista. Vista is no longer around, so that test can
simply be deleted.
Other factors can
change as well, and I'll wrap this up next week.
Questions, comments,
concerns and criticisms always welcome,
John
No comments:
Post a Comment