Filling holes in test coverage

One of the tasks in my plate is to find and fill holes in test coverage.  A good question - which I am going to avoid for now - is how features ship with holes in coverage.  Instead, I want to focus on holes that get created over time.

For instance, my team owns the Trend Lines feature in Tableau.  Here is a sample image we have used to test for years:

You can see the multiple trend lines drawn, each with a confidence band around it.  We validate that we do not alter this expected visualization with each new version of Tableau using a simple image comparison.  We also validate the math behind this feature but I want to focus on the image for now.  This test has been running and passing for years.

So how can debt accumulate with this scenario?  One way it cropped up was adding a version of Tableau that works on the Mac.  Initially, when this test was created, only a Windows version of Tableau existed.  Since that point, we have added Tableau for the Mac, Linux (via server) and different flavors of server (Windows and Linux).  So now we have to verify we draw correctly on each new platform.

The hole gets created when some new supported configuration is added.  When we added Mac support, we obviously ran all our tests.  In this case, the Windows test did not break since we changed nothing for Windows.  And because we did not add a test for the Mac, the non-existent test did not fail (again, obviously).  But we never added that test so a hole existed.  We are filling that gap now, as well as Linux tests.

Questions, comments, concerns and criticisms always welcome,

John

Updating the tasks I posted from 2 weeks ago

Two weeks ago I posted a laundry list of to do items.  We are an agile team with a  2 week sprint cycle, so it seems natural to follow up on these.

Again, in no particular order:

1. We are getting some beta feedback on our features and are working on fixes for those items.  --- I contacted the person that submitted a key piece of feedback and showed him a proposed change.  He thought it was much better, so remain tuned.

2. Some of our unit tests use integration points to set up before the test runs. 
1. This is ongoing work.  In the past 2 weeks, I have refactored 4 tests.
2. I also had to update a new test we added because the path to one of the files "is too long."  Sigh.

3. Spent 2 hours performing backlog cleanup.  Consolidating related items, assigning tasks to stories, etc…

4. I actually tested Tableau :)

5. I also had time to jump over to Tabpy on github and perform a code review and merge there.

FWIW, 2 and 4 will be focus areas for the next sprint.  Let me know if you find these updates interesting.

Questions, comments, concerns and criticisms always welcome,

John