One of the tests we
have to complete is validating our implementations for accuracy. As I have mentioned, this can be tricky
simply because we are using computers and have to contend to with rounding
errors. Another reason this is hard is
simply the nature of statistics.
Consider the
meteorologist. Given a set of weather
statistics - temperature, barometric pressure, wind speed, etc… - the
meteorologist can state "There is a 70% chance of rain tomorrow."
Tomorrow comes, and
it rains. Was the forecast correct? Of course - there was a 70% chance of
rain. Now suppose tomorrow arrives, and
it does NOT rain. Again, was the
forecast correct? Of course - there was
a 30% chance that it would not rain.
Such is the nature of statistics, and that also hits us for some of our
work.
Recently, we added a
Clustering
algorithm to Tableau. The basic idea can
be viewed as "Given a set of data points like this, divide them into equal
size groups:"
In this case, I
tried to obviously draw three clusters, with about the same number of dots in
each.
But what about this
case:
Same number of
dots. Most people would probably want 2
groups, but that would probably look like this:
The group on the
right would have more dots, but visually this seems to make sense. Using three groups would give this:
Now they all have
the same number of dots, but the groups on the right are very close together.
The bigger question
to answer, just like the meteorologist faces, is "What is
correct?" And when predicting
situations like this, that is a very difficult question to answer. One of the challenges for the test team is
creating data sets like the dots above that can let us validate our algorithm
in cases which may not be as straightforward as some others. In the test world, these are "boundary
cases," and getting this data for our clustering testing was a task we
faced.
Questions, comments,
concerns and criticisms always welcome,
John
No comments:
Post a Comment