Using Virtual Machines to help me test

Tableau gives me a couple of ways to maintain virtual machines for a test environment.  My preferred method is probably seen as "old school" at this point.  I create machines locally to run on my nice desktop machine and use them in Hyper V.  This gives me the most control of my machines, they are always accessible and I can maintain them when and how I prefer to do so.  Right now, since I am still setting them up, I have a 32 bit Windows 7 machine, a 64 bit Windows 7 machine and a Debian machine.

We also use Openstack more globally and I would be willing to bet most folks around Tableau use it by default.  It has its own advantages - I can just select what OS and such I want, and the server builds a machine for me. I can log in when it is done, it doesn't consume any resources from my desktop and I can otherwise use it as needed.  I have not had any problems with this system at all: I simply prefer to run my machines locally since that is the first way I was introduced to using VMs for testing and development.

Informally, I once did a little experiment to judge how much faster I could be as a tester with more machines.  At one point, when I was testing email clients and servers, I had 14 total machines set up in my office running many different languages of clients and servers to validate email sent and received in these situations.  I was able to go about 50% faster by having all these machines under my control than by relying on labs or others to set up and maintain them.  I could also make any changes I wanted whenever I wanted since I was the only user.  The downside was learning to be a client and server and network administrator since I had to do all the work myself.  That probably paid for itself in the long run - the better I know my product overall the better my testing will be - but I did not factor that into my calculations of spending about 1-2 hours per day tweaking all the machines.  One last advantage with this many machines is that while updating them took quite a bit of time, since I updated them one after the other, one machine was almost always ready for testing while the rest were being upgraded.

And now, back to work!

Questions, comments, concerns and criticisms always welcome,

John

Hackathon week

This is a hackathon week for us here at Tableau.  We get to work on self-directed projects, things we "know need to get done", explore new avenues and so on.  Overall, it is a very productive time and most folks look forward to this regular activity.

For this week, I am working on a simple (think V1) "fuzzing" tool to dirty the Superstore sample workbook we ship.  In this context, fuzzing means to dirty some of the fields in the data to reflect bad data that may get sent to Tableau.

An example of this is a storm damage spreadsheet I encountered while taking a data cleaning class.  This spreadsheet (since it is still in use to my knowledge I won't link to it) was a hand created list of major storms in the USA dating back to the 1950s.  When I hear the phrase "hand created" I have learned to shudder.  There is simply the potential for a type to creep in and skew the data (at best) or result in completely invalid entries (much worse).  For instance, this particular spreadsheet had 54 states listed.

A fair question to ask would be "How many states would I expect to see?"  50 is an obvious answer, since we currently have 50 states.  But that wasn't true when the data was first starting to be collected.  But we have 50 now, so 50 might be a reasonable first guess.  The District of Columbia is not a state, but is often included in data gathered by the government, so 51 "states" may also be a good expectation.

Since this is storm data, I naturally include hurricanes in this and that makes me think of the Caribbean.  Now I have to wonder if I need to include Puerto Rico in my analysis, which would raise the state count to 52.

Finally, now that I had some expectation about 50 not necessarily being the absolute accurate value I expected I began to dig through the data.  In this case, DC was included, but Puerto Rico was not.  The other three "states" were typos - things like "Florida" being spelled "Flarida" and similar.  (Quick - how many ways can you type valid names or abbreviations for "Florida"?) Once I had that cleaned up, I was down to 51 states (50 states + the District of Columbia) for my analysis.

Anyway, remembering that lesson made me want to contribute to this effort.  Data is not always (never?) clean to start.  This hackathon I am trying to help solve that challenge.

Questions, comments, concerns and criticisms always welcome,

John

Expanding unit tests for Tabpy - the Python connector for Tableau

Now that Tabpy has a unit test framework (thanks Bora!)  I decided to add a few unit tests to check for error handling in the event of malformed URLs and such.  This was a relatively straightforward task and, like many test tasks, uncovered a defect in the code.

In this case, we allow for a timeout setting - a set number of seconds to wait before "giving up" (I'm simplifying here).  The documentation for that command said that the number of seconds you tell it to wait cannot be None.  When I was looking at the unit tests here, I saw there was no check for None so I added it.  Then I thought about it a little more and realized that I could pass in a negative amount of seconds to wait.  That makes no sense at all, so I also added a check for that condition.  So at about line 90 (in current versions) of \tabpy-client\client.py, you can see the code I added:

        if query_timeout is not None and query_timeout > 0:

            self.query_timeout = query_timeout

        else:

            self.query_timeout = 0.0

The unit test for that change is also pretty simple:

        client = Client(

            endpoint="endpoint",

            query_timeout=-10.0,

            verify_certificate=False)

        self.assertEqual(client.query_timeout,0.0)

If you haven't installed Tabpy feel free to hop over there and give it a shot.  It's also now available on Pipy so you can pip install tabpy if you want.

And please add some more testing if you like!

Questions, comments, concerns and criticisms always welcome,

John

New team name!

For a while now, we have been contemplating renaming our team here at Tableau.  We have (had) been the Statistics team since I started.  While that is a great name, it really did not reflect our day to day work.  We use statistics for all that we do, but we don't create new algorithms or anything like that.  It would be roughly akin to calling a team that works on cars the Algebra team if you use algebra to compute mileage or whatever.

So after thinking about it for almost a year (!) we finally settled on the Advanced Analytics team as our name.  We're still getting used to it and I hope it starts to roll of the tongue easily as time goes by.  Our mission is the same - to help you see and understand your data - so the day to day work doesn't change.  But it does help define our role a little better for both internal teams and even when we partner with outside folks.

So here's to Advanced Analytics!

Questions, comments, concerns (and a cool nickname) always welcome,

John

Some comments about comments in code

I was looking at a code review recently and was reminded of the old piece of advice I once received:  "God comments in code do not explain what the code does, they explain why the code does what it does."

I agree completely.  I was looking at some code to validate we use the correct font.  The code would change from normal, to Light, to Bold, etc… and preceding each of those commands was a comment that said "Changing to Light", "Changing to Bold, " etc..  For a simple task like this it is not too hard to figure out why the font is being changed.  In this case, we want to validate the Bold font, the Light font and so on.  But when code gets trickier, the why the code exists becomes much more important.

Oh, one other thing that drives me up a wall.  Typos in comments.  Seriously - if I am looking for the word "Bold" then I want to find that word.  If someone had spelled it "Blod" then search would fail and life can become quite difficult if I need to change the Bold font behavior everywhere it is used.  Fortunately, this doesn't happen very often, but when I see it, I generally try to make a code change to spell the word correctly. 

Questions, comments, concerns and criticisms always welcome,

John

Snow days mean working from home

While I am a big fan of snow days in general, this week in Seattle has been tough.  It snowed surprisingly hard for a while and I was unable to make it to our offices on Monday.

While this is not a terrible situation, it meant I had to work from home for the day.  I'm not personally a big fan of working from home.  There are several reasons for this and the biggest is probably that "home is where I go when I am done working."  Mentally, it is tough for me to stay focused when I am home.

Another reason is the limited hardware I have at home, specifically multiple monitors.  I'm lucky enough to have 2 monitors at my desk (three if you count my laptop) at work.  That makes debugging and testing in general much easier - run Tableau on one monitor, debug on the other.  I've written a few articles about this in the past and  I do not have this at home.  Typing on one monitor is simply frustrating for the work I do.

I also miss the interaction with my team.  Sometimes I just need a simple question answered which is pretty fast if folks are around me.  At home, I have to rely on instant messaging, email or whatever and the delay can leave me blocked for a while.

Finally, my home network is not as fast as our work building.  Frustrating.

The task I CAN do at home is paperwork.  Updating documentation, reading designs, giving feedback, etc… are the types of work I can complete at home and not feel distracted.  As luck would have it, I had about a day's worth of reading and online training to complete while snowed in, so the day went pretty well.

Now, back to the office and (real) work!

Questions, comments, concerns and criticisms always welcome,

John

Getting Good Estimates

We use Microsoft Team Foundation Server for tracking our activities during our sprints.  You can jump over there to read up a little about this if you have never seen it.  One of the areas that we are focusing currently is estimating how much work we can realistically get done in our two week sprints.

We started tracking our work to the half day level a few months ago.  As an aside, there are several schools of thought on how to track this type of activity.  You can track by the hour or the day is one example, and there is even a difference of opinion for how to track items that will take more than 1 day.  Some folks say just use the best estimate you have, and others say always round up, and even debates about using the Fibonacci sequence for scheduling.

Where we have focused has been on two areas:

1. Simply creating a task for everything we need to do.
1. An example here is a couple of our folks are working on documentation for a new feature we are trying to start.  Each of them devoted X number of days to get the documentation done.  So far, so good.
2. The problem was that the entire team needs to read, review and respond to the documentation.  We had not devoted any scheduled time to that work in the past.  If we each need 1 or 2 days, that is a large percentage of a 2 week cycle.  We learned our lesson in the past so we devote dedicated time here.
3. Other examples would be blocking time for code reviews (more time if we know a particularly complex change will be attempted in the sprint), time for hackathons, etc…
2. Once we have those estimates, TFS gives us a summation of how much time we have and how much work we are adding to a sprint.  If we have 100 engineer days, for example, we should not try to take on 120 days of work. 
1. We have tried this in the past and it seldom works.  The extra work simply gets carried over to the next sprint, but having that long list makes prioritizing difficult. 
2. Plus, if we have a work item that partner teams need done, the state of it should accurately reflect when a change would be expected.  By pulling it into the current 2 week cycle, stakeholders may reasonably expect a change within 2 weeks.  We have worked to avoid this situation.

So now we have the correct number of work items, all with our best estimates attached on the first day of the sprint.  We take the highest priority items in first.  (This assumes we have an accurately prioritized back log of work items, which we have).  So far our burndown charts show that we have decreased our range of estimates by about 75% - we are 75% more accurate now than 2 months ago.  We'll keep iterating to get this number more accurate over time.

Questions, comments, concerns and criticisms always welcome,

John