Tuesday, June 27, 2017

Working on a tool for a hackathon

We have a culture of regular time devoted to hackathons.  We can work on what we know is important or fun or challenging - we have free reign to take on the projects that we are motivated to complete.

For my project, I am working on classifying some of our test code.  What I have to do specifically is parse through each test file looking for attributes in the code.  The goal here is to make a tool that does this so I never have to do it again.

I've been working on this for a day now, and I am reminded why I want this tool.  I have written TONS of code that opens a text (type) file and goes through it line by line.  It is always tedious, slow and I always have to look up how to do these basic tasks on the internet.  Since I only do this once every year or so, I forget the exact syntax to use and need a continual refresher.

But I got my tool done yesterday and am making sure it works today.  Then I want to move it from a command line to a nice visualization that I can monitor for changes…

Questions, comments, criticisms and complaints always welcome,
John

Friday, June 16, 2017

Test names that make sense


One of the tasks developers have is adding a test when making code change.  That is just good engineering practice - you always want to make sure your code works, and then when I make a change, I want to test that I did not break your code.  It's pretty self-explanatory, really.

The trick here is that when someone fixes a bug report.  Bug reports are tracked by number, so I may be working on bug 1234 today.  When I get a fix in place, I need to add a test.  Now, when I add the test, I need to give the test a name.

One easy way to name the test is naming it after the bug number being fixed, like this:
Test_Bug1234()

That makes it possible for anyone else that needs to look at this code to know to check the bug database for details around bug 1234.  I chose the word "possible" there specifically because while it is possible to do this, it is time consuming.  I have to switch from my IDE (I use Visual Studio) to the bug tool and dig up the bug report. 

Now imagine if I had named that test this instead:
Test_AddNewFrenchRegionsToMaps()

Now if I am reading that code, or investigating a failure, I have a MUCH better starting point.  I know that the test I potentially broke had to with French regions and maps.  If I am changing map code, I am very interested in what I might have broken and know where to start my investigation much more quickly.  I don't have to switch out of my IDE to get this data and it saves me a little bit of time overall.

So while I am going through tests, I am not renaming the old format with a bit of descriptive text.  The next challenge I might take on is trying to quantify how much time I am saving overall.

Questions, comments, concerns and criticisms always welcome,
John

Tuesday, June 6, 2017

Hardware at Tableau


I just noticed that I was working in a remote office today and logged into my primary desktop from that office.  I also realized I never documented the great hardware we use at Tableau.

It may not seem special, but all the developers here get at least 2 machines: one Windows and one Mac.  We need this since our 2 primary desktop clients both need coverage.

I chose a Windows desktop and that is what I use for email and such as well as writing code for Tableau.  It's a state of the art 16 core (or 4 depending on how you count hyperthreads) 32GB desktop.  I also have 2 monitors on my desk - a 24" HD monitor and a 22" 4K monitor.  I have learned to rely on multiple monitors since way back in 1998 and can't imagine working with only one.  Brrrr.


Since I run Windows 10 on my desktop, I got a Mac laptop for portable usage.  Nothing special here - 16GB Ram and whatever processor they were using last year (I have never checked).  I use it for note taking in meetings and general office type usage.  If I need to write code or debug or whatever, I will remote into my desktop.

And finally, the docking station I have in the remote office is even better.  It has 2 monitors and I can use the laptop as a third monitor.  In effect, I get a three monitor setup when I work remotely and that is tremendously handy.  I put Tableau on one monitor, my debugger/Visual Studio/Pycharm on the second and email/chat clients/reference notes/OneNote on the third.  It really speeds me up and is a nice perk when I can't get into my main office.

Questions, comments, concerns and criticisms always welcome,
John

Thursday, June 1, 2017

An upcoming side project for the test team


We voted this week to dedicate an upcoming sprint to focus on becoming more efficient as a team rather than focus on any given new functionality for Tableau.  The thinking here is that if we become 10% more efficient, we can deliver 10% more features in a given release over time, so this small investment now will pay large dividends in the future.

The test team chose to work on analyzing automation results.  For instance, if a given test is known to fail some large percentage of the time - let's say 99.99% for sake of argument - then if it fails tonight I might not need to make investigating it the highest priority task on my plate tomorrow.  Similarly, a test that has never failed and fails tonight might very well become my most important task tomorrow.

So what we are doing in our first steps is determining the failure rate of every single test we have.  Just tying together all that data - years worth, times several thousand tests, times multiple runs per day, et… - is a large challenge.  Then we have to mine the data for the reason for each failure.  If the failure was due to a product bug, then we need to factor out that failure from computing how often each test intermittently failed.

The data mining and computation for all of this seems like a good, achievable goal for one sprint.  Using that data in a meaningful way will be the (obvious) follow on project.

Wish us luck!

Questions, comments, concerns and criticisms always welcome,
John 

Tuesday, May 23, 2017

Sharing lessons from moving test code around


I mentioned 2 weeks ago that I was moving some tests around within our codebase.  That work is still happening and will almost certainly continue for quite some time.

One other task I am taking simultaneously is quantifying the cost of moving these tests.  This ranges from a simple hourly track of my time to including the time others need to review the code and validating the tests achieve the same coverage once they have been moved.

I'm also taking a stab at quantifying how difficult moving a test can potentially be.  For instance, a traditional unit test that happens to be in a less than ideal location is a good candidate for almost a pure "copy and paste" type of move.  Since the test is sharply focused and doesn't have many dependencies, it is very simple to move around.

Other tests that start with loading a workbook in order to validate a column is being drawn correctly (I am making up an example) have many dependencies that have to be untangled before the test can be moved.  This is at best a medium difficulty task and can easily take a large amount of time depending on how tightly both the test and product code are woven together.

For now, I am making notes on how to untie those knots and moving the tests that are easy to move.  Once I am done with my notes, I intend to look through them for common patterns, good starting points and use that data to develop a plan to start untangling the next round of tests.   And of course I will share this with others since I doubt I will have enough time - or energy :) - to do all this myself.

Questions, comments, concerns and criticisms always welcome,
John

Monday, May 15, 2017

All Hands Week


This is a bit of an unusual week.  We have booked the Washington State Convention Center in downtown Seattle for our annual company meeting.  "All Hands" is the navy phrase that we use to show that the entire company attends - we go over business strategy, technical planning, development specific tasking, TableauConference planning and so on.

I can't write much about any of this (maybe not so) obviously.  This will be my second such event and I learned a lot of information last year.  Now that I know where to focus, I expect this year to be even better!

Otherwise, I am still moving unit tests to better locations.  The easy tests to move will likely fill this week for me and then next week the work gets more challenging.  Stay tuned!

Questions, comments, concerns and criticisms always welcome,
John

Wednesday, May 10, 2017

Moving unit tests to better locations


Last week I spent identifying and removing dead code.  For what it is worth, the biggest challenge there is proving the code is not actually used.  If you know of a way to tell if an operator overload is actually called, let me know…

This week I am focused on moving some of our unit tests to a more proper location.  Some of our older tests are part of a large module that runs tests all across the product.  For instance, suppose I want to test Kmeans clustering.  As it stands right now, I either have to work some command line magic to get just those tests to run, or I run that entire module which tests areas in which I am not interested (like importing from Excel).

A better place for the Kmeans test would be in the same module that holds the Kmeans code.  That way, when I run the tests, I focus only on testing the code in which I am interested and don't need to worry about Excel importing.  There are also some speed benefits when building the test code.  Right now, the old project has references all over the product.  It has to have those wide ranging references since it has such a wide variety of tests in it.  That means a lot of file copying and such during compile time.

One of the other benefits I expect to see when done is that the time to build that older test file will shrink because I am removing code from it.  As I move the code to the correct module, I am updating all the references it used to minimize the amount of references needed to build.  So my module build time will go up, but not as much as the time saved from the older test pass.

There is one final benefit to all of this.  When I build my module now, I build both that module and the older test code.  This is necessary since I need the testing provided there in order to test any changes being made in my module.  Once I am done with this task, I will only need to build my module in order to test it since all the tests will be part of the module.  I will no longer have to "pay the price" of  building that older test project.

Questions, comments, concerns and criticisms always welcome,
John