Integration Testing, Part 7 The value of integration tests

I was reading this blog post this morning (Arlo works here, by the way) in which the author gives some advantages of a good test suite.  It is a great read and I want to use it as a springboard to summing up the benefits of integration testing.

In our example, we had one integration test that compared the written specification of a proposed muffler combined with a proposed engine.  If the engineers want to use a new muffler, we can immediately tell if it can handle the exhaust from the engine currently being used.  We don't need to pay to order a test muffler, pay to remove the old one, install the new one and then finally find out it doesn't work.  (In this example, we would actually damage the engine or some other physical component.  That seldom happens in the software world - just restore the previous build and keep going).  This test can save thousands of dollars and several days worth of work if the muffler + engine combination are not compatible.

The actual hardware test will also pay off.  If the specification is not right, or if the engine is operating outside of specifications, we can find that out before the car is sold.  To say the least, that would benefit customer satisfaction since the engine will work properly always.  You can imagine the downside to shipping a defective car - especially if good testing would have alerted us to the defect early enough to address it.

Thanks for following along with the notional integration testing thought experiment.  I'm going to get back to doing that right now!

Questions, comments, concerns and criticisms always welcome,

John

Tangential (get it?) post: I am digging into this precision based article on Code Project

I've been a loyal follower of the Code Project for quite some time and even written a few articles for them a while back.

It's no secret that my team here at Tableau takes precision and accuracy very seriously and I just discovered this article about that: https://www.codeproject.com/Articles/25294/Avoiding-Overflow-Underflow-and-Loss-of-Precision

Overflow and underflow are pretty well covered, but there are aspects of precision that are very hard to control.  I've been reading through this and thought I would share.  I'll finish up integration testing next time - this article was simply a very timely happenstance that I wanted to share.

Questions, comments, concerns and criticisms always welcome,
John

Integration Testing, part 6: Updating requirement based on integration testing

So now we have 2 types of integration tests giving us information on our muffler + engine system.  We are checking both the documentation (data sheets) of each component to ensure they will work together.  We also test the physical system itself to validate the real world behavior.  This way we can ensure the car will actually work after we build it.

There is one last aspect of this I want to cover.  There is a possibility that we actually decide to change engines or mufflers at the midpoint of the design process.  One reason for this could be the result of our testing.  If we notice that, at maximum engine RPM the muffler is at 100% capacity we may decide we need a larger muffler if the engine has a possibility of over-revving.

In this case, we need to back up a few steps and update our documentation.  If we move from Muffler Model 100 to Model 200, we need to update our specifications that we use in our computer automated test.  We have to take into account the different capacity of the new muffler - we clearly want to ensure the model of muffler we are committed to using has valid statistics.

We also will need to update our physical test as well, and this may be a little easier to understand.  If the muffler has a different size, we may need to move our sensors, the mounting bracket for the sensors, and so on.  We may need to change our tolerance levels as well.   If we had set a warning that the engine was exceeding the capacity of the old muffler, we will need to change that setting to reflect the capacity of the new muffler, for instance.

At this point we now have updated specifications, updated tests and can quickly validate how well the muffler + engine work together.  Had we missed either step in updating the changing requirements, we run the serious risk of not being able to tell that the engine + muffler would not work. 

I'll cover the costs of finding these errors when I sum up next week.

Questions, comments, concerns and criticisms always welcome,

John