unit testing

Rich recently discussed unit testing, an important aspect of his coding practice (slides; sample code). So what is unit testing and why is it important? The idea of unit testing is to write a collection of tests in parallel to writing code. The tests are on units, defined as the smallest piece of testable code, and they confirm that the units are fit for use.

You can probably convince yourself that you should be doing unit testing by considering a single example. A researcher was forced to retract several high-profile papers after discovering that a change of sign in his code resulted in an incorrect determination of the chirality of a protein. If only he and his colleagues were in the habit of unit testing!

Other motivators includes that it’s faster to unit test (the unit testing tortoise and the cavalier hare). It allows you to discover bugs while you’re familiar with the code, and reduces the number of them. It allows you to reuse code more, in part by creating evolving documentation. It makes it easier for you to alter code, for example, for optimization and refactoring (here a connection to agile software development).

Thoughts on the form that the tests should take include that you could compute expected output for a simple input, a test case (a MATLAB video on making test data). You could do boundary analysis. You could compare to alternative computation (e.g. numerical). And for random numbers, you could consider their coverage (save, display or fix the seed). Lastly, make it clear what’s being tested, and verify one condition per test.

Tools for doing it include xUnit and docTest. They can be used for organizing, running and interrogating unit tests. Here is another article on automated software testing for Matlab.

One person responded to the idea of unit testing with a soft jab, which is that one could imagine writing unit tests for unit tests, a kind of regress that most people didn’t seem overly concerned with. Someone also pointed out that Matlab is less good than it could be in that some of the checks that one would associate with unit testing could be built in. The point was also made that you only want to test important things, not trivial issues such as wrongly typed inputs, and that you shouldn’t consider the impossible task of checking things exhaustively.