★ Unit testing framework
posted Thu 4 Jan 2007 by Michael Galloy under IDLMGunit is a unit testing framework modeled on other unit testing frameworks such as JUnit. The goal is to allow easy creation and reporting of results of tests, but still allow for all needed flexibility. More details and examples after the jump.
The structure of tests created for this framework is straightforward. Individual tests are methods of a class that subclasses MGtestCase
. Each method returns 1 for success or 0 for failure (or throws an error). Each test method’s name must start with “test”. Test cases may be (but are not required to be) grouped into test suites.
The examples (FINDGENTEST
, INDGENTEST
, and INDGENSUITE
) of MGunit discussed are available for download.
To create a test, subclass MGtestCase
like:
pro findgentest__define
define = { findgentest, inherits MGtestCase }
end
A test is just a method whose name starts with “test”. MGunit will find the tests automatically (make sure they are either already compiled or above FINDGENTEST__DEFINE
in the same file). A test case may have as many tests as necessary. For example, a simple test:
function findgentest::test2
a = findgen(5)
assert, array_equal(a, [0.0, 1.0, 2.0, 3.0, 4.0]), 'Correct elements'
return, 1
end
Return 1 for success. For failure, either return 0 or throw an error. Here the helper routine ASSERT
will throw an error using the given message if its condition is not met. This will be reported as a failure along with the message. To run this test, use:
IDL> mgunit, 'findgentest'
Results can be sent to a log file with the LOG_FILE
:
IDL> mgunit, 'findgentest', log_file='results.log'
This should produce the output:
Starting test suite MGTESTSUITE (1 test case, 4 tests)
Starting FINDGENTEST (4 tests)
TEST1: failed "Wrong number of elements"
TEST2: passed
TEST3: passedTEST4: failed "Type conversion error: Unable to convert..."
Results: 2 / 4 tests passed
Results: 2 / 4 tests passed
One tricky situation is that sometimes invalid input must be tested to make sure the routine fails. In this case, throwing an error should indicate a success of the test, not a failure. In this case use the provided batch file error_is_pass
at the beginning of the routine, like:
function findgentest::test3
@error_is_pass
a = findgen('string')
return, 1
end
Multiple test cases can be specified as an array
IDL> mgunit, ['findgentest', 'indgentest']
or by creating a test suite. Test suites are very simple; they are just collections of test cases. To make a suite, subclass MGtestSuite
and use the add
method to add test classes. For example,
function indgensuite::init, _extra=e
compile_opt strictarr
if (~self->mgtestsuite::init(_extra=e)) then return, 0
self->add, ['indgentest', 'findgentest']
return, 1
end
pro indgensuite__define
define = { indgensuite, inherits MGtestSuite }
end
This test suite is available for download (source / docs). MGUNIT
can run test suites also,
IDL> mgunit, 'indgensuite'
MGUNIT
will also accept a mixed array of test suites and test cases.
March 15th, 2007 at 8:12 pm
Great stuff Mike. I’ve finally started using unit test in production code.
It would be good to include the example from the post in the source code bundle. I keep needing to refer to this post everytime I get confused between a test case and test suite.
I found that I can run a test case by itself, but there is a bug in mgtestclirunnner. Not all methods accept the level keyword as 0.
March 15th, 2007 at 8:23 pm
I was also thinking of passing keywords from the mgtestsuite::add method to the mgtestcase::init method of the test case.
self->add, [‘indgentest’, ‘findgentest’], SETTING_A=3
…
function indgentest::init, SETTING_A=3
Perhaps this is not desirable because it would void the test logic being encapsulated in the test case class?
March 16th, 2007 at 11:52 am
Glad you find it useful!
Send me an example of the bug in mgtestclirunner. I run test cases and have not had the problem you are discussing.
I do like the idea of passing keywords to tests. I have some test currently that can produce graphical output or not and having a keyword to use instead of changing the source code of the test would be nice.
March 21st, 2007 at 3:15 pm
I’d like to include persistent data that I can use in different methods of a test class. Is there a way to do this with MGunit?
March 21st, 2007 at 3:28 pm
You can have persistent data used in different methods of a test (at least a simple version of it). Setup the data in the init method, store it as instance variables, and clean it up in the cleanup method.
I have thought about adding “fixtures” which would just mean that a particular method (like “setup”) would be called before each test and another (like “teardown”) would be called after each test. Data would still be stored as instance variables of the test case. Then each test would be sure to have its pristine data not messed up by some other test.
April 27th, 2007 at 2:59 pm
[…] See “Unit testing framework” for an overview of the unit test framework. […]
July 3rd, 2007 at 5:21 pm
[…] Fixtures: update to unit testing framework and Unit testing framework for more information about the testing framework. […]
September 27th, 2008 at 12:22 pm
Hello,
Is there a way to achieve automatic compilation of tests? That is to say, if I issue the command mgunit, ‘myprojecttest’, can it be made automatically to find the file myprojecttest.pro and compile its contents?
Thanks!
Cheers,
David
September 27th, 2008 at 3:56 pm
Compilation should be automatic if the files are named according to the normal class definition scheme in IDL i.e. the “myprojecttest” would be in the file “myprojecttest__define.pro”.
Also, using test suites, it is possible for mgunit to automatically find/compile/run all the tests in a given directory hierarchy and run them.
November 2nd, 2010 at 11:34 am
Michael,
I discovered this unit test framework this week and used it with success. Excellent program. Thanks a lot
Jean