Introduction

MGunit is a unit testing framework modeled on other unit testing frameworks such as JUnit. The goal is to allow easy creation and reporting of results of tests, while still allowing for many different testing situations. Simple naming conventions replace formal creation of hierarchies and specification of tests. This allows test suites to be created with a minimum of code beyond the actual code of the tests themselves.

Basic use

Individual tests are methods of a class that subclasses MGutTestCase. Each method returns 1 for success or 0 (or throws an error) for failure. Each test method's name must start with "test".

For example, let's create some tests for the FINDGEN function. First, subclass MGutTestCase like below:

  pro findgen_ut__define
    compile_opt strictarr
    
    define = { findgen_ut, inherits MGutTestCase }
  end

A test is just a method of this class whose name starts with "test". The mgunit framework will find the tests automatically. For example, a simple test:

  function findgen_ut::test_basic
    compile_opt strictarr
  
    a = findgen(5)
    assert, array_equal(a, [0.0, 1.0, 2.0, 3.0, 4.0]), 'Correct elements'

    return, 1
  end

Return 1 for success. For failure, either return 0 or throw an error. Here the helper routine ASSERT will throw an error using the given message if its condition is not met. This will be reported as a failure along with the message. To run this test, use the following:

  IDL> mgunit, 'findgen_ut'
  "All tests" test suite starting (1 test suite/case, 1 tests)
     "findgen_ut" test case starting (4 tests)
        test_basic: passed
     Results: 1 / 1 tests passed
  Results: 1 / 1 tests passed

A test case may have as many individual tests (methods with names starting with "test") as necessary.

One tricky situation is that sometimes invalid input must be tested to make sure the routine fails. In these situations, throwing an error should indicate the success of the test, not a failure. In this case use the provided batch file error_is_pass at the beginning of the routine, like:

  function findgen_ut::test_error
    compile_opt strictarr
    @error_is_pass

    a = findgen('string')

    return, 0
  end

As an example of showing a failing test, the example test case includes a test_fail_example method with an invalid assertion. Also provided is an example of using the error_is_fail batch file in the test_baderror method. Runtime errors will cause a test to fail, but IO errors normally will not. The test_baderror test uses @error_is_fail to make an IO error cause the test to fail:

  function findgen_ut::test_baderror
    compile_opt strictarr  
    @error_is_fail

    a = findgen('another_string')

    return, 1
  end

Running all the test case now results in the following:

  IDL> mgunit, 'findgen_ut'
  "All tests" test suite starting (1 test suite/case, 4 tests)
     "findgen_ut" test case starting (4 tests)
        test_basic: passed
        test_error: passed
        test_fail_example: failed "Wrong number of elements"
        test_baderror: failed "Type conversion error: Unable to convert given STRING to Long64."
     Results: 2 / 4 tests passed
  Results: 2 / 4 tests passed

Both test failures above are expected and present only to demonstrate features of mgunit.

A single test method of a test case can be run using a . to separate the test class name from the method name::

  IDL> mgunit, 'findgen_ut.test_basic'
  "All tests" test suite starting (1 test suite/case, 1 test)
     "findgen_ut" test case starting (1 test)
        test_basic: passed (0.000078 seconds)
     Results: 1 / 1 tests passed, 0 skipped
  Results: 1 / 1 tests passed, 0 skipped

Running multiple test cases

Another test case, indgen_ut, is provided as an example. It is analogous to findgen_ut for the INDGEN routine.

Multiple test cases can be executed by specifying them as an array:

  IDL> mgunit, ['findgen_ut', 'indgen_ut']
  "All tests" test suite starting (2 test suites/cases, 10 tests)
     "findgen_ut" test case starting (5 tests)
        test_baderror: failed "Type conversion error: Unable to convert given STRING to Long64."
        test_basic: passed
        test_error: passed
        test_fail_example: failed "Wrong number of elements"
        test_incorrecterror: failed "Type conversion error: Unable to convert given STRING to Long64."
     Results: 2 / 5 tests passed
     "indgen_ut" test case starting (5 tests)
        test4: failed "Type conversion error: Unable to convert given STRING to Long64."
        test_baderror: failed "Type conversion error: Unable to convert given STRING to Long64."
        test_basic: passed
        test_error: passed
        test_fail_example: failed "Wrong number of elements"
     Results: 2 / 5 tests passed
  Results: 4 / 10 tests passed

Alternatively, test cases may be grouped into test suites. Test suites are just collections of test cases. To make a suite, subclass MGutTestSuite and use the add method in the the subclass' init method to add test classes. For example, to make a suite containing the indgen_ut and findgen_ut test cases:

  function indgen_uts::init, _extra=e
    compile_opt strictarr

    if (~self->mguttestsuite::init(_extra=e)) then return, 0

    ;self->add, ['indgen_ut', 'findgen_ut']
    self->add, /all
  
    return, 1
  end

  pro indgen_uts__define
    compile_opt strictarr
  
    define = { indgen_uts, inherits MGutTestSuite }
  end

The commented out line will specifically add indgen_ut and findgen_ut, wherever their source code files may be located. Instead, the ALL keyword is used to add all test cases in the same directory as the test suite source code file. Test cases to be found in this manner must use the convention to name the class with an appended "_ut", as in "findgen_ut".

mgunit will also accept a mixed array of test suites and test cases, as in:

  IDL> mgunit, ['findgen_ut', 'indgen_ut', 'indgen_uts']

In our case, this does not make sense because this will execute the same tests twice.

Fixtures

The setup and teardown methods of a test case class are executed before and after each individual test. By default, they are empty, but subclasses of MGutTestCase can override them to do any common setup/teardown tasks. Any data to be stored from the setup is normally saved as an instance variable of the test case class so that it can be accessed by the test and the teardown method.

Pointer and object memory leaks can be tested for using fixtures by comparing the number of current pointers and objects during setup and teardown.

Other output

Results can be sent to a log file with the FILENAME keyword:

  IDL> mgunit, 'indgen_uts', filename='test-results.log'

This will send the normal output to the results.log file.

HTML output can also be created with the boolean HTML keyword to the MGUNIT routine. Generally, the FILENAME keyword is used in conjunction with this option:

  IDL> mgunit, 'indgen_uts', filename='test-results.html', /html

Miscellaneous

Templates for the IDL Workbench are provided to make test/suite creation even faster. To use them, first navigate to the Workbench preferences. There should be a Templates section under the IDL heading. Click the "Import" button on the right and navigate to the "test-templates.xml" file in the mgunit source. Two new templates, "Test case" and "Test suite", should now be available. Typing "testcase" into a new file and then selecting *Edit > Content Assist* from the menus will create a test case which can be filled out like a form. Suites can be created the same way by typing "testsuite".

Tips

It can be useful to create a subclass of MGutTestCase for a project so that each test case in the project inherits from that class instead of directly from MGutTestCase. This case can do work common to all the tests i.e. find the location of test data, have common setup/teardown methods, etc.

The NTESTS, NPASS, and NFAIL keywords to the MGUNIT routine output the appropriate values. These can be handy for automated scripts i.e. sending email if any test fails, etc.