Changeset 132

Show
Ignore:
Timestamp:
02/13/12 15:29:40 (15 months ago)
Author:
mgalloy
Message:

Major update to docs.

Location:
trunk
Files:
6 added
3 removed
6 modified

Legend:

Unmodified
Added
Removed

  • trunk/docs/Makefile

    r126 r132  
    363363# the "namespace". 
    364364 
    365 $(docutils.)LATEX ?= latex 
    366 $(docutils.)PDFLATEX ?= pdflatex 
     365$(docutils.)LATEX ?= xelatex 
     366$(docutils.)PDFLATEX ?= xelatex 
    367367$(docutils.)DVIPS ?= dvips 
    368368$(docutils.)DVIPDF ?= dvipdf 
    369369 
    370 %.dvi: %.tex 
     370%.pdf: %.tex 
    371371        cd $(*D); $(LATEX) $(*F) 
    372  
    373 %.ps: %.dvi 
    374         $(DVIPS) $< -o $@ 
    375  
    376 ifeq "$($(docutils.)ENABLE_PDFLATEX)" "1" 
    377 %.pdf: %.pdftex 
    378         cd $(*D); $(PDFLATEX) $(*F).pdftex 
    379 else 
    380 %.pdf: %.dvi 
    381         $(DVIPDF) $< $@ 
    382 endif 
     372        cd $(*D); $(LATEX) $(*F) 
    383373 
    384374endif # ENABLE_LATEX_RULES 
    … …  
    405395 
    406396# Set `ALL` to change the things to build by default. 
    407 $(docutils.)ALL ?= html ps pdf 
     397$(docutils.)ALL ?= html pdf 
    408398 
    409399ifeq "$($(docutils.)ENABLE_COMMON_TARGETS)" "1" 

  • trunk/docs/mgunit-docs.sty

    r126 r132  
    1616\urlstyle{same} 
    1717 
    18 % use Courier for tt font, scaled a bit to match the body font 
    19 \usepackage[scaled=0.88]{couriers} 
     18% get regular fonts 
     19\usepackage{fontspec} 
     20\usepackage{xunicode} 
     21\usepackage{xltxtra} 
     22\defaultfontfeatures{Scale=MatchLowercase} 
     23\setmonofont{Monaco} 
    2024 
    2125\usepackage[runin]{abstract} 

  • trunk/docs/test-results.html

    r51 r132  
    99.passed { color: #060; } 
    1010.failed { color: #C00; } 
     11.skipped { color: #CC0; } 
     12.time { color: #888; font-size: 9pt; margin-left: 1em; } 
    1113.results { } 
    1214 
    … …  
    2426.casename { color: blue; } 
    2527</style></head><body> 
    26 <ul class="testsuite"><li><span class="suitename">All tests</span> test suite starting (1 test suite/case, 8 tests)</li> 
    27 <ul class="testsuite"><li><span class="suitename">indgen_uts</span> test suite starting (2 test suites/cases, 8 tests)</li> 
    28 <ul class="testcase"><li><span class="casename">findgen_ut</span> test case starting (4 tests)</li> 
     28<ul class="testsuite"><li><span class="suitename">All tests</span> test suite starting (1 test suite/case, 9 tests)</li> 
     29<ul class="testsuite"><li><span class="suitename">mglib_uts</span> test suite starting (2 test suites/cases, 9 tests)</li> 
     30<ul class="testcase"><li><span class="casename">mg_evalexpr_ut</span> test case starting (8 tests)</li> 
    2931<ol> 
    30 <li>test_baderror: <span class="failed">failed "Type conversion error: Unable to convert given STRING to Long64."</span></li> 
    31 <li>test_basic: <span class="passed">passed</span></li> 
    32 <li>test_error: <span class="passed">passed</span></li> 
    33 <li>test_fail_example: <span class="failed">failed "Wrong number of elements"</span></li> 
     32<li>test_basic: <span class="passed">passed</span> <span class="time">0.000316 seconds</span></li> 
     33<li>test_error1: <span class="passed">passed</span> <span class="time">0.000375 seconds</span></li> 
     34<li>test_function1: <span class="passed">passed</span> <span class="time">0.001106 seconds</span></li> 
     35<li>test_order1: <span class="passed">passed</span> <span class="time">0.000439 seconds</span></li> 
     36<li>test_order2: <span class="passed">passed</span> <span class="time">0.000818 seconds</span></li> 
     37<li>test_order3: <span class="passed">passed</span> <span class="time">0.001462 seconds</span></li> 
     38<li>test_subshash: <span class="passed">passed</span> <span class="time">0.000734 seconds</span></li> 
     39<li>test_subsstruct: <span class="passed">passed</span> <span class="time">0.000539 seconds</span></li> 
    3440</ol> 
    35 <span class="results">Results: 2 / 4 tests passed</span></ul> 
    36 <ul class="testcase"><li><span class="casename">indgen_ut</span> test case starting (4 tests)</li> 
     41<span class="results">Results: 8 / 8 tests passed, 0 skipped</span></ul> 
     42<ul class="testcase"><li><span class="casename">mg_linear_function_ut</span> test case starting (1 test)</li> 
    3743<ol> 
    38 <li>test_baderror: <span class="failed">failed "Type conversion error: Unable to convert given STRING to Long64."</span></li> 
    39 <li>test_basic: <span class="passed">passed</span></li> 
    40 <li>test_error: <span class="passed">passed</span></li> 
    41 <li>test_fail_example: <span class="failed">failed "Wrong number of elements"</span></li> 
     44<li>test_basic: <span class="passed">passed</span> <span class="time">0.000016 seconds</span></li> 
    4245</ol> 
    43 <span class="results">Results: 2 / 4 tests passed</span></ul> 
    44 <span class="results">Results: 4 / 8 tests passed</span></ul> 
    45 <span class="results">Results: 4 / 8 tests passed</span></ul> 
    46 <span id="dateline">Test results from Thu Apr  9 14:24:43 2009</span> 
     46<span class="results">Results: 1 / 1 tests passed, 0 skipped</span></ul> 
     47<span class="results">Results: 9 / 9 tests passed, 0 skipped</span></ul> 
     48<span class="results">Results: 9 / 9 tests passed, 0 skipped</span></ul> 
     49<span id="dateline">Test results from Mon Feb 13 15:49:18 2012</span> 
    4750</body></html> 

  • trunk/docs/test-results.log

    r51 r132  
    1 "All tests" test suite starting (1 test suite/case, 8 tests) 
    2    "indgen_uts" test suite starting (2 test suites/cases, 8 tests) 
    3       "findgen_ut" test case starting (4 tests) 
    4          test_baderror: failed "Type conversion error: Unable to convert given STRING to Long64." 
    5          test_basic: passed 
    6          test_error: passed 
    7          test_fail_example: failed "Wrong number of elements" 
    8       Results: 2 / 4 tests passed 
    9       "indgen_ut" test case starting (4 tests) 
    10          test_baderror: failed "Type conversion error: Unable to convert given STRING to Long64." 
    11          test_basic: passed 
    12          test_error: passed 
    13          test_fail_example: failed "Wrong number of elements" 
    14       Results: 2 / 4 tests passed 
    15    Results: 4 / 8 tests passed 
    16 Results: 4 / 8 tests passed 
     1[35m"All tests" test suite starting (1 test suite/case, 9 tests)[0m 
     2[35m   "mglib_uts" test suite starting (2 test suites/cases, 9 tests)[0m 
     3[34m      "mg_evalexpr_ut" test case starting (8 tests)[0m 
     4         test_basic: [32mpassed[0m (0.006570 seconds) 
     5         test_error1: [32mpassed[0m (0.000396 seconds) 
     6         test_function1: [32mpassed[0m (0.001040 seconds) 
     7         test_order1: [32mpassed[0m (0.000442 seconds) 
     8         test_order2: [32mpassed[0m (0.000826 seconds) 
     9         test_order3: [32mpassed[0m (0.001457 seconds) 
     10         test_subshash: [32mpassed[0m (0.004958 seconds) 
     11         test_subsstruct: [32mpassed[0m (0.000517 seconds) 
     12[34m      Results: 8 / 8 tests passed, 0 skipped[0m 
     13[34m      "mg_linear_function_ut" test case starting (1 test)[0m 
     14         test_basic: [32mpassed[0m (0.000163 seconds) 
     15[34m      Results: 1 / 1 tests passed, 0 skipped[0m 
     16[35m   Results: 9 / 9 tests passed, 0 skipped[0m 
     17[35mResults: 9 / 9 tests passed, 0 skipped[0m 

  • trunk/docs/using-mgunit.rst

    r125 r132  
    88------------ 
    99 
    10 MGunit is a unit testing framework modeled on other unit testing frameworks such as JUnit. The goal is to allow easy creation and reporting of results of tests, while still allowing for many different testing situations. Simple naming conventions replace formal creation of hierarchies and specification of tests. This allows test suites to be created with a minimum of code beyond the actual code of the tests themselves. 
     10MGunit is a unit testing framework modeled on other unit testing frameworks such as JUnit. The goal is to allow easy creation of tests and simple, clear reporting of results of the tests, while still allowing for many different testing situations. Simple naming conventions replace formal creation of hierarchies and specification of tests. This allows test suites to be created with a minimum of overhead so that the developer can focus on the actual code of the tests themselves. 
     11 
     12Developers using mgunit need to be familiar with the basic syntax for object-oriented programming in IDL. 
    1113 
    1214 
    … …  
    1618Individual tests are methods of a class that subclasses `MGutTestCase`. Each method returns `1` for success or `0` (or throws an error) for failure. Each test method's name must start with "test".  
    1719 
    18 For example, let's create some tests for the `FINDGEN` function. First, subclass `MGutTestCase` like below:: 
     20Let's look at some of the tests for routines in my library. For example, the `MG_EVALEXPR` routine evaluates a mathematical expression specified as a string, reporting the error status of parsing the expression. Let's write some tests to verify it's functionality. First, subclass `MGutTestCase` like below:: 
    1921 
    20   pro findgen_ut__define 
     22  pro mg_evalexpr_ut__define 
    2123    compile_opt strictarr 
    22      
    23     define = { findgen_ut, inherits MGutTestCase } 
     24   
     25    define = { mg_evalexpr_ut, inherits MGutTestCase } 
    2426  end 
    2527 
    2628A test is just a method of this class whose name starts with "test". The mgunit framework will find the tests automatically. For example, a simple test:: 
    2729 
    28   function findgen_ut::test_basic 
     30  function mg_evalexpr_ut::test_basic 
    2931    compile_opt strictarr 
    3032   
    31     a = findgen(5) 
    32     assert, array_equal(a, [0.0, 1.0, 2.0, 3.0, 4.0]), 'Correct elements' 
    33  
     33    result = mg_evalexpr('1 + 2', error=error) 
     34    assert, error eq 0, 'incorrect error status: %d', error 
     35    assert, result eq 3, 'incorrect result: %d', result 
     36   
    3437    return, 1 
    3538  end 
    3639 
    37 Return `1` for success. For failure, either return `0` or throw an error. Here the helper routine `ASSERT` will throw an error using the given message if its condition is not met. This will be reported as a failure along with the message. To run this test, do the following:: 
     40Tests return `1` for success. For failure, they either return `0` or throw an error. Here the helper routine `ASSERT` will throw an error using the given message if its condition is not met. This will be reported as a failure along with the message. To run this test, do the following:: 
    3841 
    39   IDL> mgunit, 'findgen_ut' 
    40   "All tests" test suite starting (1 test suite/case, 1 tests) 
    41      "findgen_ut" test case starting (4 tests) 
    42         test_basic: passed 
    43      Results: 1 / 1 tests passed 
    44   Results: 1 / 1 tests passed 
     42  IDL> mgunit, 'mg_evalexpr_ut.test_basic' 
     43  "All tests" test suite starting (1 test suite/case, 1 test) 
     44     "mg_evalexpr_ut" test case starting (1 test) 
     45        test_basic: passed (0.000314 seconds) 
     46     Results: 1 / 1 tests passed, 0 skipped 
     47  Results: 1 / 1 tests passed, 0 skipped 
     48 
     49There are actually several tests in `mg_evalexpr_ut__define.pro`; to run them all do:: 
     50 
     51  IDL> mgunit, 'mg_evalexpr_ut' 
     52  "All tests" test suite starting (1 test suite/case, 8 tests) 
     53     "mg_evalexpr_ut" test case starting (8 tests) 
     54        test_basic: passed (0.057548 seconds) 
     55        test_error1: passed (0.000403 seconds) 
     56        test_function1: passed (0.001067 seconds) 
     57        test_order1: passed (0.000455 seconds) 
     58        test_order2: passed (0.000817 seconds) 
     59        test_order3: passed (0.001468 seconds) 
     60        test_subshash: passed (0.046330 seconds) 
     61        test_subsstruct: passed (0.000618 seconds) 
     62     Results: 8 / 8 tests passed, 0 skipped 
     63  Results: 8 / 8 tests passed, 0 skipped 
    4564 
    4665A test case may have as many individual tests (methods with names starting with "test") as necessary.  
    47  
    48 One tricky situation is that sometimes invalid input must be tested to make sure the routine fails. In these situations, throwing an error should indicate the success of the test, not a failure. In this case use the provided batch file `error_is_pass` at the beginning of the routine, like:: 
    49  
    50   function findgen_ut::test_error 
    51     compile_opt strictarr 
    52     @error_is_pass 
    53  
    54     a = findgen('string') 
    55  
    56     return, 0 
    57   end 
    58  
    59 As an example of showing a failing test, the example test case includes a `test_fail_example` method with an invalid assertion. Also provided is an example of using the `error_is_fail` batch file in the `test_baderror` method. Runtime errors will cause a test to fail, but IO errors normally will not. The `test_baderror` test uses `@error_is_fail` to make an IO error cause the test to fail:: 
    60  
    61   function findgen_ut::test_baderror 
    62     compile_opt strictarr   
    63     @error_is_fail 
    64  
    65     a = findgen('another_string') 
    66  
    67     return, 1 
    68   end 
    69  
    70 Running the test case now results in the following:: 
    71  
    72   IDL> mgunit, 'findgen_ut' 
    73   "All tests" test suite starting (1 test suite/case, 4 tests) 
    74      "findgen_ut" test case starting (4 tests) 
    75         test_basic: passed 
    76         test_error: passed 
    77         test_fail_example: failed "Wrong number of elements" 
    78         test_baderror: failed "Type conversion error: Unable to convert given STRING to Long64." 
    79      Results: 2 / 4 tests passed 
    80   Results: 2 / 4 tests passed 
    81  
    82 Both test failures above are expected and present only to demonstrate features of mgunit. 
    83  
    84 A single test method of a test case can be run using a `.` to separate the test class name from the method name:: 
    85  
    86   IDL> mgunit, 'findgen_ut.test_basic' 
    87   "All tests" test suite starting (1 test suite/case, 1 test) 
    88      "findgen_ut" test case starting (1 test) 
    89         test_basic: passed (0.000078 seconds) 
    90      Results: 1 / 1 tests passed, 0 skipped 
    91   Results: 1 / 1 tests passed, 0 skipped 
    9266 
    9367 
    … …  
    9569--------------------------- 
    9670 
    97 Another test case, `indgen_ut`, is provided as an example. It is analogous to `findgen_ut` for the `INDGEN` routine. 
     71To run multiple test cases, you can pass an array of test case names to `MGUNIT`:: 
    9872 
    99 Multiple test cases can be executed by specifying them as an array:: 
     73  IDL> mgunit, ['mg_evalexpr_ut', 'mg_linear_function_ut'] 
     74  "All tests" test suite starting (2 test suites/cases, 9 tests) 
     75     "mg_evalexpr_ut" test case starting (8 tests) 
     76        test_basic: passed (0.006440 seconds) 
     77        test_error1: passed (0.000407 seconds) 
     78        test_function1: passed (0.001036 seconds) 
     79        test_order1: passed (0.000439 seconds) 
     80        test_order2: passed (0.000817 seconds) 
     81        test_order3: passed (0.001452 seconds) 
     82        test_subshash: passed (0.004932 seconds) 
     83        test_subsstruct: passed (0.000507 seconds) 
     84     Results: 8 / 8 tests passed, 0 skipped 
     85     "mg_linear_function_ut" test case starting (1 test) 
     86        test_basic: passed (0.017841 seconds) 
     87     Results: 1 / 1 tests passed, 0 skipped 
     88  Results: 9 / 9 tests passed, 0 skipped 
    10089 
    101   IDL> mgunit, ['findgen_ut', 'indgen_ut'] 
    102   "All tests" test suite starting (2 test suites/cases, 10 tests) 
    103      "findgen_ut" test case starting (5 tests) 
    104         test_baderror: failed "Type conversion error: Unable to convert given STRING to Long64." 
    105         test_basic: passed 
    106         test_error: passed 
    107         test_fail_example: failed "Wrong number of elements" 
    108         test_incorrecterror: failed "Type conversion error: Unable to convert given STRING to Long64." 
    109      Results: 2 / 5 tests passed 
    110      "indgen_ut" test case starting (5 tests) 
    111         test4: failed "Type conversion error: Unable to convert given STRING to Long64." 
    112         test_baderror: failed "Type conversion error: Unable to convert given STRING to Long64." 
    113         test_basic: passed 
    114         test_error: passed 
    115         test_fail_example: failed "Wrong number of elements" 
    116      Results: 2 / 5 tests passed 
    117   Results: 4 / 10 tests passed 
     90Typically, you will have a directory (or directory hierarchy) of test cases to run. To automatically group all the tests in a directory hierarchy in one "test suite", create a subclass of `MGutTestSuite`. For example, my library tests are grouped into the a test suite by `mglib_uts__define.pro`:: 
    11891 
    119 Alternatively, test cases may be grouped into test suites. Test suites are  just collections of test cases. To make a suite, subclass `MGutTestSuite` and use the `add` method in the the subclass' `init` method to add test classes. For example, to make a suite containing the `indgen_ut` and `findgen_ut` test cases:: 
    120  
    121   function indgen_uts::init, _extra=e 
     92  function mglib_uts::init, _extra=e 
    12293    compile_opt strictarr 
    12394 
    124     if (~self->mguttestsuite::init(_extra=e)) then return, 0 
     95    if (~self->mguttestsuite::init(_strict_extra=e)) then return, 0 
    12596 
    126     ;self->add, ['indgen_ut', 'findgen_ut'] 
    12797    self->add, /all 
    128    
     98 
    12999    return, 1 
    130100  end 
    131101 
    132   pro indgen_uts__define 
     102  pro mglib_uts__define 
    133103    compile_opt strictarr 
    134    
    135     define = { indgen_uts, inherits MGutTestSuite } 
     104 
     105    define = { mglib_uts, inherits MGutTestSuite } 
    136106  end 
    137107 
    138 The commented out line would specifically add `indgen_ut` and `findgen_ut`, wherever their source code files may be located. Instead, the `ALL` keyword is used to add all test cases in the same directory as the test suite source code file. Test cases to be found in this manner must use the convention to name the class with an appended "_ut", as in "findgen_ut". 
     108For the automatic collection of tests to work, the classnames for all the test cases must end in "ut", e.g., `mg_evalexpr_ut`. The test suite definition file is placed in the root directory for all the files for the test cases. To run all the test cases, just do:: 
    139109 
    140 mgunit will also accept a mixed array of test suites and test cases, as in:: 
    141  
    142   IDL> mgunit, ['findgen_ut', 'indgen_ut', 'indgen_uts'] 
    143  
    144 In our case, this does not make sense because this will execute the same tests twice. 
     110  IDL> mgunit, `mglib_uts` 
    145111 
    146112 
    … …  
    150116The `setup` and `teardown` methods of a test case class are executed before and after each individual test. By default, they are empty, but subclasses of `MGutTestCase` can override them to do any common setup/teardown tasks. Any data to be stored from the setup is normally saved as an instance variable of the test case class so that it can be accessed by the test and the `teardown` method. 
    151117 
    152 Pointer and object memory leaks can be tested for using fixtures by comparing the number of current pointers and objects during setup and teardown. 
     118Pointer and object memory leaks can be tested for using fixtures by comparing the number of current pointers and objects during setup and teardown. For an example of doing this, see the code for the `MGutLibTestCase` class in `mgutlibtestcase__define.pro`. Note that the example tests provided actually subclass from `MGutLibTestCase` to provide memory leak checking. 
    153119 
    154120 
    … …  
    156122------------- 
    157123 
    158 Sometimes there are tests that are only valid to run in certain circumstances. The `ASSERT` routine can be used to stop a test from counting in the final tally of passes and failures, but using the `SKIP` keyword. For example, if you only want to run a test if running IDL 8.0 or later, put the following at the beginning of the test:: 
     124Sometimes there are tests that are only valid to run in certain circumstances. The `ASSERT` routine can be used to stop a test from counting in the final tally of passes and failures by using the `SKIP` keyword. For example, if you only want a test to run if running under IDL 8.0 or later, put the following at the beginning of the test:: 
    159125 
    160   assert, long((strsplit(!version.release, ',', /extract))[0]) ge 8, $ 
     126  assert, long((strsplit(!version.release, '.', /extract))[0]) ge 8, $ 
    161127          'IDL version too old: %s', !version.release, $ 
    162128          /skip 
    163129 
     130For IDL versions before 8.0, tests including the above will not immediately stop and not be counted in the final count of passed/failed tests. 
    164131 
    165 Other output 
    166 ------------ 
     132 
     133Crashes in a test 
     134----------------- 
     135 
     136Normally, a runtime error in a test will cause the test to fail. But sometimes routines are supposed to crash on invalid input. To test those cases, i.e., where invalid inputs are purposefully fed into a routine to cause a failure, use the `CATCH` block defined in the batch file `error_is_pass.pro`. For example, the following makes sure that the `FINDGEN` routine fails when given a string argument:: 
     137 
     138  function my_routine_ut::test_basic 
     139    compile_opt strictarr 
     140    @error_is_pass 
     141     
     142    a = findgen('a string') 
     143     
     144    return, 1  
     145  end 
     146 
     147By default, runtime errors will cause a test to fail, but IO errors will not. Use the `CATCH` block present in `error_is_fail.pro` to make an IO error cause the test to fail also. 
     148 
     149 
     150Alternative output 
     151------------------ 
    167152 
    168153Results can be sent to a log file with the `FILENAME` keyword:: 
    169154 
    170   IDL> mgunit, 'indgen_uts', filename='test-results.log' 
     155  IDL> mgunit, 'mglib_uts', filename='test-results.log' 
    171156 
    172 This will send the normal output to the results.log file. 
     157This will send the normal output to the `results.log` file. 
    173158 
    174159HTML output can also be created with the boolean `HTML` keyword to the `MGUNIT` routine. Generally, the `FILENAME` keyword is used in conjunction with this option:: 
    175160 
    176   IDL> mgunit, 'indgen_uts', filename='test-results.html', /html 
     161  IDL> mgunit, 'mglib_uts', filename='test-results.html', /html 
    177162 
    178163 
    179 Miscellaneous 
    180 ------------- 
     164Test templates 
     165-------------- 
    181166 
    182167Templates for the IDL Workbench are provided to make test/suite creation even faster. To use them, first navigate to the Workbench preferences. There should be a Templates section under the IDL heading. Click the "Import" button on the right and navigate to the `test-templates.xml` file in the mgunit source. Two new templates, "Test case" and "Test suite", should now be available. Typing "testcase" into a new file and then selecting *Edit > Content Assist* from the menus will create a test case which can be filled out like a form. Suites can be created the same way by typing "testsuite". 
    … …  
    186171---- 
    187172 
    188 It can be useful to create a subclass of `MGutTestCase` for a project so that each test case in the project inherits from that class instead of directly from `MGutTestCase`. This case can do work common to all the tests i.e. find the location of test data, have common setup/teardown methods, etc. 
     173It can be useful to create a subclass of `MGutTestCase` for a project so that each test case in the project inherits from that class instead of directly from `MGutTestCase`. This case can do work common to all the tests, i.e., find the location of test data, have common setup/teardown methods, etc. The `MGutLibTestCase` class discussed in the Fixtures section is used in this manner. 
    189174 
    190 The `NTESTS`, `NPASS`, and `NFAIL` keywords to the `MGUNIT` routine output the appropriate values. These can be handy for automated scripts i.e. sending email if any test fails, etc. 
     175The `NTESTS`, `NPASS`, and `NFAIL` keywords to the `MGUNIT` routine output the appropriate values. These can be handy for automated scripts, i.e., sending email if any test fails, etc. 
    191176 

  • trunk/RELEASE

    r129 r132  
    88 
    99* Added utilities to help test GUI applications. 
     10 
     11* Updated "Using mgunit" documentation. 
    1012 
    1113