Changeset 1640 for trunk/test/README


Ignore:
Timestamp:
Dec 12, 2008, 9:52:16 PM (13 years ago)
Author:
Peter
Message:

adding some text on how to use the test suite - and also some rules I think we have decided on. refs #458

File:
1 edited

Legend:

Unmodified
Added
Removed
  • trunk/test/README

    r1517 r1640  
    11$Id$
     2
     3    The yat test suite
     4
     5
     6Usage:
     7========
     8
     9Running all tests
     10-----------------
     11
     12  make check
     13
     14
     15Interpretation
     16--------------
     17
     18  Successes:
     19    PASS  - success
     20    XFAIL - expected failure
     21
     22  Failures:
     23    FAIL  - failure
     24    XPASS - unexpected success
     25
     26  Other:
     27    SKIP  - skipped test
     28
     29
     30Getting details from failures
     31-----------------------------
     32
     33  You can run a specific test directly. Add a '-v' flag to trigger verbosity
     34
     35    ./program_test -v
     36
     37  If you want to run the test suite in verbose mode, you can invoke
     38  make as follows:
     39
     40    make YAT_TEST_VERBOSE=1 check
     41
     42  If you want to run only a subset of tests, you can invoke make as follows
     43
     44    make TESTS="first_test second_test" check
     45
     46
     47Reporting failures
     48------------------
     49
     50  Open a 'New Ticket' in the issue tracking system
     51  (http://dev.thep.lu.se/yat) with verbose output of failing tests
     52  along with which version of yat and which operating system
     53  etc. Alternatively, you can send a bug report to the address given in
     54  the failed `make check' or in `configure --help'.
     55
     56
     57
     58Writing tests
     59=============
     60
     61  If you plan to fix a bug, write the test first. Thsi way you will
     62  make sure the test catches the bug, and that it succeeds once you
     63  have fixed the bug.
     64
     65  Use the test::Suite class liberally. See other tests for examples of
     66  typical usage.
     67
     68  Tests shall return 0 at success and -1 otherwise (see
     69  test::Suite::return_value()). If the test should be skipped for some
     70  reason it shall return 77.
     71
     72  When a test is run without verbose flag, it should be silent also at
     73  failure. Achieve this behaviour by sending output through
     74  test::Suite class (see functions err() and out()).
     75
     76  If a test grows large, avoid having several tests in one long
     77  main. It creates long correlations and complicates debugging and
     78  extending the test. If you don't want to split the test into several
     79  files, split the test into several independent sub-functions.
     80
    281
    382======================================================================
     
    24103
    25104
    26 usage:
    27        program_test
    28 
    29 no print out, returns 0 if test is succesful.
    30 
    31        program_test -v
    32 
    33 verbose mode for printing extra information.
    34  
    35 Verbose mode can also be achieved by setting environment variable
    36 YAT_TEST_VERBOSE=1, which also works when running make:
    37 
    38        YAT_TEST_VERBOSE=1 make check
    39 
Note: See TracChangeset for help on using the changeset viewer.