source: trunk/test/README @ 2232

Last change on this file since 2232 was 2232, checked in by Peter, 12 years ago

refs #475. added a lazycheck target; will need some docs in test/README.

  • Property svn:eol-style set to native
  • Property svn:keywords set to Author Date Id Revision
File size: 3.7 KB
Line 
1$Id: README 2232 2010-03-26 12:21:58Z peter $
2
3    The yat test suite
4
5
6Usage:
7========
8
9Running the tests
10-----------------
11
12  To run all tests:
13
14    make check
15
16  To rerun only failed tests:
17
18    make recheck
19
20  To rerun only those tests for there are newer sources
21
22    make lazycheck
23
24  To run only a subset of tests, use TESTS variable:
25
26    make check TESTS="first_test second_test"
27
28  To enable test output you can use the VERBOSE variable
29
30    make check VERBOSE=1
31
32Interpretation
33--------------
34
35  Successes:
36    PASS  - success
37    XFAIL - expected failure
38
39  Failures:
40    FAIL  - failure
41    XPASS - unexpected success
42
43  Other:
44    SKIP  - skipped test
45
46
47Getting details from failures
48-----------------------------
49
50  You can run the test directly, `./foo_test'; it will be verbose. The
51  verbose output is also available in `foo_test.log'. A summary log is
52  created in the file `test-suite.log'.
53
54
55Reporting failures
56------------------
57
58  Open a 'New Ticket' in the issue tracking system
59  (http://dev.thep.lu.se/yat) with verbose output of failing tests
60  along with which version of yat and which operating system
61  etc. Alternatively, you can send a bug report to the address given in
62  the failed `make check' or in `configure --help'.
63
64
65
66Writing tests
67=============
68
69  If you plan to fix a bug, write the test first. This way you will
70  make sure the test catches the bug, and that it succeeds once you
71  have fixed the bug.
72
73  Use the test::Suite class liberally. See other tests for examples of
74  typical usage.
75
76  Tests shall return 0 at success and -1 otherwise (see
77  test::Suite::return_value()). If the test should be skipped for some
78  reason it shall return 77.
79
80  When writing numerical tests, it is typically desired to allow for
81  some small errors in the comparisons. When comparing two doubles
82  computed within the same program, the error bound should scale with
83  the machine precision. See test::Suite::equal(3). In other case, for
84  example, when the comparison includes a double distributed in
85  'test/data' the error bound should be fixed and reflect the
86  precision in the distributed file. See test::Suite::equal_fix(3).
87
88  If a test needs a data file, place the data in directory
89  'test/data'. However, you should feel free to re-use the data files
90  already there. It should be possible to run the test suite also in a
91  VPATH build in which builddir and srcdir are not the same. To avoid
92  problems with files not found in VPATH builds, use the function
93  test::filename() which gives you the absolute path to the file.
94
95  If a test grows large, avoid having several tests in one long
96  main. It creates long correlations and complicates debugging and
97  extending the test. If you don't want to split the test into several
98  files, split the test into several independent sub-functions.
99
100
101======================================================================
102Copyright (C) 2005 Peter Johansson
103Copyright (C) 2006 Jari Häkkinen
104Copyright (C) 2007, 2008 Jari Häkkinen, Peter Johansson
105Copyright (C) 2009 Peter Johansson
106
107This file is part of yat library, http://dev.thep.lu.se/yat
108
109The yat library is free software; you can redistribute it and/or
110modify it under the terms of the GNU General Public License as
111published by the Free Software Foundation; either version 3 of the
112License, or (at your option) any later version.
113
114The yat library is distributed in the hope that it will be useful, but
115WITHOUT ANY WARRANTY; without even the implied warranty of
116MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
117General Public License for more details.
118
119You should have received a copy of the GNU General Public License
120along with yat. If not, see <http://www.gnu.org/licenses/>.
121======================================================================
122
123
Note: See TracBrowser for help on using the repository browser.