source: trunk/test/README @ 2475

Last change on this file since 2475 was 2475, checked in by Peter, 10 years ago

typo

  • Property svn:eol-style set to native
  • Property svn:keywords set to Author Date Id Revision
File size: 4.5 KB
Line 
1$Id: README 2475 2011-04-13 14:38:19Z peter $
2
3    The yat test suite
4
5
6Usage:
7========
8
9Running the tests
10-----------------
11
12  To run all tests:
13
14    make check
15
16  You can use `-j N' for faster completion on multiprocessor systems.
17
18  To rerun only failed tests:
19
20    make recheck
21
22  To rerun only those tests for which there are newer sources
23
24    make lazycheck
25
26  To run only a subset of tests, use TESTS variable:
27
28    make check TESTS="first_test second_test"
29
30  To enable test output you can use the VERBOSE variable
31
32    make check VERBOSE=1
33
34Interpretation
35--------------
36
37  Successes:
38    PASS  - success
39    XFAIL - expected failure
40
41  Failures:
42    FAIL  - failure
43    XPASS - unexpected success
44
45  Other:
46    SKIP  - skipped test
47
48
49Getting details from failures
50-----------------------------
51
52  You can run the test directly, `./foo_test'; it will be verbose. The
53  verbose output is also available in `foo_test.log'. A summary log is
54  created in the file `test-suite.log'.
55
56
57Reporting failures
58------------------
59
60  Open a 'New Ticket' in the issue tracking system
61  (http://dev.thep.lu.se/yat) with verbose output of failing tests
62  along with which version of yat and which operating system
63  etc. Alternatively, you can send a bug report to the address given in
64  the failed `make check' or in `configure --help'.
65
66
67
68Writing tests
69=============
70
71  If you plan to fix a bug, write the test first. This way you will
72  make sure the test catches the bug, and that it succeeds once you
73  have fixed the bug.
74
75  Use the test::Suite class liberally. See other tests for examples of
76  typical usage.
77
78  Tests shall return 0 at success and -1 otherwise (see
79  test::Suite::return_value()). If the test should be skipped for some
80  reason it shall return 77.
81
82  When writing numerical tests, it is typically desired to allow for
83  some small errors in the comparisons. When comparing two doubles
84  computed within the same program, the error bound should scale with
85  the machine precision. See test::Suite::equal(3). In other case, for
86  example, when the comparison includes a double distributed in
87  'test/data' the error bound should be fixed and reflect the
88  precision in the distributed file. See test::Suite::equal_fix(3).
89
90  If a test needs a data file, place the data in directory
91  'test/data'. However, you should feel free to re-use the data files
92  already there. It should be possible to run the test suite also in a
93  VPATH build in which builddir and srcdir are not the same. To avoid
94  problems with files not found in VPATH builds, use the function
95  test::filename() which gives you the absolute path to the file.
96
97  If a test grows large, avoid having several tests in one long
98  main. It creates long correlations and complicates debugging and
99  extending the test. If you don't want to split the test into several
100  files, split the test into several independent sub-functions.
101
102C++ tests
103---------
104
105  Use the test::Suite class liberally. See other tests for examples of
106  typical usage. Add the test to EXTRA_PROGRAMS in Makefile.am. Tests
107  should have the extension `.test' as in `foo.test' and corresponding
108  source file should be named `foo.cc'.
109
110Shell tests
111-----------
112
113  Use 'required=...' for setting a space delimited list of required
114  tools (see other tests and common_defs.sh). Include common_defs.sh,
115  with '. ./common_defs.sh'.
116
117  Add test to 'DISTRIBUTED_TESTS' in Makefile.am. In order to get the
118  lazycheck to behave as expected, declare accurate dependencies. If
119  test 'foo.sh', for example, depends on file 'bar.txt', create a make
120  rule: 'foo.log: bar.txt'.
121
122  Make sure to set svn property 'svn:executable' on test.
123
124======================================================================
125Copyright (C) 2005 Peter Johansson
126Copyright (C) 2006 Jari Häkkinen
127Copyright (C) 2007, 2008 Jari Häkkinen, Peter Johansson
128Copyright (C) 2009, 2010 Peter Johansson
129
130This file is part of yat library, http://dev.thep.lu.se/yat
131
132The yat library is free software; you can redistribute it and/or
133modify it under the terms of the GNU General Public License as
134published by the Free Software Foundation; either version 3 of the
135License, or (at your option) any later version.
136
137The yat library is distributed in the hope that it will be useful, but
138WITHOUT ANY WARRANTY; without even the implied warranty of
139MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
140General Public License for more details.
141
142You should have received a copy of the GNU General Public License
143along with yat. If not, see <http://www.gnu.org/licenses/>.
144======================================================================
145
146
Note: See TracBrowser for help on using the repository browser.