1 | $Id: README 1642 2008-12-12 21:09:19Z peter $ |
---|
2 | |
---|
3 | The yat test suite |
---|
4 | |
---|
5 | |
---|
6 | Usage: |
---|
7 | ======== |
---|
8 | |
---|
9 | Running all tests |
---|
10 | ----------------- |
---|
11 | |
---|
12 | make check |
---|
13 | |
---|
14 | |
---|
15 | Interpretation |
---|
16 | -------------- |
---|
17 | |
---|
18 | Successes: |
---|
19 | PASS - success |
---|
20 | XFAIL - expected failure |
---|
21 | |
---|
22 | Failures: |
---|
23 | FAIL - failure |
---|
24 | XPASS - unexpected success |
---|
25 | |
---|
26 | Other: |
---|
27 | SKIP - skipped test |
---|
28 | |
---|
29 | |
---|
30 | Getting details from failures |
---|
31 | ----------------------------- |
---|
32 | |
---|
33 | You can run a specific test directly. Add a '-v' flag to trigger verbosity |
---|
34 | |
---|
35 | ./program_test -v |
---|
36 | |
---|
37 | If you want to run the test suite in verbose mode, you can invoke |
---|
38 | make as follows: |
---|
39 | |
---|
40 | make YAT_TEST_VERBOSE=1 check |
---|
41 | |
---|
42 | If you want to run only a subset of tests, you can invoke make as follows |
---|
43 | |
---|
44 | make TESTS="first_test second_test" check |
---|
45 | |
---|
46 | |
---|
47 | Reporting failures |
---|
48 | ------------------ |
---|
49 | |
---|
50 | Open a 'New Ticket' in the issue tracking system |
---|
51 | (http://dev.thep.lu.se/yat) with verbose output of failing tests |
---|
52 | along with which version of yat and which operating system |
---|
53 | etc. Alternatively, you can send a bug report to the address given in |
---|
54 | the failed `make check' or in `configure --help'. |
---|
55 | |
---|
56 | |
---|
57 | |
---|
58 | Writing tests |
---|
59 | ============= |
---|
60 | |
---|
61 | If you plan to fix a bug, write the test first. This way you will |
---|
62 | make sure the test catches the bug, and that it succeeds once you |
---|
63 | have fixed the bug. |
---|
64 | |
---|
65 | Use the test::Suite class liberally. See other tests for examples of |
---|
66 | typical usage. |
---|
67 | |
---|
68 | Tests shall return 0 at success and -1 otherwise (see |
---|
69 | test::Suite::return_value()). If the test should be skipped for some |
---|
70 | reason it shall return 77. |
---|
71 | |
---|
72 | When a test is run without verbose flag, it should be silent also at |
---|
73 | failure. Achieve this behaviour by sending output through |
---|
74 | test::Suite class (see functions err() and out()). |
---|
75 | |
---|
76 | If a test needs a data file, place the data in directory |
---|
77 | 'test/data'. However, you should feel free to re-use the data files |
---|
78 | already there. It should be possible to run test from any directory, |
---|
79 | in particular from builddir in a VPATH build. To achieve this, use |
---|
80 | test::Suite::filename() to get the absolute path to the file. |
---|
81 | |
---|
82 | If a test grows large, avoid having several tests in one long |
---|
83 | main. It creates long correlations and complicates debugging and |
---|
84 | extending the test. If you don't want to split the test into several |
---|
85 | files, split the test into several independent sub-functions. |
---|
86 | |
---|
87 | |
---|
88 | ====================================================================== |
---|
89 | Copyright (C) 2005 Peter Johansson |
---|
90 | Copyright (C) 2006 Jari Häkkinen |
---|
91 | Copyright (C) 2007 Jari Häkkinen, Peter Johansson |
---|
92 | Copyright (C) 2008 Peter Johansson |
---|
93 | |
---|
94 | This file is part of yat library, http://dev.thep.lu.se/yat |
---|
95 | |
---|
96 | The yat library is free software; you can redistribute it and/or |
---|
97 | modify it under the terms of the GNU General Public License as |
---|
98 | published by the Free Software Foundation; either version 3 of the |
---|
99 | License, or (at your option) any later version. |
---|
100 | |
---|
101 | The yat library is distributed in the hope that it will be useful, but |
---|
102 | WITHOUT ANY WARRANTY; without even the implied warranty of |
---|
103 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU |
---|
104 | General Public License for more details. |
---|
105 | |
---|
106 | You should have received a copy of the GNU General Public License |
---|
107 | along with yat. If not, see <http://www.gnu.org/licenses/>. |
---|
108 | ====================================================================== |
---|
109 | |
---|
110 | |
---|