source: trunk/test/README @ 1087

Last change on this file since 1087 was 1050, checked in by Peter Johansson, 11 years ago

refs #449. changes reflecting new parallel-tests driver

  • Property svn:eol-style set to native
  • Property svn:keywords set to Id
File size: 4.3 KB
Line 
1$Id: README 1050 2010-04-05 02:01:33Z peter $
2
3    The svndigest test suite
4
5
6Usage:
7========
8
9Running the tests
10-----------------
11
12  To run all tests:
13
14    make check
15
16  To rerun only failed tests:
17
18    make recheck
19
20  To rerun only those tests for there are newer sources
21
22    make lazycheck
23
24  To run only a subset of tests, use TESTS variable:
25
26    make check TESTS="first_test second_test"
27
28  To enable test output you can use the VERBOSE variable
29
30    make check VERBOSE=1
31
32Interpretation
33--------------
34
35  Successes:
36    PASS  - success
37    XFAIL - expected failure
38
39  Failures:
40    FAIL  - failure
41    XPASS - unexpected success
42
43  Other:
44    SKIP  - skipped test
45
46
47Getting details from failures
48-----------------------------
49
50  You can run the test directly, `./foo_test'; it will be verbose. The
51  verbose output is also available in `foo_test.log'. A summary log is
52  created in the file `test-suite.log'.
53
54
55Reporting failures
56------------------
57
58  Open a 'New Ticket' in the issue tracking system
59  (http://dev.thep.lu.se/svndigest) with verbose output of failing
60  tests along with which version of svndigest and which operating
61  system etc. Alternatively, you can send a bug report to the address
62  given in the failed `make check' or in `configure --help'.
63
64
65
66Writing tests
67=============
68
69  If you plan to fix a bug, write the test first. This way you will
70  make sure the test catches the bug, and that it succeeds once you
71  have fixed the bug.
72
73  Add a copyright and license followed by one or two lines describing
74  what the test does.
75
76  Tests shall returns 0 at success, 77 if it should be skipped, and
77  something else at failure. Tests should be silent also at failure
78  unless option `-v` is passed or env variable VERBOSE is set.
79
80  Add the test to XFAIL_TESTS in Makefile.am, if it tests a bug still
81  present, i.e., the test is expected to fail.
82
83C++ tests
84---------
85
86  Use the test::Suite class liberally. See other tests for examples of
87  typical usage.
88
89  Set `need_test_repo` to true in Suite constructor if the test uses
90  the test repository. That will result in the test being skipped if
91  test repo is not available, or if it is available the wc will
92  checked out (if needed) and updated.
93
94  If a test needs a data file, place the data in directory
95  'test/data'. However, you should feel free to re-use the data files
96  already there. It should be possible to run the test suite also in a
97  VPATH build in which builddir and srcdir are not the same. To avoid
98  problems with files not found in VPATH builds, use the function
99  test::src_filename() which gives you the absolute path to the
100  file. If the data file is generated during build, e.g., checked out
101  in a wc and residing in builddir, use test:filename instead.
102
103  Add the test to check_PROGRAMS in Makefile.am.
104
105Shell tests
106-----------
107
108  Use `required=...` for setting a space delimited list of required
109  tools (see other tests and init.sh). If `repo` is included in
110  `required`, a toy_project wc will be checked out and updated if test
111  repo is available, otherwise test is skipped.
112
113  Include init.sh, with `. ./init.sh`.
114
115  Use variables set in init.sh such as $srcdir, $SVNDIGEST, and $SVN.
116
117  Do not use `exit`, but use functions `exit_fail()`, `exit_skip()`,
118  or `exit_success()`.
119
120  Make sure to set svn property `svn:executable` on test.
121
122  Add test to `distributed_TESTS` in Makefile.am. In order to get the
123  lazycheck to behave as expected, declare accurate dependencies. If
124  test 'foo.sh', for example, depends on file 'bar.txt', create a make
125  rule: 'foo.log: bar.txt'.
126
127======================================================================
128Copyright (C) 2009, 2010 Peter Johansson
129
130This file is part of svndigest, http://dev.thep.lu.se/svndigest
131
132The svndigest is free software; you can redistribute it and/or modify
133it under the terms of the GNU General Public License as published by
134the Free Software Foundation; either version 3 of the License, or (at
135your option) any later version.
136
137The svndigest is distributed in the hope that it will be useful, but
138WITHOUT ANY WARRANTY; without even the implied warranty of
139MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
140General Public License for more details.
141
142You should have received a copy of the GNU General Public License
143along with svndigest. If not, see <http://www.gnu.org/licenses/>.
144======================================================================
145
146
Note: See TracBrowser for help on using the repository browser.