source: trunk/test/README @ 974

Last change on this file since 974 was 974, checked in by Peter Johansson, 12 years ago

extend list of tips when writing tests

  • Property svn:eol-style set to native
  • Property svn:keywords set to Id
File size: 4.1 KB
Line 
1$Id: README 974 2009-12-11 04:33:48Z peter $
2
3    The svndigest test suite
4
5
6Usage:
7========
8
9Running all tests
10-----------------
11
12  make check
13
14
15Interpretation
16--------------
17
18  Successes:
19    PASS  - success
20    XFAIL - expected failure
21
22  Failures:
23    FAIL  - failure
24    XPASS - unexpected success
25
26  Other:
27    SKIP  - skipped test
28
29
30Getting details from failures
31-----------------------------
32
33  You can run a specific test directly. Add a '-v' flag to trigger verbosity
34
35    ./program_test -v
36
37  If you want to run the test suite in verbose mode, you can invoke
38  make as follows:
39
40    make check VERBOSE=1
41
42  If you want to run only a subset of tests, you can invoke make as follows
43
44    make check TESTS="first_test second_test"
45
46
47Reporting failures
48------------------
49
50  Open a 'New Ticket' in the issue tracking system
51  (http://dev.thep.lu.se/svndigest) with verbose output of failing
52  tests along with which version of svndigest and which operating
53  system etc. Alternatively, you can send a bug report to the address
54  given in the failed `make check' or in `configure --help'.
55
56
57
58Writing tests
59=============
60
61  If you plan to fix a bug, write the test first. This way you will
62  make sure the test catches the bug, and that it succeeds once you
63  have fixed the bug.
64
65  Add a copyright and license followed by one or two lines describing
66  what the test does.
67
68  Tests shall returns 0 at success, 77 if it should be skipped, and
69  something else at failure. Tests should be silent also at failure
70  unless option `-v` is passed or env variable VERBOSE is set.
71
72  Add the test to XFAIL_TESTS in Makefile.am, if it tests a bug still
73  present, i.e., the test is expected to fail.
74
75C++ tests
76---------
77
78  Use the test::Suite class liberally. See other tests for examples of
79  typical usage.
80
81  Set `need_test_repo` to true in Suite constructor if the test uses
82  the test repository. That will result in the test being skipped if
83  test repo is not available, or if it is available the wc will
84  checked out (if needed) and updated.
85
86  Send output via test::Suite::out() which redirects the output to
87  cout or /dev/null depending on whether VERBOSE and option `-v` are set.
88
89  If a test needs a data file, place the data in directory
90  'test/data'. However, you should feel free to re-use the data files
91  already there. It should be possible to run the test suite also in a
92  VPATH build in which builddir and srcdir are not the same. To avoid
93  problems with files not found in VPATH builds, use the function
94  test::src_filename() which gives you the absolute path to the
95  file. If the data file is generated during build, e.g., checked out
96  in a wc and residing in builddir, use test:filename instead.
97
98  Add the test to check_PROGRAMS in Makefile.am.
99
100Shell tests
101-----------
102
103  Use `required=...` for setting a space delimited list of required
104  tools (see other tests and init.sh). If `repo` is included in
105  `required`, a toy_project wc will be checked out and updated if test
106  repo is available, otherwise test is skipped.
107
108  Include init.sh, with `. ./init.sh`.
109
110  Use variables set in init.sh such as $srcdir, $SVNDIGEST, and $SVN.
111
112  Do not use `exit`, but use functions `exit_fail()`, `exit_skip()`,
113  or `exit_success()`.
114
115  Make sure to set svn property `svn:executable` on test.
116
117  Add test to `distributed_TESTS` in Makefile.am.
118
119======================================================================
120Copyright (C) 2009 Peter Johansson
121
122This file is part of svndigest, http://dev.thep.lu.se/svndigest
123
124The svndigest is free software; you can redistribute it and/or modify
125it under the terms of the GNU General Public License as published by
126the Free Software Foundation; either version 3 of the License, or (at
127your option) any later version.
128
129The svndigest is distributed in the hope that it will be useful, but
130WITHOUT ANY WARRANTY; without even the implied warranty of
131MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
132General Public License for more details.
133
134You should have received a copy of the GNU General Public License
135along with svndigest. If not, see <http://www.gnu.org/licenses/>.
136======================================================================
137
138
Note: See TracBrowser for help on using the repository browser.