| 1 | $Id: README 1642 2008-12-12 21:09:19Z peter $ |
|---|
| 2 | |
|---|
| 3 | The yat test suite |
|---|
| 4 | |
|---|
| 5 | |
|---|
| 6 | Usage: |
|---|
| 7 | ======== |
|---|
| 8 | |
|---|
| 9 | Running all tests |
|---|
| 10 | ----------------- |
|---|
| 11 | |
|---|
| 12 | make check |
|---|
| 13 | |
|---|
| 14 | |
|---|
| 15 | Interpretation |
|---|
| 16 | -------------- |
|---|
| 17 | |
|---|
| 18 | Successes: |
|---|
| 19 | PASS - success |
|---|
| 20 | XFAIL - expected failure |
|---|
| 21 | |
|---|
| 22 | Failures: |
|---|
| 23 | FAIL - failure |
|---|
| 24 | XPASS - unexpected success |
|---|
| 25 | |
|---|
| 26 | Other: |
|---|
| 27 | SKIP - skipped test |
|---|
| 28 | |
|---|
| 29 | |
|---|
| 30 | Getting details from failures |
|---|
| 31 | ----------------------------- |
|---|
| 32 | |
|---|
| 33 | You can run a specific test directly. Add a '-v' flag to trigger verbosity |
|---|
| 34 | |
|---|
| 35 | ./program_test -v |
|---|
| 36 | |
|---|
| 37 | If you want to run the test suite in verbose mode, you can invoke |
|---|
| 38 | make as follows: |
|---|
| 39 | |
|---|
| 40 | make YAT_TEST_VERBOSE=1 check |
|---|
| 41 | |
|---|
| 42 | If you want to run only a subset of tests, you can invoke make as follows |
|---|
| 43 | |
|---|
| 44 | make TESTS="first_test second_test" check |
|---|
| 45 | |
|---|
| 46 | |
|---|
| 47 | Reporting failures |
|---|
| 48 | ------------------ |
|---|
| 49 | |
|---|
| 50 | Open a 'New Ticket' in the issue tracking system |
|---|
| 51 | (http://dev.thep.lu.se/yat) with verbose output of failing tests |
|---|
| 52 | along with which version of yat and which operating system |
|---|
| 53 | etc. Alternatively, you can send a bug report to the address given in |
|---|
| 54 | the failed `make check' or in `configure --help'. |
|---|
| 55 | |
|---|
| 56 | |
|---|
| 57 | |
|---|
| 58 | Writing tests |
|---|
| 59 | ============= |
|---|
| 60 | |
|---|
| 61 | If you plan to fix a bug, write the test first. This way you will |
|---|
| 62 | make sure the test catches the bug, and that it succeeds once you |
|---|
| 63 | have fixed the bug. |
|---|
| 64 | |
|---|
| 65 | Use the test::Suite class liberally. See other tests for examples of |
|---|
| 66 | typical usage. |
|---|
| 67 | |
|---|
| 68 | Tests shall return 0 at success and -1 otherwise (see |
|---|
| 69 | test::Suite::return_value()). If the test should be skipped for some |
|---|
| 70 | reason it shall return 77. |
|---|
| 71 | |
|---|
| 72 | When a test is run without verbose flag, it should be silent also at |
|---|
| 73 | failure. Achieve this behaviour by sending output through |
|---|
| 74 | test::Suite class (see functions err() and out()). |
|---|
| 75 | |
|---|
| 76 | If a test needs a data file, place the data in directory |
|---|
| 77 | 'test/data'. However, you should feel free to re-use the data files |
|---|
| 78 | already there. It should be possible to run test from any directory, |
|---|
| 79 | in particular from builddir in a VPATH build. To achieve this, use |
|---|
| 80 | test::Suite::filename() to get the absolute path to the file. |
|---|
| 81 | |
|---|
| 82 | If a test grows large, avoid having several tests in one long |
|---|
| 83 | main. It creates long correlations and complicates debugging and |
|---|
| 84 | extending the test. If you don't want to split the test into several |
|---|
| 85 | files, split the test into several independent sub-functions. |
|---|
| 86 | |
|---|
| 87 | |
|---|
| 88 | ====================================================================== |
|---|
| 89 | Copyright (C) 2005 Peter Johansson |
|---|
| 90 | Copyright (C) 2006 Jari Häkkinen |
|---|
| 91 | Copyright (C) 2007 Jari Häkkinen, Peter Johansson |
|---|
| 92 | Copyright (C) 2008 Peter Johansson |
|---|
| 93 | |
|---|
| 94 | This file is part of yat library, http://dev.thep.lu.se/yat |
|---|
| 95 | |
|---|
| 96 | The yat library is free software; you can redistribute it and/or |
|---|
| 97 | modify it under the terms of the GNU General Public License as |
|---|
| 98 | published by the Free Software Foundation; either version 3 of the |
|---|
| 99 | License, or (at your option) any later version. |
|---|
| 100 | |
|---|
| 101 | The yat library is distributed in the hope that it will be useful, but |
|---|
| 102 | WITHOUT ANY WARRANTY; without even the implied warranty of |
|---|
| 103 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU |
|---|
| 104 | General Public License for more details. |
|---|
| 105 | |
|---|
| 106 | You should have received a copy of the GNU General Public License |
|---|
| 107 | along with yat. If not, see <http://www.gnu.org/licenses/>. |
|---|
| 108 | ====================================================================== |
|---|
| 109 | |
|---|
| 110 | |
|---|
