Changeset 1640 for trunk/test/README

Timestamp:

Dec 12, 2008, 9:52:16 PM (13 years ago)

Author:

Peter

Message:

adding some text on how to use the test suite - and also some rules I think we have decided on. refs #458

File:

• trunk/test/README (modified) (2 diffs)

trunk/test/README

@@ -1 +1 @@
 $Id$
+
+    The yat test suite
+
+
+Usage:
+========
+
+Running all tests
+-----------------
+
+  make check
+
+
+Interpretation
+--------------
+
+  Successes:
+    PASS  - success
+    XFAIL - expected failure
+
+  Failures:
+    FAIL  - failure
+    XPASS - unexpected success
+
+  Other:
+    SKIP  - skipped test
+
+
+Getting details from failures
+-----------------------------
+
+  You can run a specific test directly. Add a '-v' flag to trigger verbosity
+
+    ./program_test -v
+
+  If you want to run the test suite in verbose mode, you can invoke
+  make as follows:
+
+    make YAT_TEST_VERBOSE=1 check
+
+  If you want to run only a subset of tests, you can invoke make as follows
+
+    make TESTS="first_test second_test" check
+
+
+Reporting failures
+------------------
+
+  Open a 'New Ticket' in the issue tracking system
+  (http://dev.thep.lu.se/yat) with verbose output of failing tests
+  along with which version of yat and which operating system
+  etc. Alternatively, you can send a bug report to the address given in
+  the failed `make check' or in `configure --help'.
+
+
+
+Writing tests
+=============
+
+  If you plan to fix a bug, write the test first. Thsi way you will
+  make sure the test catches the bug, and that it succeeds once you
+  have fixed the bug.
+
+  Use the test::Suite class liberally. See other tests for examples of
+  typical usage.
+
+  Tests shall return 0 at success and -1 otherwise (see
+  test::Suite::return_value()). If the test should be skipped for some
+  reason it shall return 77.
+
+  When a test is run without verbose flag, it should be silent also at
+  failure. Achieve this behaviour by sending output through
+  test::Suite class (see functions err() and out()).
+
+  If a test grows large, avoid having several tests in one long
+  main. It creates long correlations and complicates debugging and
+  extending the test. If you don't want to split the test into several
+  files, split the test into several independent sub-functions.
+
 
 ======================================================================
@@ -24 +103 @@
 
 
-usage:
-       program_test
-
-no print out, returns 0 if test is succesful.
-
-       program_test -v
-
-verbose mode for printing extra information.
-  
-Verbose mode can also be achieved by setting environment variable
-YAT_TEST_VERBOSE=1, which also works when running make:
-
-       YAT_TEST_VERBOSE=1 make check
-