There are two directories in the PyTrilinos package that provide python scripts for testing PyTrilinos, example and test. Generally speaking, unit tests go in the test directory and descriptive or demonstrative example scripts go in example.
Unit tests scripts shall begin with test, followed by the package name and an underscore, followed by a short description of the test, typically the class name being tested. For example:
testTeuchos_ParameterList testEpetra_MultiVector
are the base names of the unit tests for Teuchos.ParameterList and Epetra.MultiVector, respectively. In certain situations, the underscore and test description can be omitted.
Example scripts shall begin with ex, followed by the name of the primary package being demonstrated, followed by an underscore and a short description of the test. For example:
exEpetra_Comm exAztecOO_Operator
are the base names for example scripts demonstrating the Epetra.Comm class and an AztecOO solver that uses an Epetra.Operator, respectively. In certain situations, the underscore and test description can be omitted.
Running make in either the example or test directories simply copies the test scripts from the source directory to the build directory. Autotools requires that the source and build names be different, to support building in the source directory. To meet this requirement, the source files have the suffix .src and the build files have the suffix .py.
The Makefile in the test directories control which scripts get copied to the build directory, via the PACKAGE_TESTS and PACKAGE_EXAMPLES variables defined in test/Makefile.am and example/Makefile.am, respectively, where PACKAGE is replaced by the names of all the packages with python wrappers. The values of these variables should be conditional based on what packages are enabled. If a test imports a given PyTrilinos package, then it should be conditionally enabled based on the value of HAVE_PYTRILINOS_PACKAGE, where PACKAGE is replaced by the name of the package.
To run all of the enabled tests, first make sure they are all up-to-date:
$ cd BUILD/packages/PyTrilinos $ make tests examples
Then you can use the test harness make target to run all of the tests:
$ make runtests-serial
or:
$ make runtests-mpi
The results of all of the tests can be found in test/runtests-results directory.
All test scripts shall use the optparse module to parse command line options and support the following options:
-t, --testharness test local build modules; prevent loading system-installed modules -v VERBOSITY, --verbosity=VERBOSITY set the verbosity level [default 2]
The -t option is to force use of the locally-built PyTrilinos, preventing the importing of any installed PyTrilinos modules. The verbosity option is used in all test scripts and optionally in any example scripts where it makes sense.
Tests scripts use the setpath module, local to both test and example directories, to set the python search path to find the local PyTrilinos modules. Locally built modules are imported directly:
import Teuchos import Epetra
while system-installed modules are imported like this:
from PyTrilinos import Teuchos from PyTrilinos import Epetra
See any test script for examples and how the -t option is enforced.
Test scripts shall run in both serial or parallel. You may use either:
comm = Teuchos.DefaultComm.getComm()
or:
comm = Epetra.PyComm()
to obtain an appropriate communicator object for the test scripts. By convention, set the variable iAmRoot to either True or False depending on whether the communicator's rank is 0.
The test script shall output End Result: TEST PASSED if the test passes correctly. This helps the Trilinos test harness determine which tests pass and which do not, especially in parallel.
Unit tests are based on the unittest python library module. Test case classes inherit from unittest.TestCase. Individual tests are implemented as methods of these classes that begin with test. See the python documentation (http://www.python.org) for details.
Each unit test script can have one or more TestCase classes. In main(), each test case class should be added to a unittest.TestSuite object named suite.
Unit tests shall print, from processor 0, a header with a message such as "Testing Epetra.Object" with a line of asterisks above and below the message:
********************* Testing Epetra.Object *********************
The suite of tests should be run with:
verbosity = options.verbosity * int(iAmRoot) result = unittest.TextTestRunner(verbosity=verbosity).run(suite)
and the success should be determined and output via (for the case of an Epetra communicator):
errsPlusFails = comm.SumAll(len(result.errors) + len(result.failures)) if errsPlusFails == 0 and iAmRoot: print "End Result: TEST PASSED" sys.exit(errsPlusFails)
If there are no errors and no failures on any processors, then the test will pass.
Example scripts are more free-form and should be written for readability, to make for clear demonstrations of PyTrilinos usage. However, it is encouraged that example script output be consistent with unit test output whenever possible.