mirror of
https://github.com/openmc-dev/openmc.git
synced 2026-07-27 05:35:49 -04:00
Update documentation relating to test suite
This commit is contained in:
Paul Romano
parent
dd2415d350
commit
fe353aff40
6 changed files with 79 additions and 195 deletions
|
|
@ -10,9 +10,10 @@ as debugging.
|
|||
|
||||
.. toctree::
|
||||
:numbered:
|
||||
:maxdepth: 3
|
||||
:maxdepth: 2
|
||||
|
||||
styleguide
|
||||
workflow
|
||||
tests
|
||||
user-input
|
||||
docbuild
|
||||
|
|
|
|||
73
docs/source/devguide/tests.rst
Normal file
73
docs/source/devguide/tests.rst
Normal file
|
|
@ -0,0 +1,73 @@
|
|||
.. _devguide_tests:
|
||||
|
||||
==========
|
||||
Test Suite
|
||||
==========
|
||||
|
||||
Running Tests
|
||||
-------------
|
||||
|
||||
The OpenMC test suite consists of two parts, a regression test suite and a unit
|
||||
test suite. The regression test suite is based on regression or integrated
|
||||
testing where different types of input files are configured and the full OpenMC
|
||||
code is executed. Results from simulations are compared with expected
|
||||
results. The unit tests are primarily intended to test individual
|
||||
functions/classes in the OpenMC Python API.
|
||||
|
||||
The test suite relies on the third-party `pytest <https://pytest.org>`_
|
||||
package. To run either or both the regression and unit test suites, it is
|
||||
assumed that you have OpenMC fully installed, i.e., the :ref:`scripts_openmc`
|
||||
executable is available on your :envvar:`PATH` and the :mod:`openmc` Python
|
||||
module is importable. In development where it would be onerous to continually
|
||||
install OpenMC every time a small change is made, it is recommended to install
|
||||
OpenMC in development/editable mode. With setuptools, this is accomplished by
|
||||
running::
|
||||
|
||||
python setup.py develop
|
||||
|
||||
or using pip (recommended)::
|
||||
|
||||
pip install -e .[test]
|
||||
|
||||
It is also assumed that you have cross section data available that is pointed to
|
||||
by the :envvar:`OPENMC_CROSS_SECTIONS` and :envvar:`OPENMC_MULTIPOLE_LIBRARY`
|
||||
environment variables. Furthermore, to run unit tests for the :mod:`openmc.data`
|
||||
module, it is necessary to have ENDF/B-VII.1 data available and pointed to by
|
||||
the :envvar:`OPENMC_ENDF_DATA` environment variable. All data sources can be
|
||||
obtained using the ``tools/ci/travis-before-script.sh`` script.
|
||||
|
||||
To execute the test suite, go to the ``tests/`` directory and run::
|
||||
|
||||
pytest
|
||||
|
||||
If you want to collect information about source line coverage in the Python API,
|
||||
you must have the `pytest-cov <https://pypi.python.org/pypi/pytest-cov>`_ plugin
|
||||
installed and run::
|
||||
|
||||
pytest --cov=../openmc --cov-report=html
|
||||
|
||||
Adding Tests to the Regression Suite
|
||||
------------------------------------
|
||||
|
||||
To add a new test to the regression test suite, create a sub-directory in the
|
||||
``tests/regression_tests/`` directory. To configure a test you need to add the
|
||||
following files to your new test directory:
|
||||
|
||||
* OpenMC input XML files, if they are not generated through the Python API
|
||||
* **test.py** - Python test driver script; please refer to other tests to
|
||||
see how to construct. Any output files that are generated during testing
|
||||
must be removed at the end of this script.
|
||||
* **inputs_true.dat** - ASCII file that contains Python API-generated XML
|
||||
files concatenated together. When the test is run, inputs that are
|
||||
generated are compared to this file.
|
||||
* **results_true.dat** - ASCII file that contains the expected results from
|
||||
the test. The file *results_test.dat* is compared to this file during the
|
||||
execution of the python test driver script. When the above files have been
|
||||
created, generate a *results_test.dat* file and copy it to this name and
|
||||
commit. It should be noted that this file should be generated with basic
|
||||
compiler options during openmc configuration and build (e.g., no MPI, no
|
||||
debug/optimization).
|
||||
|
||||
In addition to this description, please see the various types of tests that are
|
||||
already included in the test suite to see how to create them. If all is
|
||||
implemented correctly, the new test will automatically be discovered by pytest.
|
||||
|
|
@ -89,139 +89,6 @@ features and bug fixes. The general steps for contributing are as follows:
|
|||
6. After the pull request has been thoroughly vetted, it is merged back into the
|
||||
*develop* branch of mit-crpg/openmc.
|
||||
|
||||
.. _test suite:
|
||||
|
||||
OpenMC Test Suite
|
||||
-----------------
|
||||
|
||||
The purpose of this test suite is to ensure that OpenMC compiles using various
|
||||
combinations of compiler flags and options, and that all user input options can
|
||||
be used successfully without breaking the code. The test suite is comprised of
|
||||
regression tests where different types of input files are configured and the
|
||||
full OpenMC code is executed. Results from simulations are compared with
|
||||
expected results. The test suite is comprised of many build configurations
|
||||
(e.g. debug, mpi, hdf5) and the actual tests which reside in sub-directories
|
||||
in the tests directory. We recommend to developers to test their branches
|
||||
before submitting a formal pull request using gfortran and Intel compilers
|
||||
if available.
|
||||
|
||||
The test suite is designed to integrate with cmake using ctest_. It is
|
||||
configured to run with cross sections from NNDC_ augmented with 0 K elastic
|
||||
scattering data for select nuclides as well as multipole data. To download the
|
||||
proper data, run the following commands:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
wget -O nndc_hdf5.tar.xz $(cat <openmc_root>/.travis.yml | grep anl.box | awk '{print $2}')
|
||||
tar xJvf nndc_hdf5.tar.xz
|
||||
export OPENMC_CROSS_SECTIONS=$(pwd)/nndc_hdf5/cross_sections.xml
|
||||
|
||||
git clone --branch=master git://github.com/smharper/windowed_multipole_library.git wmp_lib
|
||||
tar xzvf wmp_lib/multipole_lib.tar.gz
|
||||
export OPENMC_MULTIPOLE_LIBRARY=$(pwd)/multipole_lib
|
||||
|
||||
The test suite can be run on an already existing build using:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
cd build
|
||||
make test
|
||||
|
||||
or
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
cd build
|
||||
ctest
|
||||
|
||||
There are numerous ctest_ command line options that can be set to have
|
||||
more control over which tests are executed.
|
||||
|
||||
Before running the test suite python script, the following environmental
|
||||
variables should be set if the default paths are incorrect:
|
||||
|
||||
* **FC** - The command for a Fortran compiler (e.g. gfotran, ifort).
|
||||
|
||||
* Default - *gfortran*
|
||||
|
||||
* **CC** - The command for a C compiler (e.g. gcc, icc).
|
||||
|
||||
* Default - *gcc*
|
||||
|
||||
* **CXX** - The command for a C++ compiler (e.g. g++, icpc).
|
||||
|
||||
* Default - *g++*
|
||||
|
||||
* **MPI_DIR** - The path to the MPI directory.
|
||||
|
||||
* Default - */opt/mpich/3.2-gnu*
|
||||
|
||||
* **HDF5_DIR** - The path to the HDF5 directory.
|
||||
|
||||
* Default - */opt/hdf5/1.8.16-gnu*
|
||||
|
||||
* **PHDF5_DIR** - The path to the parallel HDF5 directory.
|
||||
|
||||
* Default - */opt/phdf5/1.8.16-gnu*
|
||||
|
||||
To run the full test suite, the following command can be executed in the
|
||||
tests directory:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python run_tests.py
|
||||
|
||||
A subset of build configurations and/or tests can be run. To see how to use
|
||||
the script run:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python run_tests.py --help
|
||||
|
||||
As an example, say we want to run all tests with debug flags only on tests
|
||||
that have cone and plot in their name. Also, we would like to run this on
|
||||
4 processors. We can run:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python run_tests.py -j 4 -C debug -R "cone|plot"
|
||||
|
||||
Note that standard regular expression syntax is used for selecting build
|
||||
configurations and tests. To print out a list of build configurations, we
|
||||
can run:
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
python run_tests.py -p
|
||||
|
||||
Adding tests to test suite
|
||||
++++++++++++++++++++++++++
|
||||
|
||||
To add a new test to the test suite, create a sub-directory in the tests
|
||||
directory that conforms to the regular expression *test_*. To configure
|
||||
a test you need to add the following files to your new test directory,
|
||||
*test_name* for example:
|
||||
|
||||
* OpenMC input XML files
|
||||
* **test_name.py** - Python test driver script, please refer to other
|
||||
tests to see how to construct. Any output files that are generated
|
||||
during testing must be removed at the end of this script.
|
||||
* **inputs_true.dat** - ASCII file that contains Python API-generated XML
|
||||
files concatenated together. When the test is run, inputs that are
|
||||
generated are compared to this file.
|
||||
* **results_true.dat** - ASCII file that contains the expected results
|
||||
from the test. The file *results_test.dat* is compared to this file
|
||||
during the execution of the python test driver script. When the
|
||||
above files have been created, generate a *results_test.dat* file and
|
||||
copy it to this name and commit. It should be noted that this file
|
||||
should be generated with basic compiler options during openmc
|
||||
configuration and build (e.g., no MPI/HDF5, no debug/optimization).
|
||||
|
||||
In addition to this description, please see the various types of tests that
|
||||
are already included in the test suite to see how to create them. If all is
|
||||
implemented correctly, the new test directory will automatically be added
|
||||
to the CTest framework.
|
||||
|
||||
Private Development
|
||||
-------------------
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue