Docs: Clean old-Makefile mentions (#5541)

This commit is contained in:
SY Wang 2026-07-07 17:46:50 +08:00 committed by GitHub
parent fe787632bd
commit b07e6de83f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
15 changed files with 298 additions and 748 deletions

View file

@ -42,11 +42,14 @@
([#5257](https://github.com/cp2k/cp2k/pull/5257))
- Drop support for GCC 8 ([#5290](https://github.com/cp2k/cp2k/pull/5290))
- Refactor DOS/PDOS input section ([#5326](https://github.com/cp2k/cp2k/pull/5326))
- Remove obsolete Cython Python bindings ([#5541](https://github.com/cp2k/cp2k/pull/5541))
- **TODO**
### Fixes
- Fix issues in the native DFT-D4 implementation. ([#5030](https://github.com/cp2k/cp2k/pull/5030))
- Analytical periodic-subspace stress for 2D systems with ANALYTIC/MT
([#5282](https://github.com/cp2k/cp2k/pull/5282))
- **TODO**
______________________________________________________________________

View file

@ -161,5 +161,7 @@ central redirections, i.e. avoid to use direct calls to external libraries in CP
- The following keywords are required: `\brief`, `\param` (for each parameter), `\retval` (for
functions).
- The following keywords are optional: `\note`, `\par`, `\date`, `\author`.
- Please run `make doxify` to format your doxygen comments, or generate templates where none exist.
- Run `./tools/precommit/precommit.py --allow-modifications <file>` before committing modified
Fortran sources. It runs Doxify before applying the Fortran formatter. To update Doxygen blocks
only, run `./tools/doxify/doxify.sh <file>`.
- See our [doxygen pages](http://doxygen.cp2k.org/files.html) for the result.

View file

@ -1,292 +1,326 @@
# Regression Testing
CP2K comes with over 5000 test input files (located in `tests` which serve as both examples on how
to use the many features in CP2K and also as a method for developers to test modifications and
extensions to CP2K. In order to reduce the chance of bugs being introduced into the code, and ensure
that all parts of the code are working. We also recommend that all users complete a test before
using a self-compiled binary for their projects.
CP2K comes with a large collection of test inputs in `tests/`. They serve both as examples of how to
use CP2K features and as regression tests for changes to the code. Running the relevant tests before
and after a modification reduces the risk of introducing unintended regressions. Users are also
strongly encouraged to run at least a representative test set before relying on a self-built CP2K
binary for production calculations.
## Dashboard
A number of regtests are run automatically by various members of our community. The results of these
tests are collected centrally at the [Dashboard](http://dashboard.cp2k.org). If errors are detected,
the developer responsible for the change should fix it immediately. The output logs provide the arch
file used for these tests, which might suggest useful settings for that particular architecture.
A number of regression-test configurations are run automatically by members of the CP2K community.
Their results are collected on the [CP2K Dashboard](https://dashboard.cp2k.org). If a dashboard
failure is associated with a change, it should be investigated before that change is merged. The
corresponding logs and configuration information can also be useful when reproducing a failure on a
similar system.
## Code Coverage
We aim that the regression test suite covers all the functionality of CP2K. For this purpose we
regularly create [Coverage reports](http://www.cp2k.org/static/coverage/) of the test-suite. If you
see parts of the code which are not well tested, please contribute to improving coverage by writing
new tests!
We aim for the regression suite to exercise as much of CP2K as practical. Regular
[coverage reports](https://www.cp2k.org/static/coverage/) help identify poorly tested paths. When
adding functionality, please add a focused test where possible; improving coverage is a useful
contribution in its own right.
## How does it work?
The regression test suite is run using the
[do_regtest](https://github.com/cp2k/cp2k/blob/master/tests/do_regtest.py) script. It performs the
following tasks:
The regression suite is run by
[`tests/do_regtest.py`](https://github.com/cp2k/cp2k/blob/master/tests/do_regtest.py). The driver:
- executes a list of tests
- compares the results (outputs) with those of the last known result (reference)
- produces a summary
- runs the standalone unit-test executables listed in `tests/UNIT_TESTS`;
- runs the input-based tests listed through `tests/TEST_DIRS` and their `TEST_FILES.toml` files;
- compares selected output values with validated references; and
- prints a summary and writes detailed failures to `error_summary` in the test work directory.
The driver reads the CP2K feature flags from `cp2k.<version> --version`, so test directories whose
requirements are not met by the executable are skipped automatically.
## Running the regtests
### Step 0: Build the executable
- Build the executable (see [Building from Source](../getting-started/build-from-source.md) and
[Building with Spack](../getting-started/build-with-spack.md)).
Build CP2K first; see [Building from Source](../getting-started/build-from-source.md) or
[Building with Spack](../getting-started/build-with-spack.md). A full run requires the selected
`cp2k.<version>` executable and the corresponding `*_unittest.<version>` executables in the same
binary directory. For example, the `psmp` variant normally uses a CMake build directory containing
`build/bin/cp2k.psmp` and the `*.psmp` unit-test executables.
### Step 1: Preparation
- Decide on a directory for doing the regtest, there will be plenty of files in this dir (after a
while) so make it something like `$HOME/rt`
- Clone a version of cp2k into `$HOME/rt`.
- Set up the arch files so that you can cleanly build cp2k (test this)
- Use a CP2K source checkout that matches the binaries being tested. The checkout provides
`tests/do_regtest.py`, the test inputs and references, and the default `data/` directory.
- Choose a work base directory outside the source tree, for example `$HOME/cp2k-regtesting`. The
driver creates a timestamped `TEST-YYYY-MM-DD_HH-MM-SS` directory below it for each run.
- Decide how many MPI ranks and OpenMP threads one individual test should use, and how many CPU
tasks are available to run several tests concurrently.
```{important}
The driver copies the `tests/` tree into its work directory. The work base directory must not be
inside the source tree's `tests/` directory. Testing a binary against a substantially different
source revision can also make numerical differences difficult to interpret.
```
### Step 2: Running
The current interface is:
```text
./tests/do_regtest.py [options] <binary-dir> <version>
```
$ tests/do_regtest.py -h
`<binary-dir>` is the directory containing the executables. `<version>` is their suffix, such as
`psmp`, `ssmp`, `pdbg`, or `sdbg`. Thus, for `<binary-dir> = ./build/bin` and `<version> = psmp`,
the driver runs `./build/bin/cp2k.psmp` and the matching unit-test executables.
A typical local MPI/OpenMP run is:
```bash
./tests/do_regtest.py \
--mpiranks 2 \
--ompthreads 2 \
--maxtasks 8 \
--workbasedir "$HOME/cp2k-regtesting" \
./build/bin psmp
```
The available options are:
```text
usage: do_regtest.py [-h] [--mpiranks MPIRANKS] [--ompthreads OMPTHREADS]
[--maxtasks MAXTASKS] [--num_gpus NUM_GPUS]
[--timeout TIMEOUT] [--maxerrors MAXERRORS]
[--mpiexec MPIEXEC] [--smoketest] [--valgrind]
[--keepalive] [--flagslow] [--debug]
[--restrictdir RESTRICTDIR] [--skipdir SKIPDIR]
[--workbasedir WORKBASEDIR] arch version
[--workbasedir WORKBASEDIR] [--cp2kdatadir CP2KDATADIR]
[--skip_unittests] [--skip_regtests]
binary_dir version
```
Runs CP2K regression test suite.
`--mpiranks` specifies the number of MPI ranks used by **each individual test**, not the number of
ranks allocated to the whole run. `--ompthreads` specifies the number of OpenMP threads per rank.
For serial variants (`ssmp` and `sdbg`), the driver uses one MPI rank regardless of `--mpiranks`.
positional arguments:
arch
version
`--maxtasks` is the total CPU-task budget available to the driver. It limits the number of test
batches that can run concurrently; approximately
options:
-h, --help show this help message and exit
--mpiranks MPIRANKS
--ompthreads OMPTHREADS
--maxtasks MAXTASKS
--num_gpus NUM_GPUS
--timeout TIMEOUT
--maxerrors MAXERRORS
--mpiexec MPIEXEC
--smoketest Runs only the first test of each directory.
--valgrind Runs tests under Valgrind memcheck. Best used together with --keepalive.
--keepalive Use a persistent cp2k-shell process to reduce startup time.
--flagslow Flag slow tests in the final summary and status report.
--debug
--restrictdir RESTRICTDIR
--skipdir SKIPDIR
--workbasedir WORKBASEDIR
```text
floor(maxtasks / (mpiranks * ompthreads))
```
batches can run at the same time, with at least one worker. Set it to the CPU capacity actually
available to the test run.
Other commonly useful options are:
- `--smoketest`: run only the first regression input in each selected directory, for a quick broad
check;
- `--restrictdir REGEX` and `--skipdir REGEX`: include or exclude test directories by regular
expression; either option may be repeated;
- `--skip_unittests` or `--skip_regtests`: deliberately omit one part of the suite;
- `--timeout SECONDS` and `--maxerrors N`: limit the duration of individual tests and the number of
errors before aborting;
- `--mpiexec 'COMMAND ... {N} ...'`: use a site-specific MPI launcher, where `{N}` is replaced by
the value of `--mpiranks`;
- `--keepalive`: reuse a persistent `cp2k --shell` process for supported directories to reduce
startup overhead;
- `--valgrind`: run executables under Valgrind memcheck, usually together with `--keepalive`;
- `--flagslow`: identify unusually slow tests; and
- `--cp2kdatadir DIR`: use a CP2K data directory other than the source tree's `data/` directory.
For the complete option list of the checked-out version, run:
```bash
./tests/do_regtest.py --help
```
For example, to check only the `QS/regtest-*` directories:
```bash
./tests/do_regtest.py \
--restrictdir 'QS/regtest-.*' \
--workbasedir "$HOME/cp2k-regtesting" \
./build/bin psmp
```
### Step 3: Interpretation
A test results can be any of the following:
The driver prints the work directory for every completed test batch and writes detailed error
messages to `error_summary`. A test result can be one of the following:
| Test Result | Meaning |
| :---------------: | :---------------------------------------------------------------------------------------- |
| `OK` | if the results match those of a previous run precisely. The execution time is also given. |
| `RUNTIME FAILURE` | if they stopped unexpectedly (e.g. core dump, or stop) |
| `WRONG RESULT` | if they produce a result that deviates (even a tiny bit) from an old reference |
- **`OK`**: The executable completed and all requested comparisons passed. The execution time is
also shown.
- **`RUNTIME FAIL`**: The CP2K or unit-test executable stopped unexpectedly or returned a nonzero
status.
- **`WRONG RESULT`**: The calculation completed, but at least one compared quantity differs from its
reference.
- **`TIMED OUT`**: The test exceeded the configured `--timeout`.
- **`HUGE OUTPUT`**: The test produced more than 2 MiB of output.
- **`N/A`**: The selected matcher determined that the comparison is not applicable.
The last two outcomes generally mean that a bug has been introduced, which requires investigation.
Since regtesting only yields information relative to a previously known result, it is most useful to
do a regtest before and after you make changes. To allow per-test numerical difference higher than
that set as a default, add third column in appropriate TEST_FILES file with a relative value of the
difference.
`RUNTIME FAIL` and `WRONG RESULT` generally require investigation. A wrong result does not by itself
justify changing the reference: first determine whether it is an intended and scientifically valid
effect of the change, a platform-dependent numerical difference, or a regression. Because the suite
compares against known references, running the relevant tests both before and after a change is
particularly informative.
## Adding Tests
The test-suite is fully controlled by the following files in the
[tests](https://github.com/cp2k/cp2k/tree/master/tests) directories.
The test suite is controlled by files in the
[`tests`](https://github.com/cp2k/cp2k/tree/master/tests) directory:
- `TEST_DIRS`: This is just a list of directories that contains tests. You can add your directory
here. Conditions on the CP2K executable (linked dependencies) or the runtime parameters (number of
MPI parameters or OpenMP threads etc.) can be also added if necessary.
- `TEST_TYPES` : This file allows you to create a new test type, i.e. to specify for which words
should be grepped and what field should be used in the numerical comparison.
- `TEST_FILES.toml` : This file exists in each test directory and contains a list of file-reference
pairs. Each pair represents a CP2K input file and a search pattern (defined in `TEST_TYPES`) which
is used to compare the found value with a reference. An input file may be used several times with
a different kind of test. Each input is only run once but different quantities of interest are
compared. You can add your file name here. Adding a comment about what it tests might help later
debugging problems if a regtest fails.
- `TEST_DIRS` lists the regression-test directories. A line can also contain conditions on the CP2K
feature flags or on the number of MPI ranks, so that a directory is run only when it is
applicable.
- `UNIT_TESTS` lists standalone unit-test executables run by the driver.
- `TEST_FILES.toml` exists in each regression-test directory. Its entries associate a CP2K input
file with zero or more matcher specifications. An input is executed only once even when several
quantities are checked.
- `matchers.py` implements the available matchers. A matcher specification normally names the
matcher and gives its reference value and tolerance; it may also select a generated output file.
To add a regression test, place a focused input in an appropriate directory (or create a new one),
record the quantities to be checked in `TEST_FILES.toml`, and ensure that the directory is listed in
`TEST_DIRS`. Adding a short comment about what a test covers can make later failures much easier to
diagnose.
```{note}
Only the test directories listed in `TEST_DIRS` are actually considered for testing. If you do not add your new test directory, bugs introduced by you or others will not be found. So, please double-check whether you have actually added your test directory.
Only directories listed in `TEST_DIRS` are considered by the driver. If a new test directory is not
listed there, the test will never be run and cannot detect later regressions.
```
When changing a reference value or tolerance, explain why in the corresponding pull request.
Tolerances should allow expected numerical variation without hiding meaningful regressions.
## Run with sbatch
What you need:
### What you need
- `sbatch` template script
- a CP2K source tree with a built CP2K
- an `sbatch` script or an existing Slurm allocation;
- a CP2K source tree containing `tests/do_regtest.py`, `tests/`, and `data/`; and
- a compatible binary directory containing the selected CP2K executables.
## Instructions
### Instructions
The way the regtest script works is that it goes through all the directories (for example
`tests/QS/regtest-admm-1/`) and launches all tests in that directory. After each directories tests
are started it checks whether the number of maximum tasks is reached, if not it also spawns the
tests from the next directory. If the maximum number of tasks to run has been reached it waits until
enough of them have finished to spawn tests from the next directory. Since the tests are usually
rather short this procedure seldomly causes oversubscription.
The driver runs the tests directory by directory and limits concurrent work according to
`--maxtasks`. Within a Slurm allocation, each individual MPI test is normally launched through
`srun`; if the requested resources are not immediately free, Slurm waits until they become
available. Consequently, a larger allocation can execute more test batches concurrently.
Also, `srun` should simply wait until nodes are free should there be no more free nodes available
within the given allocation. Hence, the more nodes (or total number of tasks) you allocate for the
`sbatch` the more tests can run in parallel. But we have to make sure `do_regtest` knows about that
number by setting `-maxtasks ${SLURM_NTASKS}`, `SLURM_NTASKS` is automatically set by `sbatch` to
the number of tasks you specified either when running `sbatch` or in the preamble of the `sbatch`
script.
`SLURM_NTASKS` is the total number of MPI tasks in the allocation. It should **not** normally be
passed to `--mpiranks`, because that would make every individual test use the entire allocation.
Instead, choose a modest number of ranks for one test, for example two, and use the allocation size
to set `--maxtasks`. When OpenMP threads are used, the CPU-task budget is usually
`SLURM_NTASKS * SLURM_CPUS_PER_TASK`.
Append the following to your `sbatch` template and at least adapt the value for `CP2K_BASE_DIR` and
possibly also the `CP2K_TEST_DIR`:
Append the following to an `sbatch` script and adapt `CP2K_BASE_DIR`, `CP2K_BINARY_DIR`, and the
chosen CP2K variant:
```
```bash
CP2K_BASE_DIR="/PATH/TO/YOUR/CP2K/SOURCE/TREE"
CP2K_BINARY_DIR="${CP2K_BASE_DIR}/build/bin"
CP2K_TEST_DIR="${SCRATCH}/cp2k_regtesting"
# CP2K_REGTEST_SCRIPT_DIR="" # only set if needed (see below)
CP2K_ARCH="local"
CP2K_VERSION="psmp"
# the following is the default, adjust if you want to run single tests with more than 2 ranks/tasks
# Resources for one individual test. Do not set this to ${SLURM_NTASKS}
# unless deliberately testing one calculation across the whole allocation.
NTASKS_SINGLE_TEST=2
NNODES_SINGLE_TEST=1 # otherwise srun will distribute the 2 tasks over 2 nodes
SRUN_CMD="srun --cpu-bind=verbose,cores"
# the following should be sufficiently generic:
OMP_NUM_THREADS="${SLURM_CPUS_PER_TASK:-1}"
MAXTASKS="$((SLURM_NTASKS * OMP_NUM_THREADS))"
mkdir -p "${CP2K_TEST_DIR}"
cd "${CP2K_TEST_DIR}"
cd "${CP2K_BASE_DIR}"
cp2k_rel_dir=$(realpath --relative-to="${CP2K_TEST_DIR}" "${CP2K_BASE_DIR}")
# srun does not like `-np`, override the complete command instead:
export cp2k_run_prefix="${SRUN_CMD} -N ${NNODES_SINGLE_TEST} -n ${NTASKS_SINGLE_TEST}"
./tests/do_regtest.py \
--mpiranks "${NTASKS_SINGLE_TEST}" \
--ompthreads "${OMP_NUM_THREADS}" \
--maxtasks "${MAXTASKS}" \
--mpiexec "srun --nodes=1 --ntasks={N} --cpus-per-task=${OMP_NUM_THREADS} --cpu-bind=cores" \
--workbasedir "${CP2K_TEST_DIR}" \
"${CP2K_BINARY_DIR}" "${CP2K_VERSION}" \
|& tee "${CP2K_TEST_DIR}/${CP2K_VERSION}.log"
"${CP2K_REGEST_SCRIPT_DIR:-${CP2K_BASE_DIR}/tools/regtesting}/do_regtest" \
-arch "${CP2K_ARCH}" \
-version "${CP2K_VERSION}" \
-nobuild \
-mpiranks ${NTASKS_SINGLE_TEST} \
-ompthreads ${OMP_NUM_THREADS} \
-maxtasks ${SLURM_NTASKS} \
-cp2kdir "${cp2k_rel_dir}" \
|& tee "${CP2K_TEST_DIR}/${CP2K_ARCH}.${CP2K_VERSION}.log"
# the above will output both to the slurm-*.out as well as a log file,
# if you want only the log file replace the `|& tee` with a `>&`.
# More options:
# -farming ... enable farming mode, see below
# -retest ... only do tests which failed in a previous run
# To write only to the log file rather than both the Slurm output and the log,
# replace '|& tee ...' with '>& ...'.
```
A complete `sbatch` script to run the regtests on CSCS Alps (Eiger) could look as follows:
The `--nodes=1` setting keeps each two-rank test on a single node. To test communication across
nodes, choose a suitable larger value of `NTASKS_SINGLE_TEST` and provide a site-specific `srun`
template, for example with `--nodes`, `--ntasks-per-node`, and the required MPI plugin or placement
options.
```
A complete generic batch-script template is:
```bash
#!/bin/bash -l
#SBATCH --time=01:00:00
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=128
#SBATCH --ntasks-per-node=64
#SBATCH --cpus-per-task=2
#SBATCH --ntasks-per-core=1
# More SBATCH options:
# If you need 512GB memory nodes (otherwise only 256GB guaranteed):
# #SBATCH --mem=497G
# To run on the debug queue (max 10 nodes, 30 min):
# #SBATCH--partition=debug
set -o errexit
set -o nounset
set -o pipefail
export MPICH_OFI_STARTUP_CONNECT=1
export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK}
export OMP_NUM_THREADS="${SLURM_CPUS_PER_TASK}"
export OMP_PROC_BIND=close
export OMP_PLACES=cores
source "${MODULESHOME}/init/bash"
# Load the compiler, MPI implementation, and libraries used to build CP2K here.
# module load ...
module load cpeGNU
module load \
cray-fftw \
ELPA/2020.11.001 \
libxsmm/1.16.1 \
libxc/5.1.3 \
Libint-CP2K/2.6.0 \
gcc/10.2.0
# Let the user see the currently loaded modules in the slurm log for completeness:
module list
CP2K_BASE_DIR="/users/timuel/work/cp2k"
CP2K_BASE_DIR="/PATH/TO/YOUR/CP2K/SOURCE/TREE"
CP2K_BINARY_DIR="${CP2K_BASE_DIR}/build/bin"
CP2K_TEST_DIR="${SCRATCH}/cp2k_regtesting"
CP2K_ARCH="Eiger-gfortran"
CP2K_VERSION="psmp"
NTASKS_SINGLE_TEST=2
NNODES_SINGLE_TEST=1
SRUN_CMD="srun --cpu-bind=verbose,cores"
# to run tests across nodes (to check for communication effects), use:
# NNODES_SINGLE_TEST=4
# SRUN_CMD="srun --cpu-bind=verbose,cores --ntasks-per-node 2"
# the following should be sufficiently generic:
MAXTASKS="$((SLURM_NTASKS * SLURM_CPUS_PER_TASK))"
mkdir -p "${CP2K_TEST_DIR}"
cd "${CP2K_TEST_DIR}"
cd "${CP2K_BASE_DIR}"
cp2k_rel_dir=$(realpath --relative-to="${CP2K_TEST_DIR}" "${CP2K_BASE_DIR}")
# srun does not like `-np`, override the complete command instead:
export cp2k_run_prefix="${SRUN_CMD} -N ${NNODES_SINGLE_TEST} -n ${NTASKS_SINGLE_TEST}"
"${CP2K_REGEST_SCRIPT_DIR:-${CP2K_BASE_DIR}/tools/regtesting}/do_regtest" \
-arch "${CP2K_ARCH}" \
-version "${CP2K_VERSION}" \
-nobuild \
-mpiranks ${NTASKS_SINGLE_TEST} \
-ompthreads ${OMP_NUM_THREADS} \
-maxtasks ${SLURM_NTASKS} \
-cp2kdir "${cp2k_rel_dir}" \
|& tee "${CP2K_TEST_DIR}/${CP2K_ARCH}.${CP2K_VERSION}.log"
./tests/do_regtest.py \
--mpiranks "${NTASKS_SINGLE_TEST}" \
--ompthreads "${OMP_NUM_THREADS}" \
--maxtasks "${MAXTASKS}" \
--mpiexec "srun --nodes=1 --ntasks={N} --cpus-per-task=${OMP_NUM_THREADS} --cpu-bind=cores" \
--workbasedir "${CP2K_TEST_DIR}" \
"${CP2K_BINARY_DIR}" "${CP2K_VERSION}" \
|& tee "${CP2K_TEST_DIR}/${CP2K_VERSION}.log"
```
## Minimal directory setup
If you want to test a precompiled executable there is a minimal directory layout you have to
reproduce to run the regtest:
The regression driver must be run from a CP2K source tree because it obtains `tests/`, `data/`, and
its own implementation from that tree. A separately built or precompiled CP2K installation can still
be tested by passing its binary directory as `<binary-dir>`.
- `cp2k-prebuilt/exe/prebuilt/*.psmp` … directory with all the executables
- `cp2k-prebuilt/tests` … directory containing the tests (can NOT be a symlink)
- `cp2k-prebuilt/data` … containing CP2Ks data
For input-based regression tests, the minimum practical layout is therefore:
An example if your HPC center uses EasyBuild to provide the CP2K package:
```text
cp2k-source
├── data/
└── tests/
├── do_regtest.py
├── TEST_DIRS
├── TEST_FILES.toml directories ...
└── matchers.py
```
cp2k-prebuilt
├── data -> /apps/eiger/UES/jenkins/1.4.0/software/CP2K/8.1-cpeGNU-21.04/data
├── exe
│   └── prebuilt -> /apps/eiger/UES/jenkins/1.4.0/software/CP2K/8.1-cpeGNU-21.04/bin
└── tests
└── bin/
└── cp2k.psmp
```
and then update the variables as follows:
Run the test driver from `cp2k-source` and point it to the prebuilt binary directory:
```
CP2K_BASE_DIR="/PATH/TO/THE/MINIMAL/DIR/cp2k-prebuilt"
CP2K_TEST_DIR="${SCRATCH}/cp2k_regtesting"
CP2K_REGTEST_SCRIPT_DIR="/PATH/TO/A/FULL/CP2K/DIR/tools/regtesting"
```bash
cd /PATH/TO/cp2k-source
CP2K_ARCH="prebuilt"
CP2K_VERSION="psmp"
./tests/do_regtest.py \
--skip_unittests \
--workbasedir "${SCRATCH}/cp2k_regtesting" \
/PATH/TO/cp2k-prebuilt/bin psmp
```
```{note}
if the `tools/regtesting` is not in that minimal directory tree as shown above you may get an error about the `timings.py` not found and there will be no timings. If you need those you should link/copy the regtesting scripts into `tools/regtest` of that minimal directory tree, at which you point you can leave the `CP2K_REGTEST_SCRIPT_DIR` variable undefined again.
```
Use `--skip_unittests` only when the precompiled installation does not provide the corresponding
`*_unittest.psmp` executables. When a compatible data directory is not available in the source tree,
add `--cp2kdatadir /PATH/TO/data` (or set `CP2K_DATA_DIR`) explicitly.

View file

@ -1,19 +0,0 @@
# FPGA
```{warning}
Support for FPGAs has not yet been ported to CMake.
```
- Use `-D__PW_FPGA` to enable FPGA support for PW (fft) calculations. Currently tested only for
Intel Stratix 10 and Arria 10 GX1150 FPGAs.
- Supports single precision and double precision fft calculations with the use of dedicated APIs.
- Double precision is the default API chosen when set using the `-D__PW_FPGA` flag.
- Single precision can be set using an additional `-D__PW_FPGA_SP` flag along with the `-D__PW_FPGA`
flag.
- Kernel code must be synthesized separately and copied to a specific location.
- See <https://github.com/pc2/fft3d-fpga/> for the kernel code and instructions for synthesis.
- Read `src/pw/fpga/README.md` for information on the specific location to copy the binaries to.
- Currently supported FFT3d sizes - 16^3, 32^3, 64^3.
- Include aocl compile flags and `-D__PW_FPGA -D__PW_FPGA_SP` to `CFLAGS`, aocl linker flags to
`LDFLAGS` and aocl libs to `LIBS`.
- When building FPGA and OFFLOAD together then `-D__NO_OFFLOAD_PW` must be used.

View file

@ -8,5 +8,4 @@ maxdepth: 2
cuda
hip
opencl
fpga
```

View file

@ -1,51 +1,72 @@
# OpenCL
OpenCL devices are currently supported for DBCSR and DBM/DBT, and can cover GPUs and other devices.
Kernels can be automatically tuned.
CP2K can use OpenCL devices through the DBCSR and DBM/DBT offload paths. This supports GPUs and
other devices that provide a suitable OpenCL implementation. The OpenCL backend relies on
[LIBXS](../libraries.md#libxs-improved-performance-for-matrix-multiplication) and
[LIBXSTREAM](../libraries.md#libxstream-opencl-offload-runtime); LIBXSMM may additionally be used
through LIBXS.
Note: the OpenCL backend uses some functionality from LIBXSMM (dependency). CP2K's offload-library
serving DBM/DBT and other libraries depends on DBCSR's OpenCL backend.
OpenCL acceleration does not provide the CUDA/HIP GRID or PW GPU backends. It is therefore most
useful for the DBCSR and DBM workloads supported by the OpenCL backend.
## Installing OpenCL and preparing the runtime environment
- Installing an OpenCL runtime depends on the operating system and the device vendor. Debian for
instance brings two packages called `opencl-headers` and `ocl-icd-opencl-dev` which can be present
in addition to a vendor-specific installation. The OpenCL header files are only necessary if
CP2K/DBCSR is compiled from source. Please note, some implementations ship with outdated OpenCL
headers which can prevent using latest features (if an application discovers such features only at
compile-time). When building from source, for instance `libOpenCL.so` is sufficient at link-time
(ICD loader). However, an Installable Client Driver (ICD) is finally necessary at runtime.
- NVIDIA CUDA, AMD HIP, and Intel OneAPI are fully equipped with an OpenCL runtime (if
`opencl-headers` package is not installed, CPATH can be needed to point into the former
installation, similarly `LIBRARY_PATH` for finding `libOpenCL.so` at link-time). Installing a
minimal or stand-alone OpenCL is also possible, e.g., following the instructions for Debian (or
Ubuntu) as given for every [release](https://github.com/intel/compute-runtime/releases) of the
[Intel Compute Runtime](https://github.com/intel/compute-runtime).
- The environment variable `ACC_OPENCL_VERBOSE` prints information at runtime of CP2K about kernels
generated (`ACC_OPENCL_VERBOSE=2`) or executed (`ACC_OPENCL_VERBOSE=3`) which can be used to check
an installation.
An OpenCL build needs both development files at configuration time and an actual device driver at
runtime:
- CMake must be able to find OpenCL headers and the OpenCL loader library. On Debian and Ubuntu, the
distribution packages are commonly named `opencl-headers` and `ocl-icd-opencl-dev`; package names
vary on other systems.
- The loader library alone is not enough to run CP2K. The system also needs an Installable Client
Driver (ICD) provided by the device vendor or another OpenCL implementation. Vendor SDKs and
runtimes, such as NVIDIA CUDA, AMD ROCm, or Intel's compute runtime, may provide the necessary
components, but their availability and installation layout are platform dependent.
- For a manually managed installation in a non-standard prefix, make OpenCL, LIBXS, and LIBXSTREAM
discoverable by CMake. `CMAKE_PREFIX_PATH` is usually the most convenient mechanism; see
[](../../getting-started/build-from-source.md) for the general CMake workflow.
- Set `ACC_OPENCL_VERBOSE=2` to print information about generated kernels, or `ACC_OPENCL_VERBOSE=3`
to print information about executed kernels. These settings are useful for checking that CP2K
reaches the intended OpenCL runtime and device.
## Building CP2K with OpenCL-based DBCSR
- CP2K's toolchain supports `--enable-opencl` to select DBCSR's OpenCL backend. This can be combined
with `--enable-cuda` (`--gpu-ver` is then imposed) to use a GPU for CP2K's GRID and PW components
(no OpenCL support yet) with DBM's CUDA implementation to be preferred.
- For manually writing an ARCH-file, add `-D__OPENCL` and `-D__DBCSR_ACC` to `CFLAGS` and add
`-lOpenCL` to the `LIBS` variable, i.e., `OFFLOAD_CC` and `OFFLOAD_FLAGS` can duplicate `CC` and
`CFLAGS` (no special offload compiler needed). Please also set `OFFLOAD_TARGET = opencl` to enable
the OpenCL backend in DBCSR. For OpenCL, it is not necessary to specify a GPU version (e.g.,
`GPUVER = V100` would map/limit to `exts/dbcsr/src/acc/opencl/smm/params/tune_multiply_V100.csv`).
In fact, `GPUVER` limits tuned parameters to the specified GPU, whereas by default all tuned
parameters are embedded (`exts/dbcsr/src/acc/opencl/smm/params/*.csv`) and applied at runtime. If
auto-tuned parameters are not available for DBCSR, well-chosen defaults will be used to populate
kernels at runtime.
- Auto-tuned parameters are embedded into the binary, i.e., CP2K does not rely on a hard-coded
location. Setting `OPENCL_LIBSMM_SMM_PARAMS=/path/to/csv-file` environment variable can supply
parameters for an already built application, or `OPENCL_LIBSMM_SMM_PARAMS=0` can disable using
tuned parameters. Refer to <https://cp2k.github.io/dbcsr/> on how to tune kernels (parameters).
Configure CP2K with CMake as usual and add:
## Building CP2K with OpenCL-based DBM library
```bash
cmake -S . -B build -GNinja \
-DCP2K_USE_ACCEL=OPENCL \
-DCP2K_USE_LIBXS=ON \
-DCMAKE_PREFIX_PATH=/path/to/dependencies
cmake --build build --parallel
```
- Pass `-DCP2K_USE_ACCEL=OPENCL` to CMake in addition to following above instructions for "Building
CP2K with OpenCL-based DBCSR". An additional Makefile rule can be necessary to transform OpenCL
code into a ressource header file.
`CP2K_USE_LIBXS=ON` is required for the OpenCL backend. CMake also requires OpenCL and LIBXSTREAM,
and the DBCSR installation selected by CMake must support the requested OpenCL configuration. The
CMake summary identifies the discovered OpenCL and LIBXSTREAM dependencies. No GPU architecture
needs to be specified for an OpenCL build.
CMake supplies the required compile definitions, include paths, and link dependencies. In
particular, do not duplicate these settings through manually added compiler or linker flags.
The DBCSR OpenCL kernels use tuned small-matrix-multiplication parameters where available. The
available parameter sets are embedded into the application, so the executable does not depend on a
fixed external parameter-file location. If no suitable tuned parameters are available, DBCSR uses
fallback defaults to construct kernels at runtime.
Set `OPENCL_LIBSMM_SMM_PARAMS=/path/to/csv-file` to override the built-in parameters for an existing
application, or set `OPENCL_LIBSMM_SMM_PARAMS=0` to disable them. See the
[DBCSR documentation](https://cp2k.github.io/dbcsr/) for details on kernel-parameter tuning.
## Building CP2K with the OpenCL-based DBM library
`-DCP2K_USE_ACCEL=OPENCL` also enables the OpenCL-capable DBM build path. The `CP2K_ENABLE_DBM_GPU`
option is enabled by default when an accelerator backend is selected; set
`-DCP2K_ENABLE_DBM_GPU=OFF` to build without DBM offload while retaining the rest of the selected
configuration.
During the build, CMake generates the header that embeds the DBM OpenCL kernel source and adds the
required target dependencies automatically. No additional build rule or generated source file has to
be maintained by the user.
The CUDA/HIP-specific GRID and PW acceleration options do not enable corresponding OpenCL backends.
To disable DBCSR acceleration explicitly, configure with `-DCP2K_DBCSR_USE_CPU_ONLY=ON`.

View file

@ -967,10 +967,11 @@ CONTAINS
! filter cpuids by vlen to show more relevant information
IF (m_cpuid_vlen(cpuid_static) < m_cpuid_vlen(cpuid)) THEN
! base/machine_cpuid.c relies on the (same) target flags as the Fortran code
CALL cp_hint(__LOCATION__, "The compiler target flags ("// &
TRIM(m_cpuid_name(cpuid_static))//") used to build this binary cannot exploit "// &
"all extensions of this CPU model ("//TRIM(m_cpuid_name(cpuid))//"). "// &
"Consider compiler target flags as part of FCFLAGS and CFLAGS (ARCH file).")
CALL cp_hint(__LOCATION__, "The compiler target ("// &
TRIM(m_cpuid_name(cpuid_static))//") used to build this binary does not "// &
"enable all instruction-set extensions available on this CPU ("// &
TRIM(m_cpuid_name(cpuid))//"). Consider reconfiguring and rebuilding CP2K "// &
"for this target system to enable them.")
END IF
WRITE (UNIT=output_unit, FMT="()")

View file

@ -1216,6 +1216,4 @@ MODULE openpmd_api
END SUBROUTINE openpmd_mesh_set_unit_dimension
#endif
! In two lines of this file, `make pretty` applies a wrong indentation and keeps it for the rest of this file.
! Hence, we end up a bit further to the right than we should.
END MODULE openpmd_api

View file

@ -1 +0,0 @@
cp2k.c

View file

@ -1,27 +0,0 @@
# CP2K Python Bindings
## Installation
There is a target `py-cython-bindings` in the global `Makefile` to build the Python bindings. The
shared object can be found in: `<CP2K_SOURCE_DIR>/lib/<ARCH>/<VERSION>/python`
Only the Python headers and a NumPy installation are required.
## Development
To regenerate the C file from the `cp2k.pyx`, `Cython` is required and should be called as follows:
```sh
cd <CP2K_SOURCE_DIR>/src/start/python
cython cp2k.pyx
```
Unittests can be found in the `test/` directory. They must be run in separate Python interpreter
instances due to side effects in the library.
## Known Issues
- If libcp2k is built with MPI support, you may get an MPI initialization error depending on your
MPI implementation/configuration. In that case MPI must be initialized first by using Mpi4py and
the Fortran MPI communicator handler must be passed down the CP2K via the respective `...comm`
functions. The reason for this is documented here: <https://github.com/jhedev/mpi_python>

View file

@ -1,161 +0,0 @@
# cython: language_level=2
# vim: set ts=4 sw=4 tw=0 :
import numpy as np
from cpython.mem cimport PyMem_Malloc, PyMem_Free
cdef extern from "../libcp2k.h":
ctypedef int force_env_t
void cp2k_get_version(char* version_str, int str_length)
void cp2k_init()
void cp2k_init_without_mpi()
void cp2k_finalize()
void cp2k_finalize_without_mpi()
void cp2k_create_force_env(force_env_t* new_force_env, const char* input_file_path, const char* output_file_path)
void cp2k_create_force_env_comm(force_env_t* new_force_env, const char* input_file_path, const char* output_file_path, int mpi_comm)
void cp2k_destroy_force_env(force_env_t force_env)
void cp2k_set_positions(force_env_t force_env, const double* new_pos, int n_el)
void cp2k_set_velocities(force_env_t force_env, const double* new_vel, int n_el)
void cp2k_get_result(force_env_t force_env, const char* description, double* result, int n_el)
void cp2k_get_natom(force_env_t force_env, int* natom)
void cp2k_get_nparticle(force_env_t force_env, int* nparticle)
void cp2k_get_positions(force_env_t force_env, double* pos, int n_el)
void cp2k_get_forces(force_env_t force_env, double* force, int n_el)
void cp2k_get_potential_energy(force_env_t force_env, double* e_pot)
void cp2k_calc_energy_force(force_env_t force_env)
void cp2k_calc_energy(force_env_t force_env)
void cp2k_run_input(const char* input_file_path, const char* output_file_path)
void cp2k_run_input_comm(const char* input_file_path, const char* output_file_path, int mpi_comm)
def get_version_string():
n = 255 * sizeof(char)
data = <char *>PyMem_Malloc(n)
if not data:
raise MemoryError()
versionstr = ''
try:
cp2k_get_version(data, n)
versionstr = data.decode('UTF-8')
finally:
PyMem_Free(data)
return versionstr
def init(manage_mpi = True):
if manage_mpi:
cp2k_init()
else:
cp2k_init_without_mpi()
def finalize(manage_mpi = True):
if manage_mpi:
cp2k_finalize()
else:
cp2k_finalize_without_mpi()
def run_input(input_file_path, output_file_path = None, mpi_comm = None):
input_file_path = input_file_path.encode('UTF-8')
if output_file_path is None:
output_file_path = u'__STD_OUT__'.encode('UTF-8')
else:
output_file_path = output_file_path.encode('UTF-8')
if mpi_comm:
cp2k_run_input_comm(input_file_path, output_file_path, mpi_comm)
else:
cp2k_run_input(input_file_path, output_file_path)
def create_force_env(input_file_path, output_file_path, mpi_comm = None):
cdef force_env_t fenv
if mpi_comm:
cp2k_create_force_env_comm(&fenv, input_file_path, output_file_path, mpi_comm)
else:
cp2k_create_force_env(&fenv, input_file_path, output_file_path)
cdef class ForceEnvironment(object):
cdef force_env_t _force_env
def __init__(self, input_file_path not None, output_file_path = None, mpi_comm = None):
input_file_path = input_file_path.encode('UTF-8')
if output_file_path is None:
output_file_path = u'__STD_OUT__'.encode('UTF-8')
else:
output_file_path = output_file_path.encode('UTF-8')
if mpi_comm:
cp2k_create_force_env_comm(&self._force_env, input_file_path, output_file_path, mpi_comm)
else:
cp2k_create_force_env(&self._force_env, input_file_path, output_file_path)
def destroy(self):
cp2k_destroy_force_env(self._force_env)
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, traceback):
self.destroy()
property positions:
def __get__(self):
positions = np.zeros([3*self.nparticle], dtype=np.double)
cdef double [::1] positions_view = positions
cp2k_get_positions(self._force_env, &positions_view[0], positions_view.shape[0])
return positions
def __set__(self, double[::1] positions not None):
if positions.shape[0] != 3*self.nparticle:
raise ValueError('the positions array must have exactly {} (3*nparticle) elements'.format(3*self.nparticle))
cp2k_set_positions(self._force_env, &positions[0], positions.shape[0])
def set_velocities(self, double[::1] velocities not None):
if velocities.shape[0] != 3*self.nparticle:
raise ValueError('the velocities array must have exactly {} (3*nparticle) elements'.format(3*self.nparticle))
cp2k_set_velocities(self._force_env, &velocities[0], velocities.shape[0])
property natom:
def __get__(self):
cdef int natom
cp2k_get_natom(self._force_env, &natom)
return natom
property nparticle:
def __get__(self):
cdef int nparticle
cp2k_get_nparticle(self._force_env, &nparticle)
return nparticle
def get_result(self, description):
results = np.zeros([3*self.nparticle], dtype=np.double)
cdef double [::1] results_view = results
cp2k_get_result(self._force_env, description.encode('UTF-8'), &results_view[0], results_view.shape[0])
return results
property forces:
def __get__(self):
forces = np.zeros([3*self.nparticle], dtype=np.double)
cdef double [::1] forces_view = forces
cp2k_get_forces(self._force_env, &forces_view[0], forces_view.shape[0])
return forces
def calc_energy_force(self):
cp2k_calc_energy_force(self._force_env)
def calc_energy(self):
cp2k_calc_energy(self._force_env)
property potential_energy:
def __get__(self):
cdef double e_pot
cp2k_get_potential_energy(self._force_env, &e_pot)
return e_pot

View file

@ -1,84 +0,0 @@
#!/usr/bin/env python3
# vim: set ts=4 sw=4 tw=0 :
from tempfile import NamedTemporaryFile
import numpy as np
import cp2k
TEST_FILE_CONTENT = """
&FORCE_EVAL
METHOD Quickstep
&DFT
BASIS_SET_FILE_NAME BASIS_SET
POTENTIAL_FILE_NAME POTENTIAL
LSD
&MGRID
CUTOFF 140
&END MGRID
&QS
EPS_DEFAULT 1.0E-8
&END QS
&SCF
EPS_DIIS 0.1
EPS_SCF 1.0E-4
IGNORE_CONVERGENCE_FAILURE
MAX_DIIS 4
MAX_SCF 3
SCF_GUESS atomic
&PRINT
&RESTART OFF
&END
&END
&END SCF
&XC
&XC_FUNCTIONAL Pade
&END XC_FUNCTIONAL
&END XC
&END DFT
&SUBSYS
&CELL
ABC 8.0 4.0 4.0
&END CELL
&COORD
H 0.000000 0.000000 0.000000
H 1.000000 0.000000 0.000000
&END COORD
&KIND H
BASIS_SET DZV-GTH-PADE
POTENTIAL GTH-PADE-q1
&END KIND
&END SUBSYS
&END FORCE_EVAL
&GLOBAL
PROJECT libcp2k_unittest_H2
&END GLOBAL
"""
def sample_run(input_file_path):
with cp2k.ForceEnvironment(input_file_path, "/dev/null") as fenv:
print("potential energy: {:e}".format(fenv.potential_energy))
print("calculating energy..")
fenv.calc_energy()
print(".. done!")
print("potential energy: {:e}".format(fenv.potential_energy))
print("positions:")
pos = fenv.positions
print(pos)
print("zeroify positions..")
fenv.positions = np.zeros(pos.shape)
print("positions:")
print(fenv.positions)
if __name__ == "__main__":
cp2k.init()
tempfile = NamedTemporaryFile(mode="w+")
tempfile.write(TEST_FILE_CONTENT)
tempfile.flush()
sample_run(tempfile.name)
cp2k.finalize()

View file

@ -1,28 +0,0 @@
#!/usr/bin/env python3
# vim: set ts=4 sw=4 tw=0 :
from distutils.core import setup
from distutils.extension import Extension
import numpy
try:
from Cython.Build import cythonize
USE_CYTHON = True
EXT = "pyx"
except ImportError:
USE_CYTHON = False
EXT = "c"
cythonize = lambda ext: ext
extensions = [
Extension(
"cp2k",
["cp2k.{}".format(EXT)],
include_dirs=[numpy.get_include()],
libraries=["cp2k"],
),
]
setup(name="cp2k", ext_modules=cythonize(extensions))

View file

@ -1,103 +0,0 @@
#!/usr/bin/env python3
# vim: set ts=4 sw=4 tw=0 :
import unittest
from tempfile import NamedTemporaryFile
import numpy as np
import cp2k
TEST_FILE_CONTENT = """
&FORCE_EVAL
METHOD Quickstep
&DFT
BASIS_SET_FILE_NAME BASIS_SET
POTENTIAL_FILE_NAME POTENTIAL
LSD
&MGRID
CUTOFF 140
&END MGRID
&QS
EPS_DEFAULT 1.0E-8
&END QS
&SCF
EPS_DIIS 0.1
EPS_SCF 1.0E-4
IGNORE_CONVERGENCE_FAILURE
MAX_DIIS 4
MAX_SCF 3
SCF_GUESS atomic
&PRINT
&RESTART OFF
&END
&END
&END SCF
&XC
&XC_FUNCTIONAL Pade
&END XC_FUNCTIONAL
&END XC
&END DFT
&SUBSYS
&CELL
ABC 8.0 4.0 4.0
&END CELL
&COORD
H 0.000000 0.000000 0.000000
H 1.000000 0.000000 0.000000
&END COORD
&KIND H
BASIS_SET DZV-GTH-PADE
POTENTIAL GTH-PADE-q1
&END KIND
&END SUBSYS
&END FORCE_EVAL
&GLOBAL
PROJECT libcp2k_unittest_H2
&END GLOBAL
"""
def setUpModule():
cp2k.init()
def tearDownModule():
cp2k.finalize()
class TestForceEnvironment(unittest.TestCase):
def setUp(self):
self._input = NamedTemporaryFile(mode="w+")
self._input.write(TEST_FILE_CONTENT)
self._input.flush()
self._output = NamedTemporaryFile()
self._fenv = cp2k.ForceEnvironment(self._input.name, self._output.name)
def tearDown(self):
self._fenv.destroy()
self._input.close()
self._output.close()
def test_numbers(self):
self.assertEqual(self._fenv.natom, 2)
self.assertEqual(self._fenv.nparticle, 2)
def test_calc_energy(self):
self._fenv.calc_energy()
self.assertLess(self._fenv.potential_energy, 0)
def test_calc_forces(self):
self._fenv.calc_energy_force()
self.assertTrue(np.any(abs(self._fenv.forces) > 0.0))
def test_positions(self):
old_pos = self._fenv.positions
zeros = np.zeros(old_pos.shape, dtype=np.double)
self._fenv.positions = zeros
self.assertTrue(np.all(self._fenv.positions == zeros))
self._fenv.positions = old_pos
if __name__ == "__main__":
unittest.main()

View file

@ -1,85 +0,0 @@
#!/usr/bin/env python3
import unittest
from tempfile import NamedTemporaryFile
import cp2k
TEST_FILE_CONTENT = """
&FORCE_EVAL
METHOD Quickstep
&DFT
BASIS_SET_FILE_NAME BASIS_SET
POTENTIAL_FILE_NAME POTENTIAL
LSD
&MGRID
CUTOFF 140
&END MGRID
&QS
EPS_DEFAULT 1.0E-8
&END QS
&SCF
EPS_DIIS 0.1
EPS_SCF 1.0E-4
IGNORE_CONVERGENCE_FAILURE
MAX_DIIS 4
MAX_SCF 3
SCF_GUESS atomic
&PRINT
&RESTART OFF
&END
&END
&END SCF
&XC
&XC_FUNCTIONAL Pade
&END XC_FUNCTIONAL
&END XC
&END DFT
&SUBSYS
&CELL
ABC 8.0 4.0 4.0
&END CELL
&COORD
H 0.000000 0.000000 0.000000
H 1.000000 0.000000 0.000000
&END COORD
&KIND H
BASIS_SET DZV-GTH-PADE
POTENTIAL GTH-PADE-q1
&END KIND
&END SUBSYS
&END FORCE_EVAL
&GLOBAL
PROJECT libcp2k_unittest_H2
&END GLOBAL
"""
def setUpModule():
cp2k.init()
def tearDownModule():
cp2k.finalize()
class TestBasic(unittest.TestCase):
def setUp(self):
self._input = NamedTemporaryFile(mode="w+")
self._input.write(TEST_FILE_CONTENT)
self._input.flush()
self._output = NamedTemporaryFile()
def tearDown(self):
self._input.close()
self._output.close()
def test_version_string(self):
self.assertIn("CP2K version", cp2k.get_version_string())
def test_run_input(self):
cp2k.run_input(self._input.name, self._output.name)
if __name__ == "__main__":
unittest.main()