comp-chem/cp2k
1
0
Fork 0
mirror of https://github.com/cp2k/cp2k.git synced 2026-07-28 22:25:32 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

Use mpiexec in favor of mpirun

Browse source 
mpiexec is standardized whereas mpirun is not.
This commit is contained in:
Frederick Stein 2023-02-02 11:20:48 +01:00 • committed by Ole Schütt
parent dc4cb0e83c
commit 41df8e9e1b
6 changed files with 28 additions and 28 deletions
Download patch file Download diff file Expand all files Collapse all files

2
tools/docker/scripts/test_aiida.sh
View file

@ -45,7 +45,7 @@ adduser --disabled-password --gecos "" ubuntu
echo "ubuntu ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
# link mpi executables into path
MPI_INSTALL_DIR=$(dirname "$(command -v mpirun)")
MPI_INSTALL_DIR=$(dirname "$(command -v mpiexec)")
for i in "${MPI_INSTALL_DIR}"/*; do ln -sf "$i" /usr/bin/; done
echo -e "\n========== Installing AiiDA-CP2K plugin =========="

20
tools/plan_mpi_omp/README.md
View file

@ -26,7 +26,7 @@ be better for performance (`16x6`). To easily place the ranks, Intel MPI
is used:
```bash
mpirun -np 16 \
mpiexec -np 16 \
-genv I_MPI_PIN_DOMAIN=auto -genv I_MPI_PIN_ORDER=bunch \
-genv OMP_PLACES=threads -genv OMP_PROC_BIND=SPREAD \
-genv OMP_NUM_THREADS=6 \
@ -49,7 +49,7 @@ elements (`PE`). By default, execution slots are counted in number of physical
cores which yields `--map-by slot:PE=3` for the same system (mentioned above).
```bash
mpirun -np 16 --map-by slot:PE=3 \
mpiexec -np 16 --map-by slot:PE=3 \
-x OMP_PLACES=threads -x OMP_PROC_BIND=SPREAD \
-x OMP_NUM_THREADS=6 \
exe/Linux-x86-64-intelx/cp2k.psmp workload.inp
@ -59,7 +59,7 @@ mpirun -np 16 --map-by slot:PE=3 \
ranks between sockets (see above) appears hard to achieve with OpenMPI
therefore an undersubscribed system may not be recommended. To display and
to log the pinning and thread affinization at the startup of an application,
`mpirun --report-bindings` can be used.
`mpiexec --report-bindings` can be used.
The end of the next section continues with our example and extends execution
to multiple nodes of the above-mentioned system.
@ -105,30 +105,30 @@ filling each node). For the given example, 8 ranks per node with
12 threads per rank is chosen (`8x12`) and MPI-executed:
```bash
mpirun -perhost 8 -host node1,node2,node3,node4,node5,node6,node7,node8 \
mpiexec -perhost 8 -host node1,node2,node3,node4,node5,node6,node7,node8 \
-genv I_MPI_PIN_DOMAIN=auto -genv I_MPI_PIN_ORDER=bunch -genv I_MPI_DEBUG=4 \
-genv OMP_PLACES=threads -genv OMP_PROC_BIND=SPREAD -genv OMP_NUM_THREADS=12 \
exe/Linux-x86-64-intelx/cp2k.psmp workload.inp
```
**NOTE**: For Intel MPI as well as OpenMPI, mpirun's host-list (`mpirun -host`) is setup with unique node-names, and this is the only style that is
**NOTE**: For Intel MPI as well as OpenMPI, mpiexec's host-list (`mpiexec -host`) is setup with unique node-names, and this is the only style that is
explained in this article. There is a competing style where nodes names are
duplicated for the sake of enumerating available ranks (or "execution slots"
in case of OpenMPI), which is not exercised in this article.
For OpenMPI, the quantity (per node) of the previously mentioned "execution
slots" (measured in number of physical cores) are sometimes not known to
OpenMPI (depends on cluster/scheduler setup). For instance, `mpirun` may be
OpenMPI (depends on cluster/scheduler setup). For instance, `mpiexec` may be
complaining about an attempt to use too many execution slots simply because
OpenMPI believes all systems represent a single such slot (instead of 2x24
cores it only "sees" a single core per system). In such case, it is not
recommended to "oversubscribe" the system because rank/thread affinity will
likely be wrong (`mpirun --oversubscribe`). Instead, the list of unique nodes
likely be wrong (`mpiexec --oversubscribe`). Instead, the list of unique nodes
names (`-host`) may be augmented with the number of physical cores on each of
the nodes (e.g., ":48" in our case).
```bash
mpirun -npernode 8 -host
mpiexec -npernode 8 -host
node1:48,node2:48,node3:48,node4:48,node5:48,node6:48,node7:48,node8:48 \
--map-by slot:PE=6 --report-bindings \
-x OMP_PLACES=threads -x OMP_PROC_BIND=SPREAD -x OMP_NUM_THREADS=12 \
@ -136,9 +136,9 @@ node1:48,node2:48,node3:48,node4:48,node5:48,node6:48,node7:48,node8:48 \
```
**NOTE**: It can be still insufficient to augment the nodes with the expected
number of slots (`:48`). If OpenMPI's mpirun is still complaining, it might
number of slots (`:48`). If OpenMPI's mpiexec is still complaining, it might
be caused and solved by the job scheduler. For example, `qsub` (PBS) may be
instructed with `-l select=8:mpiprocs=48` in the above case (`mpirun` in this
instructed with `-l select=8:mpiprocs=48` in the above case (`mpiexec` in this
job can use less than 48 ranks per node).
The plan-script also suggests close-by configurations (lower and higher

8
tools/toolchain/install_cp2k_toolchain.sh
View file

@ -305,17 +305,17 @@ with_cosma="__INSTALL__"
with_libvori="__INSTALL__"
with_libtorch="__DONTUSE__"
# for MPI, we try to detect system MPI variant
if (command -v mpirun > /dev/null 2>&1); then
if (command -v mpiexec > /dev/null 2>&1); then
# check if we are dealing with openmpi, mpich or intelmpi
if (mpirun --version 2>&1 | grep -s -q "HYDRA"); then
if (mpiexec --version 2>&1 | grep -s -q "HYDRA"); then
echo "MPI is detected and it appears to be MPICH"
export MPI_MODE="mpich"
with_mpich="__SYSTEM__"
elif (mpirun --version 2>&1 | grep -s -q "Open MPI"); then
elif (mpiexec --version 2>&1 | grep -s -q "Open MPI"); then
echo "MPI is detected and it appears to be OpenMPI"
export MPI_MODE="openmpi"
with_openmpi="__SYSTEM__"
elif (mpirun --version 2>&1 | grep -s -q "Intel"); then
elif (mpiexec --version 2>&1 | grep -s -q "Intel"); then
echo "MPI is detected and it appears to be Intel MPI"
with_gcc="__DONTUSE__"
with_intel="__SYSTEM__"

4
tools/toolchain/scripts/stage1/install_intelmpi.sh
View file

@ -29,7 +29,7 @@ case "${with_intelmpi}" in
;;
__SYSTEM__)
echo "==================== Finding Intel MPI from system paths ===================="
check_command mpirun "intelmpi" && MPIRUN="$(command -v mpirun)" || exit 1
check_command mpiexec "intelmpi" && MPIRUN="$(command -v mpiexec)" || exit 1
if [ "${with_intel}" != "__DONTUSE__" ]; then
check_command mpiicc "intelmpi" && MPICC="$(command -v mpiicc)" || exit 1
check_command mpiicpc "intelmpi" && MPICXX="$(command -v mpiicpc)" || exit 1
@ -56,7 +56,7 @@ case "${with_intelmpi}" in
check_dir "${pkg_install_dir}/bin"
check_dir "${pkg_install_dir}/lib"
check_dir "${pkg_install_dir}/include"
check_command ${pkg_install_dir}/bin/mpirun "intel" && MPIRUN="${pkg_install_dir}/bin/mpirun" || exit 1
check_command ${pkg_install_dir}/bin/mpiexec "intel" && MPIRUN="${pkg_install_dir}/bin/mpiexec" || exit 1
if [ "${with_intel}" != "__DONTUSE__" ]; then
check_command ${pkg_install_dir}/bin/mpiicc "intel" && MPICC="${pkg_install_dir}/bin/mpiicc" || exit 1
check_command ${pkg_install_dir}/bin/mpiicpc "intel" && MPICXX="${pkg_install_dir}/bin/mpiicpc" || exit 1

10
tools/toolchain/scripts/stage1/install_mpich.sh
View file

@ -68,7 +68,7 @@ case "${with_mpich}" in
check_dir "${pkg_install_dir}/bin"
check_dir "${pkg_install_dir}/lib"
check_dir "${pkg_install_dir}/include"
check_install ${pkg_install_dir}/bin/mpirun "mpich" && MPIRUN="${pkg_install_dir}/bin/mpirun" || exit 1
check_install ${pkg_install_dir}/bin/mpiexec "mpich" && MPIRUN="${pkg_install_dir}/bin/mpiexec" || exit 1
check_install ${pkg_install_dir}/bin/mpicc "mpich" && MPICC="${pkg_install_dir}/bin/mpicc" || exit 1
check_install ${pkg_install_dir}/bin/mpicxx "mpich" && MPICXX="${pkg_install_dir}/bin/mpicxx" || exit 1
check_install ${pkg_install_dir}/bin/mpifort "mpich" && MPIFC="${pkg_install_dir}/bin/mpifort" || exit 1
@ -79,7 +79,7 @@ case "${with_mpich}" in
;;
__SYSTEM__)
echo "==================== Finding MPICH from system paths ===================="
check_command mpirun "mpich" && MPIRUN="$(command -v mpirun)" || exit 1
check_command mpiexec "mpich" && MPIRUN="$(command -v mpiexec)" || exit 1
check_command mpicc "mpich" && MPICC="$(command -v mpicc)" || exit 1
if [ $(command -v mpic++ > /dev/null 2>&1) ]; then
check_command mpic++ "mpich" && MPICXX="$(command -v mpic++)" || exit 1
@ -104,7 +104,7 @@ case "${with_mpich}" in
check_dir "${pkg_install_dir}/bin"
check_dir "${pkg_install_dir}/lib"
check_dir "${pkg_install_dir}/include"
check_command ${pkg_install_dir}/bin/mpirun "mpich" && MPIRUN="${pkg_install_dir}/bin/mpirun" || exit 1
check_command ${pkg_install_dir}/bin/mpiexec "mpich" && MPIRUN="${pkg_install_dir}/bin/mpiexec" || exit 1
check_command ${pkg_install_dir}/bin/mpicc "mpich" && MPICC="${pkg_install_dir}/bin/mpicc" || exit 1
check_command ${pkg_install_dir}/bin/mpicxx "mpich" && MPICXX="${pkg_install_dir}/bin/mpicxx" || exit 1
check_command ${pkg_install_dir}/bin/mpifort "mpich" && MPIFC="${pkg_install_dir}/bin/mpifort" || exit 1
@ -116,9 +116,9 @@ case "${with_mpich}" in
esac
if [ "${with_mpich}" != "__DONTUSE__" ]; then
if [ "${with_mpich}" != "__SYSTEM__" ]; then
mpi_bin="${pkg_install_dir}/bin/mpirun"
mpi_bin="${pkg_install_dir}/bin/mpiexec"
else
mpi_bin="mpirun"
mpi_bin="mpiexec"
fi
MPICH_LIBS="-lmpifort -lmpicxx -lmpi"
cat << EOF > "${BUILDDIR}/setup_mpich"

12
tools/toolchain/scripts/stage1/install_openmpi.sh
View file

@ -76,7 +76,7 @@ case "${with_openmpi}" in
check_dir "${pkg_install_dir}/bin"
check_dir "${pkg_install_dir}/lib"
check_dir "${pkg_install_dir}/include"
check_install ${pkg_install_dir}/bin/mpirun "openmpi" && MPIRUN="${pkg_install_dir}/bin/mpirun" || exit 1
check_install ${pkg_install_dir}/bin/mpiexec "openmpi" && MPIRUN="${pkg_install_dir}/bin/mpiexec" || exit 1
check_install ${pkg_install_dir}/bin/mpicc "openmpi" && MPICC="${pkg_install_dir}/bin/mpicc" || exit 1
check_install ${pkg_install_dir}/bin/mpicxx "openmpi" && MPICXX="${pkg_install_dir}/bin/mpicxx" || exit 1
check_install ${pkg_install_dir}/bin/mpifort "openmpi" && MPIFC="${pkg_install_dir}/bin/mpifort" || exit 1
@ -87,7 +87,7 @@ case "${with_openmpi}" in
;;
__SYSTEM__)
echo "==================== Finding OpenMPI from system paths ===================="
check_command mpirun "openmpi" && MPIRUN="$(command -v mpirun)" || exit 1
check_command mpiexec "openmpi" && MPIRUN="$(command -v mpiexec)" || exit 1
check_command mpicc "openmpi" && MPICC="$(command -v mpicc)" || exit 1
check_command mpic++ "openmpi" && MPICXX="$(command -v mpic++)" || exit 1
check_command mpifort "openmpi" && MPIFC="$(command -v mpifort)" || exit 1
@ -107,7 +107,7 @@ case "${with_openmpi}" in
check_dir "${pkg_install_dir}/bin"
check_dir "${pkg_install_dir}/lib"
check_dir "${pkg_install_dir}/include"
check_command ${pkg_install_dir}/bin/mpirun "openmpi" && MPIRUN="${pkg_install_dir}/bin/mpirun" || exit 1
check_command ${pkg_install_dir}/bin/mpiexec "openmpi" && MPIRUN="${pkg_install_dir}/bin/mpiexec" || exit 1
check_command ${pkg_install_dir}/bin/mpicc "openmpi" && MPICC="${pkg_install_dir}/bin/mpicc" || exit 1
check_command ${pkg_install_dir}/bin/mpic++ "openmpi" && MPICXX="${pkg_install_dir}/bin/mpic++" || exit 1
check_command ${pkg_install_dir}/bin/mpifort "openmpi" && MPIFC="${pkg_install_dir}/bin/mpifort" || exit 1
@ -119,13 +119,13 @@ case "${with_openmpi}" in
esac
if [ "${with_openmpi}" != "__DONTUSE__" ]; then
if [ "${with_openmpi}" != "__SYSTEM__" ]; then
mpi_bin="${pkg_install_dir}/bin/mpirun"
mpi_bin="${pkg_install_dir}/bin/mpiexec"
mpicxx_bin="${pkg_install_dir}/bin/mpicxx"
else
mpi_bin="mpirun"
mpi_bin="mpiexec"
mpicxx_bin="mpicxx"
fi
# check openmpi version as reported by mpirun
# check openmpi version as reported by mpiexec
raw_version=$(${mpi_bin} --version 2>&1 |
grep "(Open MPI)" | awk '{print $4}')
major_version=$(echo ${raw_version} | cut -d '.' -f 1)