mirror of
https://github.com/nwchemgit/nwchem.git
synced 2026-07-21 14:35:21 -04:00
cd7bbc776b
this is the third of three changesets designed to remove obsolete SGI platforms. first, SGITFP, then the 32-bit version, SGI_N32, and now the SGI target, all of which are associated with MIPS/IA64 machines from days gone by. Signed-off-by: Jeff Hammond <jeff.science@gmail.com>
306 lines
12 KiB
TeX
306 lines
12 KiB
TeX
%
|
|
% $Id$
|
|
%
|
|
%A more complete description should be available at
|
|
%\begin{verbatim}
|
|
% http://emsl.pnl.gov:2080/docs/nwchem/nwchem.html
|
|
%\htmladdnormallink{http://www.emsl.pnl.gov:2080/docs/nwchem/nwchem.html}
|
|
%{http://www.emsl.pnl.gov:2080/docs/nwchem/nwchem.html}
|
|
%\end{verbatim}
|
|
|
|
The command required to invoke NWChem is machine dependent, whereas
|
|
most of the NWChem input is machine independent\footnote{Machine
|
|
dependence within the input arises from file names, machine
|
|
specific resources, and differing services provided by the operating system.} .
|
|
|
|
\section{Sequential execution}
|
|
|
|
To run NWChem sequentially on nearly all UNIX-based platforms simply
|
|
use the command \verb+nwchem+ and provide the name of the input file
|
|
as an argument (See section \ref{sec:inputstructure} for more information).
|
|
This does assume that either \verb+nwchem+ is in your path or you have
|
|
set an alias of \verb+nwchem+ to point to the appropriate executable.
|
|
|
|
Output is to standard output, standard error and Fortran unit 6
|
|
(usually the same as standard output). Files are created by default
|
|
in the current directory, though this may be overridden in the input
|
|
(section \ref{sec:dirs}).
|
|
|
|
Generally, one will run a job with the following command:
|
|
|
|
\verb+nwchem input.nw >& input.out &+
|
|
|
|
\section{Parallel execution on UNIX-based parallel machines
|
|
including workstation clusters using TCGMSG}
|
|
\label{sec:procgrp}
|
|
|
|
These platforms require the use of the TCGMSG\footnote{Where required
|
|
TCGMSG is automatically built with NWChem.} \verb+parallel+ command
|
|
and thus also require the definition of a process-group (or procgroup)
|
|
file. The process-group file describes how many processes to start,
|
|
what program to run, which machines to use, which directories to work
|
|
in, and under which userid to run the processes. By convention the
|
|
process-group file has a \verb+.p+ suffix.
|
|
|
|
The process-group file is read to end-of-file. The character \verb+#+
|
|
(hash or pound sign) is used to indicate a comment which continues to
|
|
the next new-line character. Each line describes a cluster of
|
|
processes and consists of the following whitespace separated fields:
|
|
|
|
\begin{verbatim}
|
|
userid hostname nslave executable workdir
|
|
\end{verbatim}
|
|
|
|
\begin{itemize}
|
|
\item \verb+userid+ -- The user-name on the machine that will be executing the
|
|
process.
|
|
|
|
\item \verb+hostname+ -- The hostname of the machine to execute this process.
|
|
If it is the same machine on which parallel was invoked
|
|
the name must match the value returned by the command
|
|
hostname. If a remote machine it must allow remote execution
|
|
from this machine (see man pages for rlogin, rsh).
|
|
|
|
\item \verb+nslave+ -- The total number of copies of this process to be executing
|
|
on the specified machine. Only ``clusters'' of identical processes
|
|
specified in this fashion can use shared memory to communicate.
|
|
If no shared memory is supported on machine \verb+<hostname>+ then
|
|
only the value one (1) is valid.
|
|
|
|
\item \verb+executable+ -- Full path name on the host \verb+<hostname>+ of the image to execute.
|
|
If \verb+<hostname>+ is the local machine then a local path will
|
|
suffice.
|
|
|
|
\item \verb+workdir+ -- Full path name on the host \verb+<hostname>+ of the directory to
|
|
work in. Processes execute a chdir() to this directory before
|
|
returning from pbegin(). If specified as a ``.'' then remote
|
|
processes will use the login directory on that machine and local
|
|
processes (relative to where parallel was invoked) will use
|
|
the current directory of parallel.
|
|
\end{itemize}
|
|
|
|
For example, if your file \verb+"nwchem.p"+ contained the following
|
|
\begin{verbatim}
|
|
d3g681 pc 4 /msrc/apps/bin/nwchem /scr22/rjh
|
|
\end{verbatim}
|
|
then 4 processes running NWChem would be started on the machine
|
|
\verb+pc+ running as user \verb+d3g681+ in directory \verb+"/scr22/rjh"+.
|
|
To actually run this simply type:
|
|
\begin{verbatim}
|
|
parallel nwchem big_molecule.nw
|
|
\end{verbatim}
|
|
|
|
{\em N.B.} : The first process specified (process zero) is the only
|
|
process that
|
|
\begin{itemize}
|
|
\item opens and reads the input file, and
|
|
\item opens and reads/updates the database.
|
|
\end{itemize}
|
|
Thus, if your file systems are physically distributed (e.g., most
|
|
workstation clusters) you must ensure that process zero can correctly
|
|
resolve the paths for the input and database files.
|
|
|
|
{\em N.B.} In releases of NWChem prior to 3.3 additional processes
|
|
had to be created on workstation clusters to support remote access to
|
|
shared memory. This is no longer the case. The TCGMSG process group
|
|
file now just needs to refer to processes running NWChem.
|
|
|
|
\section{Parallel execution on UNIX-based parallel machines
|
|
including workstation clusters using MPI}
|
|
|
|
To run with MPI, \verb+parallel+ should not be used. The way
|
|
we usually run nwchem under MPI are the following
|
|
|
|
\begin{itemize}
|
|
\item using mpirun:
|
|
\begin{verbatim}
|
|
mpirun -np 8 $NWCHEM_TOP/bin/$NWCHEM_TARGET/nwchem input.nw
|
|
\end{verbatim}
|
|
\item If you have all nodes connected via shared memory
|
|
and you have installed the ch\_shmem version of MPICH,
|
|
you can do
|
|
\begin{verbatim}
|
|
$NWCHEM_TOP/bin/$NWCHEM_TARGET/nwchem -np 8 h2o.nw
|
|
\end{verbatim}
|
|
\end{itemize}
|
|
|
|
\section{Parallel execution on MPPs}
|
|
|
|
All of these machines require use of different commands in order to
|
|
gain exclusive access to computational resources.
|
|
|
|
\section{IBM SP}
|
|
|
|
If using POE (IBM's Parallel Operating Environment) interactively,
|
|
simply create the list of nodes to use in the file \verb+"host.list"+ in
|
|
the current directory and invoke NWChem with
|
|
\begin{verbatim}
|
|
nwchem <input_file> -procs <n>
|
|
\end{verbatim}
|
|
where \verb+n+ is the number of processes to use. Process 0 will run
|
|
on the first node in \verb+"host.list"+ and must have access to the
|
|
input and other necessary files. Very significant performance gains
|
|
may be had by setting the following environment variables before
|
|
running NWChem (or setting them using POE command line options).
|
|
\begin{itemize}
|
|
\item \verb+setenv MP_EUILIB us+ --- dedicated user space
|
|
communication over the switch (the default is IP over the switch
|
|
which is much slower).
|
|
\item \verb+setenv MP_CSS_INTERRUPT yes+ --- enable interrupts when a
|
|
message arrives (the default is to poll which significantly slows
|
|
down global array accesses).
|
|
\end{itemize}
|
|
In addition, if the IBM is running PSSP version 3.1, or later
|
|
\begin{itemize}
|
|
\item \verb+setenv MP_MSG_API lapi+, or
|
|
\item \verb+setenv MP_MSG_API mpi,lapi+ (if using both GA and MPI)
|
|
\end{itemize}
|
|
|
|
For batch execution, we recommend use of the \verb+llnw+ command which
|
|
is installed in \verb+/usr/local/bin+ on the EMSL/PNNL IBM SP. If you
|
|
are not running on that system, the \verb+llnw+ script may be found in
|
|
the NWChem distribution directory contrib/loadleveler.
|
|
Interactive help may be obtained with the command \verb+llnw -help+.
|
|
Otherwise, the very simplest job to run NWChem in batch using Load
|
|
Leveller is something like this
|
|
\begin{verbatim}
|
|
#!/bin/csh -x
|
|
# @ job_type = parallel
|
|
# @ class = small
|
|
# @ network.lapi = css0,not_shared,US
|
|
# @ input = /dev/null
|
|
# @ output = <OUTPUT_FILE_NAME>
|
|
# @ error = <ERROUT_FILE_NAME>
|
|
# @ environment = COPY_ALL; MP_PULSE=0; MP_SINGLE_THREAD=yes; MP_WAIT_MODE=yield; restart=no
|
|
# @ min_processors = 7
|
|
# @ max_processors = 7
|
|
# @ cpu_limit = 1:00:00
|
|
# @ wall_clock_limit = 1:00:00
|
|
# @ queue
|
|
#
|
|
|
|
cd /scratch
|
|
|
|
nwchem <INPUT_FILE_NAME>
|
|
\end{verbatim}
|
|
|
|
Substitute \verb+<OUTPUT_FILE_NAME>+, \verb+<ERROUT_FILE_NAME>+ and
|
|
\verb+<INPUT_FILE_NAME>+ with the {\em full} path of the appropriate
|
|
files. Also, if you are using an SP with more than one processor per node,
|
|
you will need to substitute
|
|
|
|
\begin{verbatim}
|
|
# @ network.lapi = css0,shared,US
|
|
# @ node = NNODE
|
|
# @ tasks_per_node = NTASK
|
|
\end{verbatim}
|
|
for the lines
|
|
\begin{verbatim}
|
|
# @ network.lapi = css0,not_shared,US
|
|
# @ min_processors = 7
|
|
# @ max_processors = 7
|
|
\end{verbatim}
|
|
where \verb+NNODE+ is the number of physical nodes to be used and
|
|
\verb+NTASK+ is the
|
|
number of tasks per node.
|
|
|
|
These files and the NWChem executable must be in a file system
|
|
accessible to all processes. Put the above into a file (e.g.,
|
|
\verb+"test.job"+) and submit it with the command
|
|
\begin{verbatim}
|
|
llsubmit test.job
|
|
\end{verbatim}
|
|
It will run a 7 processor, 1 hour job in the queue \verb+small+. It
|
|
should be apparent how to change these values.
|
|
|
|
Note that on many IBM SPs, including that at EMSL, the local scratch
|
|
disks are wiped clean at the beginning of each job and therefore
|
|
persistent files should be stored elsewhere. PIOFS is recommended for
|
|
files larger than a few MB.
|
|
|
|
\section{Cray T3E}
|
|
|
|
\begin{verbatim}
|
|
mpprun -n <npes> $NWCHEM_TOP/bin/$NWCHEM_TARGET/nwchem <input_file>
|
|
\end{verbatim}
|
|
|
|
where \verb+npes+ is the number of processors and \verb+input_file+ is the
|
|
name of your input file.
|
|
|
|
% no longer the case with modern kernels
|
|
%\section{Linux}
|
|
%
|
|
%If running in parallel across multiple machines you should consider
|
|
%applying this patch to your kernel to boost the performance of TCP/IP
|
|
%\begin{itemize}
|
|
%\item \htmladdnormallink{http://www.icase.edu/coral/LinuxTCP.html}{http://www.icase.edu/coral/LinuxTCP.html}
|
|
%%\end{itemize}
|
|
|
|
\section{Alpha systems with Quadrics switch}
|
|
|
|
\begin{verbatim}
|
|
prun -n <npes> $NWCHEM_TOP/bin/$NWCHEM_TARGET/nwchem <input_file>
|
|
\end{verbatim}
|
|
|
|
where \verb+npes+ is the number of processors and \verb+input_file+ is the
|
|
name of your input file.
|
|
|
|
\section{Windows 98 and NT}
|
|
|
|
\begin{verbatim}
|
|
$NWCHEM_TOP/bin/win32/nw32 <input_file>
|
|
\end{verbatim}
|
|
|
|
where and \verb+input_file+ is the
|
|
name of your input file.
|
|
If you use WMPI, you must have a file named {\bf \tt nw32.pg} in the
|
|
\verb+ $NWCHEM_TOP/bin/win32+ directory; the file must only contains the
|
|
following single line
|
|
\begin{verbatim}
|
|
local 0
|
|
\end{verbatim}
|
|
|
|
|
|
|
|
\section{Tested Platforms and O/S versions}
|
|
|
|
\begin{itemize}
|
|
\item IBM SP with Power 3 and Power 4 nodes, AIX 5.1
|
|
and PSSP 3.4; IBM RS6000 workstation, AIX 5.1. Xlf 8.1.0.0 and
|
|
8.1.0.1 are known to produce bad code.
|
|
\item SUN workstations with Solaris 2.6 and 2.8. Fujitsu SPARC systems
|
|
(thanks to Herbert Fr\"uchtl) with Parallelnavi compilers.
|
|
\item HP DEC alpha workstation , Tru64 V5.1,
|
|
Compaq Fortran V5.3, V5.4.2, V5.5.1
|
|
\item Linux with Intel x86 cpus.
|
|
NWChem Release 4.5 has been tested on RedHat 6.x and 7.x,
|
|
Mandrake 7.x.
|
|
We have tested NWChem on Linux for the Power PC Macintosh with
|
|
Yellow Dog 2.4.
|
|
These all use the GCC compiler at different levels.
|
|
The Intel Fortran Compiler version 7 is supported.
|
|
The Portland Group Compiler has been tested in a less robust manner.
|
|
Automatic generation of SSE2 optimized code is available when the
|
|
Intel compiler is used (ifc vs g77 performances gain of 40\% in
|
|
some benchmarks)
|
|
A somewhat Athlon optimized code can be generated under the GNU
|
|
or Intel compilers by typing {\tt make \_CPU=k7}.
|
|
GCC3 specific options can be turned on by typing {\tt make GCC31=y}
|
|
\item HP 9000/800 workstations with HPUX B.11.00. f90 must be used for
|
|
compilation.
|
|
\item Intel x86 with Windows 2000 has been tested with Compaq Visual Fortran
|
|
6.0 and 6.1 with WMPI 1.3 or NT-Mpich.
|
|
NT-MPICH is available from
|
|
\htmladdnormallink{http://www-unix.mcs.anl.gov/\~\space ashton/mpich.nt/}{http://www-unix.mcs.anl.gov/~ashton/mpich.nt/}
|
|
|
|
\item Intel IA64 under Linux (with Intel compilers version 7 and later)
|
|
and under HPUX.
|
|
\item Fujitsu VPP computers.
|
|
|
|
|
|
\end{itemize}
|
|
|
|
%%% Local Variables:
|
|
%%% mode: latex
|
|
%%% TeX-master: "user"
|
|
%%% End:
|