MPI Tutorial
MPI's "Hello World"
Name this little "Hello World" program hello.c:
#include <stdio.h>
#include <mpi.h>
int main(int argc, char *argv[]) {
int numprocs, rank, namelen;
char processor_name[MPI_MAX_PROCESSOR_NAME];
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &numprocs);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Get_processor_name(processor_name, &namelen);
printf("Process %d on %s out of %d\n", rank, processor_name, numprocs);
MPI_Finalize();
}
Compiling and Linking MPI Programs
Once you have your MPI example program, you can compile and link
it with the MPI version that supports the network type you need.
- Linux with Ethernet or Infiniband (OpenMPI):
mpicc hello.c -o hello
Open MPI is using the following environment variables for alternative
compilers:
C: OMPI_CC
C++: OMPI_CXX
Fortran 77: OMPI_F77
Fortran 90: OMPI_FC
The comple list of environment variables for compilers, compiler flags, linkters,
and linkger flags can be found in the
OpenMPI FAQ
Running MPI Programs
Logon to the pinto cluster: ssh pinto
Make sure your MPI program is compiled for these machines (RHEL5).
Add the machines you want your parallel program to run on to a file:
alfw@pinto> cat machinefile.pinto
pinto01
pinto01
pinto02
pinto02
This will first use pinto01 and then use pinto02.
If you need more processes than CPUs are in your machinefile, MPI will
start again at the top of the file.
To run your program interactively, execute one of these commands
LSF Integration and Usage
Currently there is one LSF queue defined for running MPI jobs via LSF. The mpi-ibq.
The mpi-ibq has been designated as the default production MPI queue and is configured to
use the 32 node pinto Infiniband cluster. Presently the mpi-ibq is setup to use a maximum of
8 processors per machine. Which gives this queue a total of 256 available processors.
Before you submit an MPI job to LSF, make sure you have the correct
mpirun binary in your $PATH environment variable.
On the pinto cluster this is accomplished by using mpi-selector. The system default
is set for the cluster but you may choose to use a different version of OpenMPI and
prefer to set a user level default.
To submit your MPI programs via LSF to either cluster, you have to compile
your MPI jobs just as you would for interactive job launch.
Instead of using mpirun for program start, you have to use the
following LSF command from either a morab or norica pinto machine:
To submit jobs to the default mpi-ibq (pinto cluster):
bsub -a mympi -n <number of processors> <OpenMPI_program>
Where -a mympi is the esub required to run OpenMPI jobs.
Example:
bsub -a mympi -n 10 ~/Prog/OpenMPI/bin/hello++
Each MPI queue by default is configured with the span resource empty (span[]) to
allow jobs to be scheduled on machines that have an empty processor based on normal
LSF load considerations. If you wish to specify a specific spanning resource option
to suit your parallel batch job you may use -R "span[ptile=<value>]" on the
bsub command.
Where <value> is the number of processors you wish to use on each host.
Example:
bsub -a mympi -n 10 -R "span[ptile=2]" ~/Prog/OpenMPI/bin/hello++
In this example, "span[ptile=2]" will require the job to run on
two processors per host.
Some jobs may perform better when run on a single processor per host. In this case you would want
to specify -R "span[ptile=1]". Consider that we currently have 64 hosts allocated to the
mpiq. If you use "span[ptile=1]" the -n <number of processors> option
on bsub could be no greater than 32 for your job to run.
Example:
bsub -a mympi -n 32 -R "span[ptile=1]" ~/Prog/OpenMPI/bin/hello++
Submitting an OpenMPI job to use the old package verson of OpenMPI job presently works by using the
-a openmpi parameter.
Example:
bsub -q mpi_ibq -a openmpi -n 10 ~/Prog/OpenMPI/bin/hello++
References
