Compilers and Communication Libraries
The table below lists compilers installed on the cluster. Unix man page
documentation is available for individual compilers.
The Portland Group compilers are not binary compatible with gnu meaning
that they will not accept object files or libraries created by the gnu compilers.
Compiler drivers, such as mpicc, are provided for use when compiling
programs containing MPI
message passing directives.
The implementation of MPI is
mpich
using an abstract communication
device implemented in GM,
VMI and VAPI.
Compilers Available on LQCD
| Compiler Name and Version |
Binary Location |
Binary Name(s) |
|
Gnu Compilers Version 3.4.6 |
/usr/bin |
g++, gcc, g77 |
|
Gnu Compilers Version 3.2.3 |
/usr/local/gcc-3.2.3/bin |
g++, gcc, g77 |
|
Gnu Compilers Version 3.4.5 |
/usr/local/gcc-3.4.5/bin |
g++, gcc, g77 |
| Intel Compiler Version 7.0 |
/fnal/ups/prd/intel/v7_0/ia32/bin |
icc,
ifc (C and C++, F77 and F90) |
| Intel Compiler Version 8.1 (*) |
/fnal/ups/prd/intel/v8_1/bin |
icc,
ifort (C and C++, F77 and F90) |
|
/usr/local/mvapich-intel |
mpicc, mpiCC, mpif77, mpif90 |
|
/usr/local/mpich-gm-intel |
mpicc, mpiCC, mpif77, mpif90 |
|
/usr/local/mpich-vmi-intel |
mpicc, mpif77, mpif90 |
| Intel Compiler Version 9.1 |
/fnal/ups/prd/intel/v9_1/bin |
icc,
ifort (C and C++, F77 and F90) |
|
/usr/local/mvapich-intel-9.1 |
mpicc, mpiCC, mpif77, mpif90 |
| Portland Group (**) |
/fnal/ups/prd/pgi/v5_0/linux86/5.0/bin |
pgCC, pgcc, pgf77, pgf90 |
|
/usr/local/mvapich-pgi |
>mpicc, mpif77, mpif90 |
|
/usr/local/mpich-gm-pgi |
>mpicc, mpif77, mpif90 |
|
/usr/local/mpich-vmi-pgi |
>mpicc, mpif77, mpif90 |
(*) Use the following:
. /usr/local/etc/setups.sh or source /usr/local/etc/setups.csh
setup intel v8_1
to add the Intel compiler version 8.1 and libraries to your environment.
(**) Use the following:
. /usr/local/etc/setups.sh or source /usr/local/etc/setups.csh
setup pgi
to add the Portland Group compiler and libraries to your environment and to access
the license manager. If you'd prefer not to use "setup", be sure to set
LM_LICENSE_FILE=/fnal/ups/prd/pgi/v5_0/license.dat.
(**)The binaries that are generated using the PGI compilers are not GNU compatible.
Applications compiled with the Portland Group and Intel compilers must be statically
linked with their respective object libraries since the dynamically linkable libraries are not
available from cluster worker nodes.
Compilers Available on KAON
| Compiler Name and Version |
Binary Location |
Binary Name(s) |
|
Gnu Compilers Version 3.4.4 |
/usr/bin |
g++, gcc, g77 |
|
/usr/local/mvapich |
mpicc, mpiCC, mpif77, mpif90 |
|
/usr/local/openmpi |
mpicc, mpiCC, mpif77, mpif90 |
|
Gnu Compilers Version 3.3.4 |
/usr/local/gcc-3.3.4/bin |
g++, gcc, g77 |
|
Gnu Compilers Version 4.1.0 |
/usr/bin/gcc4 |
g++, gcc, g77 |
| Intel Compiler Version 9.1 (*) |
/fnal/ups/prd/intel/v9_1/bin |
icc,
ifc (C and C++, F77 and F90) |
| Portland Group (**) |
/fnal/ups/prd/pgi/v6.2-3/linux86-64/6.2/bin |
pgCC, pgcc, pgf77, pgf90 |
|
/usr/local/mvapich-pgi |
>mpicc, mpiCC, mpif77, mpif90 |
| QLogic Pathscale Compiler |
/opt/pathscale/bin |
pathcc, pathCC, pathf90, pathf95 |
(*) Use the following:
. /usr/local/etc/setups.sh or source /usr/local/etc/setups.csh
setup intel v9_1
to add the Intel compiler and libraries to your environment.
(**) Use the following:
. /usr/local/etc/setups.sh or source /usr/local/etc/setups.csh
setup pgi
to add the Portland Group compiler and libraries to your environment and to access
the license manager. If you'd prefer not to use "setup", be sure to set
LM_LICENSE_FILE=/fnal/ups/prd/pgi/v6.2-3/license.dat.
(**)The binaries that are generated using the PGI compilers are not GNU compatible.
Applications compiled with the Portland Group, Pathscale and Intel compilers must be statically
linked with their respective object libraries since the dynamically linkable libraries are not
available from cluster worker nodes.
Compiling an MPI application
Two MPI versions are available on the Infiniband (pion) cluster - mvapich, which uses
the Ohio State University
MPI and mpich-vmi, which uses the National Center
for Supercomputing Applications (NCSA) MPI based on their VMI adapter layer. We recommend using
mvapich; however, mpich-vmi will allow you to use the same binaries on
the Myrinet and Infiniband clusters. Two MPI versions are also available on the Myrinet (qcd)
cluster - mpich-gm, which uses the
Myrinet MPI and mpich-vmi.
To build codes, use
$ /usr/local/mvapich/bin/mpicc # or mpiCC, mpif77, etc... (Infiniband)
or
$ /usr/local/mpich-vmi/bin/mpicc # or mpiCC, mpif77, etc... (Infiniband, Myrinet)
or
$ /usr/local/mpich-gm/bin/mpicc # or mpiCC, mpif77, etc... (Myrinet)
or
One of the MPICH versions as compiled with the Portland Group or Intel compiler.
To run codes, use
$ /usr/local/mvapich/bin/mpirun_rsh (Infiniband)
or
$ /usr/local/mpich-vmi/bin/mpirun (Infiniband, Myrinet)
or
$ /usr/local/mpich-gm/bin/mpirun (Myrinet)
Please note that the syntax of these three versions of mpirun is different.
Please use the following syntax:
mvapich: (Infiniband)
$ mpirun_rsh -rsh -np X your.binary arguments
mpich-vmi: (Infiniband)
$ mpirun -np X -specfile openib your.binary arguments
mpich-vmi: (Myrinet)
$ mpirun -np X -specfile myrinet your.binary arguments
mpich-gm: (Myrinet)
$ mpirun -np X your.binary arguments
Please note that "-specfile openib" for Infiniband or "-specfile myrinet"
for Myrinet is critical for mpich-vmi. Your jobs will fail otherwise.
To use a compiler, first check that its executable path is in your shell's
PATH environment variable. Use the unix env command to see how PATH is
currently set. If the desired compiler is not in your PATH, edit
your .bashrc (for bash or sh users) or .cshrc (for tcsh or csh users)
to add the compiler's path. For example, to add mpiCC, mpicc and mpif77
to your path, add the following line to your shell's start-up script:
| .bashrc | .cshrc |
| export PATH=$PATH:/usr/local/mvapich/bin |
setenv PATH $PATH:/usr/local/mvapich/bin |
Here is how to compile example program cpi.c which comes with the mvapich
software distribution.
$ cp /usr/local/mvapich/example/cpi.c . # copy source to your directory
$ mpicc cpi.c -o cpi # compile
|