FNAL - LQCD Documentation

New Users / Account Renewal

User Authentication

Kerberos and SSH Troubleshooting

Building your code - The Runtime Environment

Submitting jobs to the TORQUE Batch System

Project Allocations

Software Documentation Details

Hardware Details

Filesystem Details

Mass/Tape Storage Details

Transferring Files

Compilers

FAQs

Runtime Environment

To simplify the use of the multiple Lattice QCD cluster facilities at BNL, FNAL, and JLAB, participants at these different sites have agreed on a set of common features. Makefiles and scripts that take advantage of these common features instead of relying on site specific characteristics will be easier to port. Users who become familiar with the standard environment should be able to move between facilities with little difficulty.

Paths and Environment Variables

When operating in the standard environment, you may locate SciDAC lattice gauge related libraries, binaries, and header files using a number of environment variables. On the cluster login nodes (for e.g. jpsi1.fnal.gov), these variables are assigned automatically for you when you login. If you wish to use a version of these libraries other than the default installation, reassign these environment variables as desired, either directly or using a configuration management system such as UPS.

The facilities currently supported and their corresponding variable names are:

Environment Variable Name Value
QMP_DIR qmp Installation Directory
QLA_DIR qla Installation Directory
QIO_DIR qio Installation Directory
QDP_DIR qdp Installation Directory
MPI_DIR mpi Installation Directory
GCC_DIR gcc Installation Directory
MILC_DIR MILC example Location Directory

Note that each of these environment variables points to the top level of the directory structure corresponding to that facility. To access libraries or headers, append the standard subdirectory name. For example, your Makefile might include the following CFLAG

-I $QLA_DIR/include

Building Your Code

For portability, your Makefiles should refer to the SciDAC libraries and other standard facilities using the environment variables listed above rather than hard coded paths.

As an example, look in $MILC_DIR for a working sample of the MILC improved staggered code. In the directory ks_imp_dyn, there are two Makefiles, Make_linux_qdp and Make_linux_qmp, which may be used for building the QDP and MPI versions of the code. In Make_linux_qdp, the standard environment variables are used to avoid hard coded paths. A few lines from that Makefile illustrate the usage:

  QDP = $(QDP_DIR)
QLA = $(QLA_DIR)
QIO = $(QIO_DIR)
QMP = $(QMP_DIR)

To build your own versions of binaries from this MILC example, cut and paste the following lines:

mkdir my_milc_sample
cd my_milc_sample
(cd $MILC_DIR; tar -cf - .) | tar -xf -
cd ks_imp_dyn/
make -f Make_linux_qdp clean
make -f Make_linux_qdp su3_rmd
mv su3_rmd su3_rmd_qdp

usqcd-webmaster@usqcd.org