FNAL - LQCD Documentation

New Users / Account Renewal

User Authentication

Kerberos and SSH Troubleshooting

Project Allocations

SLURM Batch System

Software Details

Hardware Details

Filesystem Details

Mass/Tape Storage Details

Globus Online

FAQs

LIVE Cluster Status

Contact Us

Compilers

  • These are installed under /usr/local/ -OR- use the module av command to list available compilers.
    [@lattice:/usr/local]$~ module av
    
       Anaconda/2-5.2.0           (D)    fftw/3.3.8/serial   (D)
       Anaconda/3-5.2.0                  gcc/4.8.4           (D)
       autoconf/2.69                     gcc/6.4.0
       automake/1.15                     gcc/8.1.0
       cmake/3.11.4                      git/2.9.5
       cuda/8.0                          julia/0.6.3
       cuda/9.0                   (D)    m4/1.4.18
       cuda/9.1                          mpi/mvapich2/2.2
       cuda/9.2                          mpi/openmpi/1.10.17 (D)
       fftw/3.3.8/mvapich2-2.2           mpi/openmpi/2.0.1
       fftw/3.3.8/openmpi-1.10.17        mpi/openmpi/2.1.3
       fftw/3.3.8/openmpi-2.0.1          mpi/openmpi/3.1.1
       fftw/3.3.8/openmpi-2.1.3          singularity/2.5.2
       fftw/3.3.8/openmpi-3.1.1
    
      Where:
       D:  Default Module
     
      List generated on Jul 16, 2018
    

  • We have found mixed success in launching MPI jobs with mpirun and srun. Here is a recommended list of launch commands for each MPI version (last updated: Jul 16, 2018):

    MPImpirun -np Nsrun -n N
    mvapich2-2.2FAIL - UPMI_KVS_GET errorPASS
    openmpi-1.10.7PASSFAIL - launches N rank 1 binaries
    openmpi-2.0.1PASSPASS
    openmpi-2.1.3PASSPASS
    openmpi-3.1.1PASSPASS

Useful Documentation for Lattice Cluster Computing

Fermilab Software Projects

USQCD Software Projects

usqcd-webmaster@usqcd.org