Navigation

Spack

AMD Toolchain with SPACK

Micro Benchmarks/Synthetic

SPACK HPC Applications

Introduction

GROMACS is a versatile package for performing molecular dynamics for systems with hundreds to millions of particles.  GROMACS is primarily designed for biochemical molecules like proteins, lipids, and nucleic acids that have a multitude of complicated bonded interactions.  Since GROMACS is extremely fast at calculating the non-bonded interactions typically dominating simulations, many researchers use it for research on non-biological systems, such as polymers.  GROMACS supports all the usual algorithms expected from a modern molecular dynamics implementation.

GROMACS official website: https://www.gromacs.org/

Build GROMACS using Spack

Reference to add external packages to Spack: Build Customization (Adding external packages to Spack)

# Format For Building GROMACS with AOCC
$ spack -d install -v gromacs@<Version Number> %aocc@<Version Number> target=<zen2/zen3> +openmp +blas +lapack ^amdlibflame@<Version Number> ^amdblis@<Version Number> ^amdfftw@<Version Number> ^openmpi@<Version Number>
# Example: For building GROMACS 2020.4 with AOCC-3.0.0 AOCL-3.0 and Open MPI-4.0.3
$ spack -d install -v gromacs@2020.4 %aocc@3.0.0 target=zen3 +blas +lapack ^amdlibflame@3.0 ^amdblis@3.0 ^amdfftw@3.0 ^openmpi@4.0.3
# Example: For building GROMACS 2020.4 with AOCC-2.3.0, AOCL-2.2 and Open MPI-4.0.3
$ spack -d install -v gromacs@2020.4 %aocc@2.3.0 target=zen2 cflags="-Ofast -ffp-contract=fast" cxxflags="-Ofast -ffp-contract=fast" +blas +lapack ^amdlibflame@2.2 ^amdblis@2.2 ^amdfftw@2.2 ^openmpi@4.0.3
# Example: For building GROMACS 2020.4 with AOCC-2.2.0, AOCL-2.2 and Open MPI-4.0.3
$ spack -d install -v gromacs@2020.4 %aocc@2.2.0 target=zen2 cflags="-Ofast -ffp-contract=fast" cxxflags="-Ofast -ffp-contract=fast" +blas ^amdblis@2.2 ^amdfftw@2.2 ^openmpi@4.0.3

Please use any combination of below components/Applications and its versions.

Component/Application Versions Applicable
GROMACS 2020.4, 2020.3, 2020.2
AOCC 3.0.0, 2.3.0, 2.2.0
AOCL 3.0, 2.2

Specifications and Dependencies

Symbol Meaning
-d To enable debug output
-v To enable verbose
@ To specify version number
% To specify compiler
target=zen2 / target=zen3 To enable target zen2 or zen3 architecture
+openmp Build with openmp variant
cflags To add cflags to the Spack env using command line

-Ofast: Enables all the optimizations from -O3 along with other aggressive optimizations that may violate strict compliance with language standards.

ffp-contract=fast enables floating-point expression contraction such as forming of fused multiply-add operations if the target has native support for them.

cxxflags To add cxxflags to Spack env using command line

Ofast: Enables all the optimizations from -O3 along with other aggressive optimizations that may violate strict compliance with language standards.

ffp-contract=fast enables floating-point expression contraction such as forming of fused multiply-add operations if the target has native support for them.

+blas Blas variant specifier
+lapack Lapack variant specifier
^amdlibflame Build with amdlibflame dependency
^amdblis Build with amdblis dependency
^amdfftw Build with amdfftw dependency
^openmpi Build with Open MPI 4.0.3

Running GROMACS

GROMACS has vast variety of input data sets.  The following example steps we will run GROMACS with the commonly used “water_GMX50_bare” benchmark data set.

Running GROMACS is a two-step process:

  1. Generating the input file (.tpr)
  2. Running the generated .tpr file

Examples for running GROMACS on AMD 2nd and 3rd Gen EPYC processors are presented below.

Running GROMACS on AMD 2nd Gen EPYC Processors

Setting Environment
# Format for loading Gromacs build with AOCC
$ spack load gromacs@<Version Number> %aocc@<Version Number>
# Example: Load Gromacs build with AOCC-2.2 module into environment
$ spack load gromacs@2020.4 %aocc@2.2.0

 

Obtaining Benchmarks
# obtaining bench-mark "water_GMX50_bare" i.e. water 3072 and 1536
# This is the standard benchmark sets that can be downloaded
$ wget http://ftp.gromacs.org/pub/benchmarks/water_GMX50_bare.tar.gz
tar -xvf water_GMX50_bare.tar.gz
cd water-cut1.0_GMX50_bare/3072
# To generate the .tpr input file
# To get installed folder path: spack cd -i gromacs@2020.4 %aocc@2.2.0
export GMX=/<gromacs_installed_path>/bin/gmx_mpi
$ $GMX grompp -f pme.mdp -c conf.gro -p topol.top -o water_pme.tpr

Run command using Open MPI

Water3072: Single Node
# Run Command
$ mpirun --bind-to core --map-by hwthread -use-hwthread-cpus -np 256 -mca btl vader,self $GMX mdrun -notunepme -dlb yes -v -resethway -noconfout -nsteps 4000 -s water_pme.tpr

Running GROMACS on AMD 3rd Gen EPYC Processors

Setting Environment
# Format for loading GROMACS build with AOCC
$ spack load gromacs@<Version Number> %aocc@<Version Number>
# Example: Load GROMACS-2020.4 build with AOCC-3.0 module into environment
$ spack load gromacs@2020.4 %aocc@3.0.0

 

Obtaining Benchmarks
# obtaining datasets "water_GMX50_bare" i.e. water 3072 and 1536 .
# This is the standard benchmark sets that can be downloaded
$ wget http://ftp.gromacs.org/pub/benchmarks/water_GMX50_bare.tar.gz
tar -xvf water_GMX50_bare.tar.gz
cd water-cut1.0_GMX50_bare/3072
# To generate the .tpr input file
# To get installed folder path: spack cd -i gromacs@2020.4 %aocc@3.0.0
export GMX=/<gromacs_installed_path>/bin/gmx_mpi
$ $GMX grompp -f  pme.mdp  -c conf.gro -p topol.top -o water_pme.tpr

Run command using Open MPI

Water3072: Single Node
# Run Command
$ mpirun --bind-to core --map-by hwthread -use-hwthread-cpus -np 256 -mca btl vader,self $GMX mdrun -notunepme -dlb yes -v -resethway -noconfout -nsteps 4000 -s water_pme.tpr