Navigation

Spack

AMD Toolchain with SPACK

Micro Benchmarks/Synthetic

SPACK HPC Applications

Introduction

CP2K is a quantum chemistry and solid state physics software package that can perform atomistic simulations of solid state, liquid, molecular, periodic, material, crystal, and biological systems. CP2K provides a general framework for different modeling methods such as DFT using the mixed Gaussian and plane waves approaches.

CP2K is written in Fortran 2008 and can be run efficiently in parallel using a combination of OpenMP multi-threading and MPI using AOCC and AOCL. The Spack framework and the instructions below provide a convenient way to build CP2K optimized for your platform and package versions.

The CP2K official website: https://www.cp2k.org

Build CP2K using Spack

Reference to add external packages to Spack: Build Customization (Adding external packages to Spack)

# Format For Building CP2K
$ spack -d install -v cp2k@<Version Number>+elpa %aocc@<Version Number> target=<zen2/zen3>  ^amdfftw@<Version Number> ^amdscalapack@<Version Number> ^amdblis@<Version Number> ^amdlibflame@<Version Number> ^libint@<Version Number> ^libxsmm@<Version Number> ^libxc@<Version Number> ^elpa@<Version Number>+openmp ^openmpi@<Version Number>+cxx fabrics=auto
# Example: For building CP2K 7.1 with ELPA 2019, AOCC 3.0 and AOCL 3.0
$ spack -d install -v cp2k@7.1+elpa %aocc@3.0.0 target=zen3 ldflags="-Wl,-z,muldefs" ^amdfftw@3.0 ^amdscalapack@3.0 ^amdblis@3.0 ^amdlibflame@3.0 ^libint@2.6.0 ^libxsmm@1.11 ^libxc@4.2.3 ^elpa@2019.11.001+openmp ^python@2 ^openmpi@4.0.3+cxx fabrics=auto
# Example: For building CP2K 6.1 with ELPA 2017, AOCC 3.0 and AOCL 3.0
$ spack -d install -v cp2k@6.1+elpa %aocc@3.0.0 target=zen3 ldflags="-Wl,-z,muldefs" ^amdfftw@3.0 ^amdscalapack@3.0 ^amdblis@3.0 ^amdlibflame@3.0 ^libint@1.1.6 ^libxsmm@1.11 ^libxc@4.2.3 ^elpa@2017.05.001+openmp ^python@2 ^openmpi@4.0.3+cxx fabrics=auto
# Example: For building CP2K 6.1 with ELPA 2017, AOCC 2.3 and AOCL 2.2
$ spack -d install -v cp2k@6.1+elpa %aocc@2.3.0 target=zen2 ldflags="-Wl,-z,muldefs" ^amdfftw@2.2 ^amdscalapack@2.2 ^amdblis@2.2 ^amdlibflame@3.0 ^libint@1.1.6 ^libxsmm@1.11 ^libxc@4.2.3 ^elpa@2017.05.001+openmp ^python@2 ^openmpi@4.0.3+cxx fabrics=auto
# Example: For building CP2K 6.1 with ELPA 2017, AOCC 2.2 and AOCL 2.2
$ spack -d install -v cp2k@6.1+elpa %aocc@2.2.0  target=zen2 ldflags="-Wl,-z,muldefs" ^amdfftw@2.2 ^amdscalapack@2.2 ^amdblis@2.2 ^amdlibflame@3.0 ^libint@1.1.6 ^libxsmm@1.11 ^libxc@4.2.3 ^elpa@2017.05.001+openmp ^python@2 ^openmpi@4.0.3+cxx fabrics=auto

Versions supported and compatibility:

Component/Application Versions Applicable
CP2K 7.1 6.1
AOCC 3.0.0 3.0.0, 2.3.0, 2.2.0
AOCL 3.0 3.0, 2.2

Specifications and Dependencies

Symbol Meaning
-d To enable debug output
-v To enable verbose
@ To specify version number
% To specify compiler
+elpa Build with ELPA variant
^amdfftw Build with AMDFFTW
^amdscalapack Build with AMDSCALAPACK
^amdblis Build with AMDBLIS
^libint Build with LIBINT
^libxsmm Build with LIBXSMM
^libxc~shared Build with LIBXC static only
^elpa+openmp Build with ELPA of openmp variant
^open­mpi­@4.0.3+cxx Use openm­pi@­4.0.3 for build with cxx enabled
fabri­cs=­auto Use fabri­cs=­auto variant for openmpi, by default fabri­cs=­none

Running  CP2K

Performance Factors

The default optimization level when building CP2K 6.1 or 7.1 in the Spack environment is -O1.  The CP2K runtime performance is affected by (1) compile-time optimizations and (2) process-binding arguments provided to mpirun at runtime.

  1. Compile-time optimization: The flagset -O3 -ffast-math -Hx,47,0x10000008 provides a better performance of ~6% (albeit with a reduced accuracy associated -ffast-math) when compared to the Spack/AOCC default -O1.
    Note: When using -O2/-O3 with AOCC 2.2/2.3, it is necessary to use the flag -Hx,47,0x10000008, to overcome an optimization issue.
  2. Process binding/pinning at runtime is observed to significantly affect CP2K performance both positively or negatively, depending on problem type and problem size.  AMD recommends the user test performance using different options, such as --bind-to core--bind-to socket etc. or no binding at all.  For example, with the benchmark case H2O-dft-ls, not binding MPI processes to hardware has been observed to deliver the best performance.
Setting Environment
# Format to load CP2K build with AOCC
$ spack load cp2k@<Version Number> %aocc@<Version Number>
# Example: Load CP2K-6.1 module into environment
$ spack load cp2k@6.1 %aocc@3.0.0

 

Obtaining Benchmarks
# The dataset H2O-dft-ls.NREP4 may be downloaded (manually) from the location below
# https://github.com/cp2k/cp2k/blob/master/benchmarks/QS_DM_LS/H2O-dft-ls.NREP4.inp

 

Run command using Open MPI
# CP2K installed directory : spack cd -i cp2k@6.1 %aocc@3.0.0
export OMP_NUM_THREADS=1
$ mpirun -np 128 <cp2k_install_dir>/bin/cp2k.psmp H2O-dft-ls.NREP4.inp