AMD Toolchain with SPACK

Micro Benchmarks/Synthetic

SPACK HPC Applications


ROMS is a free-surface, terrain-following, primitive equations ocean model widely used by the scientific community for a diverse range of applications. ROMS includes accurate and efficient physical and numerical algorithms and several coupled models for biogeochemical, bio-optical, sediment, and sea ice applications.

Official Website for ROMS:

Getting ROMS Source Code

A manual download is required for ROMS. Spack will search your current working directory for the download files.

Check the steps to obtaining ROMS in the URL:

Please maintain the source code package format as “roms_3.8_source.tar.gz” from the source code by running the command “tar -czvf roms_3.8_source.tar.gz ./ROMS_source“.

Build ROMS using Spack

Reference to add external packages to Spack: Build Customization (Adding external packages to Spack)

# Format For Building ROMS with AOCC
$ spack -d install -v --no-checksum roms@<Version> %aocc@<Version> target=<zen2/zen3> ^hdf5@<Version> ^netcdf-c@<Version> ^netcdf-fortran@<Version> ^openmpi@<Version>
# Example: For building ROMS 3.8 with AOCC 3.2 and AOCL 3.1
$ spack -d install -v --no-checksum roms@3.8 %aocc@3.2.0 target=zen3 ^amdlibm@3.1 ^hdf5@1.10.7+hl+cxx+fortran ^netcdf-c@4.7.0 ^netcdf-fortran@4.5.2~shared ^openmpi@4.1.1 fabrics=auto

Use any combination of the following components/applications and their corresponding versions:

Component/Application Versions Applicable
ROMS 3.8
AOCC 3.2.0
AOCL 3.1

Specifications and Dependencies

Symbol Meaning
-d To enable debug output
-v To enable verbose
@ To specify version number
–no-checksum Don’t check SHA-checksums
target=zen3 Build for AMD “Zen3” architecture
^hdf5+hl+cxx+fortran To build with hdf5 for C++ and Fortran variants
^netcdf-c To build with netcdf-c
^netcdf-fortran To build with netcdf-fortran
^open­mpi­ fabri­cs=­auto To build with Open MPI and using fabrics=auto

Running ROMS

Obtaining Benchmarks

  • For testing ROMS performance, the following workload has been considered:
    • Make sure ROMS/ directory from the source code available in your system before running the benchmark.
    • is a standard input file can be found in “ROMS/External/
    • Modify to increase its grid size to maximum fit in the resolution in X- direction.
    • Recommended benchmark settings are:
      • Lm==8192, Mm==256, N==30
      • NtileI==8, NtileJ==8
      • NTIMES==1000
Setting Environmennt
# Format for loading ROMS build with AOCC
$ spack load roms@<Version> %aocc<Version>
# Example : Load ROMS 3.8 build with AOCC 3.2 module into environment
$ spack load roms@3.8 %aocc@3.2.0


Run Command
$ export OMP_NUM_THREADS=1
# Run the required input file
# -np = NtileI X NtileJ, from the above recommended benchmark, -np = 8x8 = 64
$ time mpirun -np 64 --bind-to l3cache romsM /ROMS/External/