Skip to content

Commit

Permalink
Add gaea-c5 config, bugfix for Narwhal config (supersedes #439) (#445)
Browse files Browse the repository at this point in the history
* Add gaea-c5 config. Tested with ufs-weather-model-static template.
* Bug fix in Narwhal compiler config, and update of Gaea C5 config
* Progress on documentation update in doc/source/Platforms.rst
* Final touches to gaea-c5 site config (packages)

Co-authored-by: Alexander Richert <[email protected]>
  • Loading branch information
climbfuji and Alexander Richert authored Jan 20, 2023
1 parent 6bb35ae commit fab1fc5
Show file tree
Hide file tree
Showing 7 changed files with 315 additions and 10 deletions.
27 changes: 27 additions & 0 deletions configs/sites/gaea-c5/compilers.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
compilers:
- compiler:
spec: [email protected]
paths:
cc: cc
cxx: CC
f77: ftn
fc: ftn
flags: {}
operating_system: sles15
target: any
modules:
- PrgEnv-intel/8.3.3
- intel/2022.0.2
- craype/2.7.15
- libfabric/1.11.0.4.128
environment:
prepend_path:
PATH: '/opt/cray/pe/gcc/10.3.0/snos/bin'
CPATH: '/opt/cray/pe/gcc/10.3.0/snos/include'
LD_LIBRARY_PATH: '/opt/cray/pe/gcc/10.3.0/snos/lib:/opt/cray/pe/gcc/10.3.0/snos/lib64'
set:
# OpenSUSE on Gaea C5 sets CONFIG_SITE so
# Automake-based builds are installed in lib64
# which confuses some packages.
CONFIG_SITE: ''
extra_rpaths: []
2 changes: 2 additions & 0 deletions configs/sites/gaea-c5/config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
config:
build_jobs: 6
18 changes: 18 additions & 0 deletions configs/sites/gaea-c5/mirrors.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
mirrors:
local-source:
fetch:
url: file:///lustre/f2/pdata/esrl/gsd/spack-stack/source-cache
access_pair:
- null
- null
access_token: null
profile: null
endpoint_url: null
push:
url: file:///lustre/f2/pdata/esrl/gsd/spack-stack/source-cache
access_pair:
- null
- null
access_token: null
profile: null
endpoint_url: null
4 changes: 4 additions & 0 deletions configs/sites/gaea-c5/modules.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
modules:
default:
enable::
- lmod
241 changes: 241 additions & 0 deletions configs/sites/gaea-c5/packages.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,241 @@
packages:
all:
compiler:: [[email protected]]
providers:
mpi:: [[email protected]]
cray-mpich:
externals:
- spec: [email protected]%[email protected]~wrappers
modules:
- libfabric
- craype-network-ofi
- cray-mpich/8.1.16
esmf:
version: [8.3.0b09]
variants: ~xerces ~pnetcdf +pio esmf_os=Linux esmf_comm=mpich3
python:
buildable: false
externals:
- spec: [email protected]
modules: [python/3.9.12]
autoconf:
externals:
- spec: [email protected]
prefix: /usr
automake:
externals:
- spec: [email protected]
prefix: /usr
bash:
externals:
- spec: [email protected]
prefix: /usr
binutils:
externals:
- spec: [email protected]
prefix: /usr
bison:
externals:
- spec: [email protected]
prefix: /usr
# Don't use, it's missing the headers
#bzip2:
# externals:
# - spec: [email protected]
# prefix: /usr
cmake:
buildable: false
externals:
- spec: [email protected]
modules: [cmake/3.23.1]
coreutils:
externals:
- spec: [email protected]
prefix: /usr
cpio:
externals:
- spec: [email protected]
prefix: /usr
curl:
externals:
- spec: [email protected]+gssapi+ldap+nghttp2
prefix: /usr
diffutils:
externals:
- spec: [email protected]
prefix: /usr
dos2unix:
externals:
- spec: [email protected]
prefix: /usr
ecflow:
buildable: False
externals:
- spec: [email protected]+ui+static_boost
prefix: /lustre/f2/scratch/Dom.Heinzeller/spack-stack-c5/ecflow-5.8.4
modules: [ecflow/5.8.4]
file:
externals:
- spec: [email protected]
prefix: /usr
findutils:
externals:
- spec: [email protected]
prefix: /usr
flex:
externals:
- spec: [email protected]+lex
prefix: /usr
gawk:
externals:
- spec: [email protected]
prefix: /usr
gettext:
externals:
- spec: [email protected]
prefix: /usr
ghostscript:
externals:
- spec: [email protected]
prefix: /usr
git:
buildable: false
externals:
- spec: [email protected]
modules: [git/2.35.2]
git-lfs:
buildable: false
externals:
- spec: [email protected]
modules: [git-lfs/2.11.0]
gmake:
externals:
- spec: [email protected]
prefix: /usr
groff:
externals:
- spec: [email protected]
prefix: /usr
hwloc:
externals:
- spec: [email protected]
prefix: /usr
# This package is currently incomplete (no headers), but still works
krb5:
externals:
- spec: [email protected]
#prefix: /usr/lib/mit
prefix: /usr
libfuse:
externals:
- spec: [email protected]
prefix: /usr
- spec: [email protected]
prefix: /usr
libtirpc:
variants: ~gssapi
libtool:
externals:
- spec: [email protected]
prefix: /usr
# This package is currently incomplete (no headers), but still works
libxaw:
externals:
- spec: [email protected]
prefix: /usr
libxml2:
externals:
- spec: [email protected]
prefix: /usr
# This package is currently incomplete (no headers) and doesn't work
# for us. But it's only needed to build libxaw, for which we can use
# the existing (incomplete) installation in /usr, see above
#libxpm:
# externals:
# - spec: [email protected]
# prefix: /usr
lustre:
externals:
- spec: [email protected]_rc2_cray_113_g62287d0
prefix: /usr
m4:
externals:
- spec: [email protected]
prefix: /usr
ncurses:
externals:
- spec: [email protected]+termlib abi=6
prefix: /usr
openjdk:
externals:
- spec: [email protected]_8-suse-150000.3.83.1-x8664
prefix: /usr
openssh:
externals:
- spec: [email protected]
prefix: /usr
openssl:
externals:
- spec: [email protected]
prefix: /usr
perl:
externals:
- spec: [email protected]~cpanm+shared+threads
prefix: /usr
pkg-config:
buildable: false
externals:
- spec: [email protected]
prefix: /usr
qt:
externals:
- spec: [email protected]
prefix: /lustre/f2/scratch/Dom.Heinzeller/spack-stack-c5/qt-5.15.2/5.15.2/gcc_64
rdma-core:
externals:
- spec: [email protected]
prefix: /usr
rsync:
externals:
- spec: [email protected]
prefix: /usr
ruby:
externals:
- spec: [email protected]
prefix: /usr
sed:
externals:
- spec: [email protected]
prefix: /usr
slurm:
externals:
- spec: [email protected]
prefix: /usr
subversion:
externals:
- spec: [email protected]
prefix: /usr
tar:
externals:
- spec: [email protected]
prefix: /usr
texinfo:
externals:
- spec: [email protected]
prefix: /usr
wget:
externals:
- spec: [email protected]
prefix: /usr
which:
externals:
- spec: [email protected]
prefix: /usr
xz:
externals:
- spec: [email protected]
prefix: /usr
zip:
externals:
- spec: [email protected]
prefix: /usr
2 changes: 1 addition & 1 deletion configs/sites/narwhal/compilers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ compilers::
target: any
modules:
- PrgEnv-intel/8.3.2
- intel/2021.3.0
- intel-classic/2021.3.0
environment:
prepend_path:
PATH: '/opt/cray/pe/gcc/10.3.0/snos/bin'
Expand Down
31 changes: 22 additions & 9 deletions doc/source/Platforms.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ spack-stack-v1
+----------------------------------------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------------+
| NASA Discover GNU | Dom Heinzeller | ``/discover/swdev/jcsda/spack-stack/spack-stack-v1/envs/skylab-3.0.0-gnu-10.1.0/install`` |
+----------------------------------------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------------+
| NAVY HPCMP Narwhal | Dom Heinzeller | currently broken - will be repaired soon |
| NAVY HPCMP Narwhal | Dom Heinzeller | ``/p/app/projects/NEPTUNE/spack-stack/spack-stack-v1/envs/skylab-3.0.0-intel-2021.4.0/install`` |
+----------------------------------------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------------+
| NCAR-Wyoming Casper | Dom Heinzeller | ``/glade/work/jedipara/cheyenne/spack-stack/spack-stack-v1/envs/skylab-3.0.0-intel-19.1.1.217-casper/install`` |
+----------------------------------------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------------+
Expand All @@ -41,7 +41,9 @@ spack-stack-v1
+----------------------------------------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------------+
| NOAA Parallel Works (AWS, Azure, Gcloud) | | not yet supported - coming soon |
+----------------------------------------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------------+
| NOAA RDHPCS Gaea | Dom Heinzeller | ``/lustre/f2/pdata/esrl/gsd/spack-stack/spack-stack-v1/envs/skylab-3.0.0-intel-2021.3.0/install`` |
| NOAA RDHPCS Gaea (C3/C4) | Dom Heinzeller | ``/lustre/f2/pdata/esrl/gsd/spack-stack/spack-stack-v1/envs/skylab-3.0.0-intel-2021.3.0/install`` |
+----------------------------------------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------------+
| NOAA RDHPCS Gaea (C5) | Alex Richert / Dom Heinzeller | not yet supported - coming soon |
+----------------------------------------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------------+
| NOAA RDHPCS Hera Intel | Hang Lei / Dom Heinzeller | ``/scratch1/NCEPDEV/global/spack-stack/spack-stack-v1/envs/skylab-3.0.0-intel-2021.5.0/install`` |
+----------------------------------------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------------+
Expand Down Expand Up @@ -142,17 +144,14 @@ For ``spack-stack-1.2.0``/``skylab-3.0.0`` with GNU, load the following modules
NAVY HPCMP Narwhal
------------------------------

.. note::
``spack-stack-1.2.0``/``skylab-3.0.0`` (and previous ``spack-stack-1.1.0``/``skylab-2.0.0``) is broken and will be fixed soon.

The following is required for building new spack environments and for using spack to build and run software.

.. code-block:: console
module unload PrgEnv-cray
module load PrgEnv-intel/8.3.2
module unload intel
module load intel/2021.3.0
module load intel-classic/2021.4.0
module unload cray-mpich
module load cray-mpich/8.1.14
module unload cray-python
Expand All @@ -167,8 +166,8 @@ For ``spack-stack-1.2.0``/``skylab-3.0.0`` with Intel, load the following module

.. code-block:: console
module use /p/app/projects/NEPTUNE/spack-stack/spack-stack-v1/envs/skylab-3.0.0-intel-2021.3.0/install/modulefiles/Core
module load stack-intel/2021.3.0
module use /p/app/projects/NEPTUNE/spack-stack/spack-stack-v1/envs/skylab-3.0.0-intel-2021.4.0/install/modulefiles/Core
module load stack-intel/2021.4.0
module load stack-cray-mpich/8.1.14
module load stack-python/3.9.7
Expand Down Expand Up @@ -272,7 +271,7 @@ The following is required for building new spack environments and for using spac
.. _Platforms_Gaea:

------------------------------
NOAA RDHPCS Gaea
NOAA RDHPCS Gaea (C3/C4)
------------------------------

The following is required for building new spack environments and for using spack to build and run software. Don't use ``module purge`` on Gaea!
Expand Down Expand Up @@ -304,6 +303,20 @@ For ``spack-stack-1.2.0``/``skylab-3.0.0`` with Intel, load the following module
cmake -DCMAKE_CROSSCOMPILING_EMULATOR="/usr/bin/srun;-n;1" -DMPIEXEC_EXECUTABLE="/usr/bin/srun" -DMPIEXEC_NUMPROC_FLAG="-n" PATH_TO_SOURCE
------------------------------
NOAA RDHPCS Gaea (C5)
------------------------------

The following is required for building new spack environments and for using spack to build and run software. Don't use ``module purge`` on Gaea!

.. code-block:: console
module load PrgEnv-intel/8.3.3
module load intel/2022.0.2
module load cray-mpich/8.1.16
module load python/3.9.12
.. _Platforms_Hera:

------------------------------
Expand Down

0 comments on commit fab1fc5

Please sign in to comment.