Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for geos-gcm to spack-stack (add mepo, geos-gcm-env; update esmf/mapl variants) #953

Merged
merged 84 commits into from
Jan 18, 2024

Conversation

climbfuji
Copy link
Collaborator

@climbfuji climbfuji commented Jan 16, 2024

Summary

This PR updates the spack submodule pointer for the changes described in JCSDA/spack#372: add package mepo and meta-package geos-gcm-env. The PR also re-enables the shared variant for esmf, see additional information below as to why that is. Finally, it adds geos-gcm-env to skylab-dev and unified-dev.

Update 2024-01-17:

  1. A bug fix PR for our spack fork is needed to fix shared ESMF build errors on macOS (see Building shared ESMF on macOS fails in linker stage #956): Bug fix for building shared ESMF on macOS, bug fix for building pflogger on Ubuntu with Intel (the latter is still WIP) spack#393
  2. We need to turn on the mpi variant for pflogger at least on the Ubuntu AWS ParallelCluster systems for the reasons described in Installation issue: pflogger: MPI is picked up and enabled when variant is set to false spack/spack#42137 (see also Should pflogger have an option for enabling/disabling MPI Goddard-Fortran-Ecosystem/pFlogger#106).

Additional information

  • GEOS builds its own version of mapl (by default, shared libraries) and therefore we don't need to load the mapl module for now. However, to facilitate a future switch to spack-stack mapl, we update the mapl config accordingly.
  • esmf and mapl are ... interesting. If esmf is static, then mapl needs to be static. If esmf is shared, then mapl needs to be shared. Violate these requirements and you'll be hit with double-free corruption segmentation faults when an executable unwinds its stack upon exit. See Add mepo and geos-gcm-env spack#372 and Possible missing MPI_Type_free in ESMCI_VMKernel? esmf-org/esmf#209 for more information.
  • Historically, the UFS wants everything to be static, so this was the configuration in spack-stack. But GEOS wants a shared esmf to support its shared mapl, and there is really no reason why we should restrict the esmf build to static only because of the UFS. If the UFS requires a static esmf, but its build system uses the shared version when both are available, then the UFS needs to be fixed (not spack-stack trimmed down to static esmf).

Testing

  • Successfully built geos-gcm-env on macOS with apple-clang, on AWS ParallelCluster with intel and gnu, and on Discover with intel and gnu
  • Build geos-gcm (branch geos-cf) on Discover and ran a one-day c12 end-to-end experiment (thanks to @mathomp4 for all his help)
  • CI

Applications affected

GEOS and UFS applications (the latter because esmf is now also available as shared library)

Systems affected

All

Dependencies

Issue(s) addressed

Finally! Resolves #242

Checklist

  • This PR addresses one issue/problem/enhancement, or has a very good reason for not doing so.
  • These changes have been tested on the affected systems and applications.
  • All dependency PRs/issues have been resolved and this PR can be merged.

@@ -142,7 +142,7 @@
# https://github.com/JCSDA/spack-stack/issues/769
# also: ... extdata2g segfault UFS?
version: ['2.40.3']
variants: ~shared ~extdata2g ~pflogger
variants: +shared +pflogger ~f2py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a note that I think these are all the defaults of mapl. But probably good to keep them explicit until people get used to it. :)

@climbfuji
Copy link
Collaborator Author

Thanks for the review @mathomp4 . Now I have to figure out the CI build failures (pflogger on ubuntu with intel, esmf static on macOS).

@climbfuji climbfuji force-pushed the feature/geos_gcm_dependencies branch from 46d609d to 5e26435 Compare January 18, 2024 03:30
@climbfuji climbfuji marked this pull request as ready for review January 18, 2024 16:29
@climbfuji climbfuji enabled auto-merge (squash) January 18, 2024 19:00
@climbfuji climbfuji merged commit b59ed4d into JCSDA:develop Jan 18, 2024
7 checks passed
@climbfuji climbfuji deleted the feature/geos_gcm_dependencies branch January 18, 2024 19:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
INFRA JEDI Infrastructure NOAA-EMC OAR-EPIC NOAA Oceanic and Atmospheric Research and Earth Prediction Innovation Center
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

Additional packages requested for building NASA GEOS model
4 participants