Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for building and using cylc environments #1448

Merged
merged 59 commits into from
Jan 14, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
9d0ab96
Unpin or update several packages in configs/common/packages.yaml
climbfuji Nov 14, 2024
4b59184
Update .gitmodules and submodule pointer for spack for code review an…
climbfuji Nov 17, 2024
4898371
Pin py-netcdf to version 1.7.1 with variant mpi enabled
climbfuji Nov 17, 2024
819a2f1
Merge branch 'feature/spst-upd-20241031-follow-up-mpiifx-mpiifort-sit…
climbfuji Nov 18, 2024
db66b49
Use different versions of cdo for older Intel classic compilers
climbfuji Nov 18, 2024
5a442d7
Also use [email protected] with apple-clang@14
climbfuji Nov 18, 2024
f63c7f0
Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…
climbfuji Nov 20, 2024
f9575af
Update submodule pointer for spack
climbfuji Nov 21, 2024
5cbd9f4
Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…
climbfuji Dec 2, 2024
8ee0c5b
Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…
climbfuji Dec 5, 2024
02ccc78
Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…
climbfuji Dec 6, 2024
6cc18d5
Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…
climbfuji Dec 6, 2024
46ad103
Merge branch 'develop' into feature/unpin_update_common_packages
climbfuji Dec 9, 2024
71aee0b
Merge branch 'develop' into feature/unpin_update_common_packages
climbfuji Dec 10, 2024
ee2d6a4
Merge branch 'develop' into feature/unpin_update_common_packages
climbfuji Dec 11, 2024
b51652d
Update .gitmodules and submodule pointer for spack for code review an…
climbfuji Dec 11, 2024
ebe83fd
Configure variant cylc for neptune-python-env, enable in neptune-dev …
climbfuji Dec 11, 2024
11ca0a2
Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…
climbfuji Dec 11, 2024
fa21d33
Merge branch 'develop' into feature/unpin_update_common_packages
climbfuji Dec 13, 2024
9b2757c
Remove hardcoded version of py-numpy from all tier-1 site configs
climbfuji Dec 13, 2024
2411498
Bug fixes for S4 site config: correct GNU backend for Intel, no exter…
climbfuji Dec 14, 2024
6c61e0f
Remove 'prefer' for cdo in configs/common/packages.yaml, since it cla…
climbfuji Dec 16, 2024
705f80f
Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…
climbfuji Dec 18, 2024
520a0b6
Remove commented-out package entries from configs/common/packages.yam…
climbfuji Dec 18, 2024
eef5c0b
Merge branch 'feature/unpin_update_common_packages' of https://github…
climbfuji Dec 18, 2024
862e490
Merge upstream branch
climbfuji Dec 18, 2024
da8dfd4
Fix typo in configs/common/packages.yaml
climbfuji Dec 18, 2024
02cc2d5
Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…
climbfuji Dec 19, 2024
c113381
Remove '+excel' from py-pandas dependency in spack-ext/repos/spack-st…
climbfuji Dec 19, 2024
f7990ce
Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…
climbfuji Dec 19, 2024
3f1fd93
In configs/common/packages.yaml, pin py-numpy to 1.26 to avoid duplic…
climbfuji Dec 19, 2024
dadb7b1
Update submodule pointer for spack
climbfuji Dec 20, 2024
b7b81b7
Update submodule pointer for spack
climbfuji Dec 21, 2024
040366e
Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…
climbfuji Dec 21, 2024
be1b69c
Add configs/templates/cylc-dev
climbfuji Dec 26, 2024
329f253
Update git and git-lfs packages on Atlantis
climbfuji Dec 26, 2024
dd2173c
Update configs/templates/cylc-dev/spack.yaml: new versions of py-cylc…
climbfuji Dec 26, 2024
d2726d2
Add cylc-dev to templates for buildcache builds in .github/workflows/…
climbfuji Dec 26, 2024
f8f60b9
Update submodule pointer for spack
climbfuji Dec 26, 2024
e78b289
Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…
climbfuji Dec 26, 2024
50afc84
Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…
climbfuji Jan 7, 2025
1048dc5
Update submodule pointer for spack
climbfuji Jan 8, 2025
ede071b
Revert adding cylc variant to neptune-env
climbfuji Jan 8, 2025
7a3b460
[skip ci] Debug cylc-dev build in .github/workflows/ubuntu-ci-x86_64-…
climbfuji Jan 8, 2025
4f0f463
Exclude old/deprecated [email protected] from 'spack external find' on Ubunt…
climbfuji Jan 8, 2025
9e180cc
Update submodule pointer for spack
climbfuji Jan 8, 2025
0c87803
Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…
climbfuji Jan 8, 2025
772bb08
Replace bad 'set +e' with 'set -e' in all self-hosted runner workflows
climbfuji Jan 8, 2025
f120565
Revert .gitmodules and update submodule pointer for spack
climbfuji Jan 9, 2025
28b0fae
Revert temporary changes in .github/workflows/ubuntu-ci-x86_64-gnu.yaml
climbfuji Jan 9, 2025
9f8a7f5
Update .gitmodules and submodule pointer for spack for code review an…
climbfuji Jan 9, 2025
a5904ba
In configs/templates/cylc-dev/spack.yaml: rename view; path is relati…
climbfuji Jan 9, 2025
3f79cfc
configs/common/packages_{intel,oneapi}.yaml: Build qt with gcc for In…
climbfuji Jan 10, 2025
9a443a6
Revert .gitmodules and update submodule pointer for spack
climbfuji Jan 10, 2025
ab2e628
[skip ci] Update configs/sites/tier1/narwhal/compilers.yaml: add comm…
climbfuji Jan 10, 2025
6b53f10
[skip ci] Update comment about cylc-wrapper in configs/templates/cylc…
climbfuji Jan 10, 2025
c396d68
Merge branch 'develop' of https://github.com/jcsda/spack-stack into f…
climbfuji Jan 10, 2025
d04d22c
Update documentation for building cylc environments
climbfuji Jan 13, 2025
d3b140f
Clean up new documentation in doc/source/PreConfiguredSites.rst
climbfuji Jan 13, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion configs/sites/tier1/narwhal/compilers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ compilers::
cxx: CC
f77: ftn
fc: ftn
# For cylc-dev, can't use Cray wrappers
# For cylc-dev, can't use Cray wrappers (https://github.com/spack/spack/issues/48515)
#cc: /opt/cray/pe/gcc/10.3.0/snos/bin/gcc
#cxx: /opt/cray/pe/gcc/10.3.0/snos/bin/g++
#f77: /opt/cray/pe/gcc/10.3.0/snos/bin/gfortran
Expand Down
34 changes: 32 additions & 2 deletions doc/source/PreConfiguredSites.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,14 @@ The following naming conventions are used on all fully-supported (tier 1) sites.
+----------------------------------+---------------------------------------------------------+-------------------+------------------------------+
| ``neptune-dev`` | NEPTUNE standalone environment (with xNRL Python) | ``ne`` | ``ne-oneapi-2024.2.1`` |
+----------------------------------+---------------------------------------------------------+-------------------+------------------------------+
| ``cylc-dev`` | Environment for running cylc (separate from other envs) | ``ce`` | ``ce-gcc-10.3.0`` |
+----------------------------------+---------------------------------------------------------+-------------------+------------------------------+
| ``gsi-addon-dev`` | GSI addon (chained) environment on top of unified env. | ``gsi`` | ``gsi-gcc-13.3.0`` |
+----------------------------------+---------------------------------------------------------+-------------------+------------------------------+
| ``unified-dev`` with new ESMF | Unified environment with new ESMF (chained from ``ue``) | ``ue-esmf870b99`` | ``ue-esmf870b99-aocc-4.2.0`` |
+----------------------------------+---------------------------------------------------------+-------------------+------------------------------+

To support users who consistently want the latest release, on NOAA RDHPCS tier 1 platforms, soft links pointing to the modulefiles associated with the latest release of the Unified Environment are provided under the main spack-stack directory. The usage consists of ``module use /path/to/spack-stack/latest-ue-<compiler>``, and then loading the spack-stack meta-modules as usual. These soft links should be updated when each release is finalized.

.. _Preconfigured_Sites_Tier1:

Expand Down Expand Up @@ -562,6 +565,8 @@ Create local environment

The following instructions install a new spack environment on a pre-configured site. Instructions for creating a new site config on a configurable system (i.e. a generic Linux or macOS system) can be found in :numref:`Section %s <NewSiteConfigs>`. The options for the ``spack stack`` extension are explained in :numref:`Section %s <SpackStackExtension>`.

The following instructions apply to the basic environments (``unified-dev``, ``skylab-dev``, ``neptune-dev``). Add-on (i.e. chained) environments and the ``cylc`` environment require special instructions (see below).

.. code-block:: console

git clone --recurse-submodules https://github.com/jcsda/spack-stack.git
Expand Down Expand Up @@ -614,8 +619,6 @@ The following instructions install a new spack environment on a pre-configured s
# Check permissions for systems where non-owning users/groups need access
${SPACK_STACK_DIR}/util/check_permissions.sh

To support users who consistently want the latest release, on NOAA RDHPCS tier 1 platforms, soft links pointing to the modulefiles associated with the latest release of the Unified Environment are provided under the main spack-stack directory. The usage consists of ``module use /path/to/spack-stack/latest-ue-<compiler>``, and then loading the spack-stack meta-modules as usual. These soft links should be updated when each release is finalized.

.. note::
You may want to capture the output from :code:`spack concretize` and :code:`spack install` comands in log files.
For example:
Expand All @@ -625,6 +628,33 @@ To support users who consistently want the latest release, on NOAA RDHPCS tier 1
spack concretize 2>&1 | tee log.concretize
spack install [--verbose] [--fail-fast] 2>&1 | tee log.install

For installing chained environments, the user is referred to section :numref:`%s <Add_Test_Packages>`.

The ``cylc`` environment is another special environment. This environment is not chained, thus the instructions provided above until and including the ``spack install`` are still valid. The following differences apply:

1. The ``cylc`` environment can only be built with a reasonably recent version of the GNU compilers (``gcc`` version ``10`` or later, as long as spack-stack in general supports the version; see section :numref:`%s <NewSiteConfigs>` for a compiler compatibility matrix). Further, on Cray systems, the Cray compiler wrappers can not be used (see the comment in ``configs/sites/tier1/narwhal/compilers.yaml``). Note that, as long as the ``cylc`` environment does not use MPI (which is currently the case), it is not necessary to toggle the ``wrappers`` variant for the external ``cray-mpich`` package when using the native compilers without the Cray wrappers.

2. Instead of creating ``tcl`` or ``lmod`` modules and the associated meta-modules, this environment creates a spack environment view in ``/path/to/spack-stack/envs/env-name/view``. In order to use the ``cylc`` installation provided in that view, the user is advised to create a ``cylc-wrapper`` similar to the following bash script, and then create an alias ``cylc`` pointing to the wrapper script. This approach ensures that the environment in which ``cylc`` operates is encapsulated from the environment that users or ``cylc`` tasks operate in.

.. code-block:: bash

> cat cylc-wrapper

#!/bin/bash

# Define environment variable CYLC_INSTALL_PREFIX here,
# pointing to /path/to/spack-stack/envs/env-name/view,
# or make sure the user did set it before calling the wrapper
[ -z ${CYLC_INSTALL_PREFIX} ] && echo "CYLC_INSTALL_PREFIX not set!" && exit 1

PATH=${CYLC_INSTALL_PREFIX}/bin:$PATH
unset PYTHONPATH
cylc "$@"

> alias

alias cylc='/path/to/cylc-wrapper'


.. _Preconfigured_Sites_ExtendingEnvironments:

Expand Down
Loading