Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Notebooks #656

Merged
merged 29 commits into from
Nov 20, 2024
Merged
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
24ab620
UW-627: Adding a test Jupyter notebook (#508)
Byrnetp Jun 13, 2024
d83399f
Merge branch 'main' into notebooks
maddenp-noaa Jul 10, 2024
429d5c1
UW-644 Use testbook to test the example Jupyter notebook (#528)
Byrnetp Jul 16, 2024
0fdd366
UW-645 Automate the tests for Jupyter notebooks using GitHub Actions …
Byrnetp Jul 22, 2024
2733a10
Updated readme with binder links (#535)
lexbuko22 Jul 26, 2024
f5977c9
Added binder link to user guide (#544)
lexbuko22 Jul 26, 2024
c81ede8
Merge branch 'main' into notebooks
maddenp-noaa Jul 26, 2024
765fe83
Adding compare() example (#547)
lexbuko22 Jul 26, 2024
e1b37fa
Move binder/environment.yml (#549)
maddenp-noaa Jul 29, 2024
452139a
Add formatting and linting
maddenp-noaa Aug 17, 2024
168e203
Merge branch 'main' into notebooks
maddenp-noaa Aug 17, 2024
2aadaa3
UW-651 Jupyter Notebook: Template Mode (#581)
Byrnetp Aug 20, 2024
a842fa3
notebooks update (#593)
maddenp-noaa Aug 26, 2024
c6d6040
UW-666 Jupyter Notebook: File System Tool (#598)
Byrnetp Sep 3, 2024
3561964
UW-667 Jupyter notebook: Config Tool Part I (#607)
Byrnetp Sep 25, 2024
2c75b30
Jupyter notebook: Config Tool Part II (#611)
Byrnetp Oct 17, 2024
04b802b
Jupyter Notebook: Rocoto Tool (#637)
Byrnetp Oct 31, 2024
e69e3dc
Jupyter Notebook: Configure Experiment Cookbook (#653)
Byrnetp Nov 19, 2024
127b18a
Update notebooks branch from main (#655)
maddenp-noaa Nov 19, 2024
03e9b5b
Merge branch 'main' into notebooks
maddenp-noaa Nov 19, 2024
098b257
Add missing newlines
maddenp-noaa Nov 19, 2024
6ab79eb
Add exec bit to test-nb.sh
maddenp-noaa Nov 19, 2024
fc00bac
Incorporate notebook tests into main CI tests
maddenp-noaa Nov 19, 2024
e7a3bff
Update notebooks/install-deps
maddenp-noaa Nov 19, 2024
89672cd
Update notebook output cells
maddenp-noaa Nov 19, 2024
2f7cac6
Test fixes
maddenp-noaa Nov 19, 2024
33a2879
Test fixes
maddenp-noaa Nov 19, 2024
a8f250c
Test fixes
maddenp-noaa Nov 19, 2024
c382d43
Merge branch 'notebooks' of github.com:ufs-community/uwtools into not…
maddenp-noaa Nov 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Jupyter Notebook: Configure Experiment Cookbook (#653)
* Add notebook, unit tests, Binder link

* Apply suggestions from code review

Co-authored-by: Christina Holt <56881914+christinaholtNOAA@users.noreply.github.com>

* Add user.PARMdir

* Apply suggestions from code review

Co-authored-by: Christina Holt <56881914+christinaholtNOAA@users.noreply.github.com>

* Update Binder links

* Add validation example using chgres_cube driver

* Move header to binder_links.rst

* Apply suggestions from review

* Apply suggestions from code review

Co-authored-by: Paul Madden <136389411+maddenp-noaa@users.noreply.github.com>

* notebook name change and driver validation update

---------

Co-authored-by: Christina Holt <56881914+christinaholtNOAA@users.noreply.github.com>
Co-authored-by: Paul Madden <136389411+maddenp-noaa@users.noreply.github.com>
3 people authored Nov 19, 2024
commit e69e3dc46a7707413088fe6de724a03798359ee3
2 changes: 2 additions & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
@@ -236,6 +236,8 @@ mpas
| **CLI**: ``uw mpas -h``
| **API**: ``import uwtools.api.mpas``

.. include:: /shared/binder_links.rst

------------------

**Disclaimer**
18 changes: 2 additions & 16 deletions docs/sections/user_guide/index.rst
Original file line number Diff line number Diff line change
@@ -7,20 +7,6 @@ User Guide
installation
cli/index
api/index

.. raw:: html

<ul>
<li>API Jupyter Notebooks
<ul>
<li><a href="https://mybinder.org/v2/gh/ufs-community/uwtools/notebooks?labpath=notebooks%2Fconfig.ipynb">Config Tool</a>
<li><a href="https://mybinder.org/v2/gh/ufs-community/uwtools/notebooks?labpath=notebooks%2Ffs.ipynb">File System Tool</a>
<li><a href="https://mybinder.org/v2/gh/ufs-community/uwtools/notebooks?labpath=notebooks%2Frocoto.ipynb">Rocoto Tool</a>
<li><a href="https://mybinder.org/v2/gh/ufs-community/uwtools/notebooks?labpath=notebooks%2Ftemplate.ipynb">Template Tool</a>
</ul>
</ul>

.. toctree::
:maxdepth: 2

yaml/index

.. include:: /shared/binder_links.rst
13 changes: 13 additions & 0 deletions docs/shared/binder_links.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
Jupyter Notebooks
-----------------

* Jupyter Notebook Tutorials (API)

* `Config Tool <https://mybinder.org/v2/gh/ufs-community/uwtools/notebooks?labpath=notebooks%2Fconfig.ipynb>`_
* `File System Tool <https://mybinder.org/v2/gh/ufs-community/uwtools/notebooks?labpath=notebooks%2Ffs.ipynb>`_
* `Rocoto Tool <https://mybinder.org/v2/gh/ufs-community/uwtools/notebooks?labpath=notebooks%2Frocoto.ipynb>`_
* `Template Tool <https://mybinder.org/v2/gh/ufs-community/uwtools/notebooks?labpath=notebooks%2Ftemplate.ipynb>`_

* Cookbooks

* `Configuring an Experiment with UW Tools <https://mybinder.org/v2/gh/ufs-community/uwtools/notebooks?labpath=notebooks%2Fexp-config-cb.ipynb>`_
401 changes: 401 additions & 0 deletions notebooks/exp-config-cb.ipynb

Large diffs are not rendered by default.

27 changes: 27 additions & 0 deletions notebooks/fixtures/exp-config/base-file.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
task_make_ics:
chgres_cube:
execution:
executable: "execdir/chgres_cube"
namelist:
update_values:
config:
cycle_day: !int "{{ cycle.strftime('%d') }}"
cycle_hour: !int "{{ cycle.strftime('%H') }}"
cycle_mon: !int "{{ cycle.strftime('%m') }}"
convert_atm: true
convert_nst: true
convert_sfc: true
data_dir_input_grid: "{{ task_make_ics.chgres_cube.rundir }}"
external_model: "GFS"
input_type: "gaussian_nemsio"
mosaic_file_target_grid: "path/to/example_mosaic.halo.nc"
tg3_from_soil: false
tracers:
- sphum
- liq_wat
tracers_input:
- spfh
- clwmr
varmap_file: "{{ user.PARMdir }}/ufs_utils/varmap_tables/GFSphys_var_map.txt"
vcoord_file_target_grid: "path/to/global_hyblev.165.txt"
rundir: '{{ workflow.EXPTDIR }}/make_ics'
6 changes: 6 additions & 0 deletions notebooks/fixtures/exp-config/fv3-rap-physics.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
task_make_ics:
chgres_cube:
namelist:
update_values:
config:
varmap_file: "{{ user.PARMdir }}/ufs_utils/varmap_tables/GSDphys_var_map.txt"
6 changes: 6 additions & 0 deletions notebooks/fixtures/exp-config/user.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
user:
ACCOUNT: zrtrr
MACHINE: hera
PARMdir: /path/to/ufs-srweather-app/parm
workflow:
EXPTDIR: /path/to/my/output
35 changes: 35 additions & 0 deletions notebooks/tests/test_exp_config_cb.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
from testbook import testbook
from uwtools.config.formats.yaml import YAMLConfig


def test_exp_config():
with open("fixtures/exp-config/base-file.yaml", "r", encoding="utf-8") as f:
base_cfg = f.read().rstrip()
with open("fixtures/exp-config/fv3-rap-physics.yaml", "r", encoding="utf-8") as f:
fv3_rap_phys = f.read().rstrip()
with open("fixtures/exp-config/user.yaml", "r", encoding="utf-8") as f:
user_cfg = f.read().rstrip()
with testbook("exp-config-cb.ipynb", execute=True) as tb:
assert tb.cell_output_text(1) == ""
assert tb.cell_output_text(3) == base_cfg
assert tb.cell_output_text(5) == fv3_rap_phys
assert tb.cell_output_text(7) == user_cfg
assert tb.cell_output_text(9) == str(YAMLConfig("fixtures/exp-config/base-file.yaml"))
updated_cfg = (
"cycle_day: !int '{{ cycle.strftime(''%d'') }}'",
"varmap_file: '{{ user.PARMdir }}/ufs_utils/varmap_tables/GSDphys_var_map.txt'",
"PARMdir: /path/to/ufs-srweather-app/parm",
)
assert all(x in tb.cell_output_text(11) for x in updated_cfg)
deref_cfg = (
"data_dir_input_grid: /path/to/my/output/make_ics",
"rundir: /path/to/my/output/make_ics",
)
assert all(x in tb.cell_output_text(13) for x in deref_cfg)
validate_out = (
"INFO Validating config against internal schema: chgres-cube",
"INFO 0 UW schema-validation errors found",
"INFO Validating config against internal schema: platform",
"chgres_cube valid schema: State: Ready",
)
assert all(x in tb.cell_output_text(15) for x in validate_out)