End-to-end Auto Mesh Gen

[felicio93]

General Plan:

The plan is to create an fully automated mesh generation tool for the land-ocean continuum that takes advantage of SCHISM's flexibility. That encompasses the ability of creating individual meshes for the floodplain, river, and ocean domains, following their specific mesh generation requirements, as well as emerging them accordingly.

Development Team: @felicio93, @SorooshMani-NOAA, @feiye-vims, @hyungjuyoo, @saeed-moghimi-noaa, @josephzhang8,

Software Development

• River Mesh

  • https://github.com/noaa-ocs-modeling/SurgeTeamCoordination/issues/504
  • We need to decide what the new river mesh iteration will be

• Ocean Mesh

  • Ocean side mesh: Eddy regime / non-eddy / Transition Ocean Mesh #149
  • Ocean side mesh: around islands #150

• FloodPlain Mesh

  • Reproduce the STOFS overland mesh criteria (500m everywhere, 1000m at the coastline

• Mesh Merging and Refinement

  • https://github.com/noaa-ocs-modeling/SurgeTeamCoordination/issues/460
  • Weird patches of very small cells #100
  • Contour extraction error due to matplotlib #144
  • https://github.com/noaa-ocs-modeling/SurgeTeamCoordination/issues/492

• Vertical Grid

  • Automate v-grid generation #147
  • Add the capability of passing multiple theta_b theta_f and hc

• Mesh Metadata and Storage

  • https://github.com/noaa-ocs-modeling/SurgeTeamCoordination/issues/520

Testing

• First Iteration:
  https://github.com/SorooshMani-NOAA/river-in-mesh/issues/3

Documentation

• Abstracts

  • https://github.com/noaa-ocs-modeling/SurgeTeamCoordination/issues/516

• Papers

  • https://github.com/noaa-ocs-modeling/SurgeTeamCoordination/issues/521

• Technical Reports

---

Development Iterations

We organize the development in 2 months iterations for which we define a Minimum Viable Product. Each iterations is a separate ticket. Feel free to add your suggestions/priorities. The Iteration 00 was completed with the successful validation of our first complete 2D mesh: https://github.com/SorooshMani-NOAA/river-in-mesh/issues/3

• Iteration 01: 07/15/2024 - 09/13/2024
  End-to-end Auto Mesh Gen - Iteration 01 #166

• Iteration 02: 09/16/2024 - 11/15/2024
  End-to-end Auto Mesh Gen - Iteration 02 #167

• Iteration 03: 01/16/2024 - 01/17/2025
  TBD

[josephzhang8]

Thx @felicio93. I'd highly encourage you to write this up for Ocean Modelling Special Issue. It's highly relevant. Thx

[SorooshMani-NOAA]

@felicio93 the generic OCSMesh focused tasks I had in mind are:

• Parallelization of size function calculation (collector type). The issue is that hfun object is not picklable due to reference to files object
• Optimizing operations (e.g. contour calculation that we discussed)
• Subsetting an existing mesh with remeshing the boundaries to match the subsetting shape
• Parallelizing using Dask; right now the parallelization is limited to a single node.

[hyungjuyoo]

Hi, I am Hyung Ju, Yoo (post-doc) from Joseph's group.

Thank you for inviting me.

[felicio93]

tagging @janahaddad for reference

[pbranson]

Chiming in here, I work at CSIRO in Australia and have been testing out OCS mesh these past few weeks, with mostly positive results - thanks for your work on this!

@felicio93 the generic OCSMesh focused tasks I had in mind are:

• Parallelization of size function calculation (collector type). The issue is that hfun object is not picklable due to reference to files object
• Parallelizing using Dask; right now the parallelization is limited to a single node.

Can I suggest that you take a look at using FSSpec for file handling, it allows for a uniform interface between POSIX and cloud storage, are generally lazy locking, which allows for pickling of python file-like objects that are also supported by rasterio: see rasterio/rasterio#2905 These aspects also help with dask

On the dask front, you could consider using https://corteva.github.io/rioxarray/html/rioxarray.html to interface with rasterio which provides an xarray container overlay on rasterio that can leverage all of xarray+dask computational graph generation. It would likely be some effort to transition away from the current "windows" approach, so you could perhaps maintain that for multi-threaded processing within larger dask-chunks - this kind of approach can actually be beneficial as it reduces the overall size of the dask graph which can become an overhead for very large datasets.