Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

let fre cmor run process tripolar ocean data #303

Open
wants to merge 37 commits into
base: main
Choose a base branch
from
Open

let fre cmor run process tripolar ocean data #303

wants to merge 37 commits into from

Conversation

ilaflott
Copy link
Member

@ilaflott ilaflott commented Dec 24, 2024 •
edited
Loading

this PR will ideally enable fre cmor run to process ocean data on a native tripolar grid. also will do some general clean up here and there. relevant background: #268 PCMDI/cmor#766 and PCMDI/cmor#767

Describe your changes

  • remove GitPython pip-only dependency that can only be fulfilled by terrible habits
  • add PCMDI/cmip6-cmor-tables as submodule dependency
  • set minimum python version to 3.9 minimum
  • remove redundant click context forwarding in fre cmor
  • re-set os.environ['HOME'] back to inital value in test_fre_make_cli.py
  • puts meta.yaml source: url: <this repo's URL> back to source: path: . in light of pipeline-error-induced insanity.
  • pylint edits to address scary-ish no-value-for-parameter errors revealed by adjusting click scaffolding
  • NotImplemented errors for fre functions that do not yet have any real guts to them in fre-cli
  • get rid of sys.exit where possible
  • ... more to come ... work-in-progress ...

Issue ticket number and link (if applicable)

will close #298 (ideally)

Checklist before requesting a review

  • I ran my code
  • I tried to make my code readable
  • I tried to comment my code
  • I wrote a new test, if applicable
  • I wrote new instructions/documentation, if applicable
  • I ran pytest and inspected it's output
  • I ran pylint and attempted to implement some of it's feedback

@ilaflott
add in cmip6-cmor-tables as a submodule, remove un-maintained gitpyth…
6dce8ef 
…on pip-only dependency... nasty tech debt i should get rid of before its too late!
@ilaflott ilaflott added bug Something isn't working new functioning New feature or request tricky Likely to encounter significant friction to solution priority: HIGH labels Dec 24, 2024
@ilaflott ilaflott self-assigned this Dec 24, 2024
@ilaflott ilaflott linked an issue Dec 24, 2024 that may be closed by this pull request
CMORmeal: successfully process ocean variables on tripolar grid with cmor #298
Open
9 tasks
@ilaflott
Copy link
Member Author

so asking conda/mamba to use python3.10 or greater is surprisingly an issue. Dinking around with this locally, the first thing it complains about is cylc-uiserver. Given that this has yet-to-be-utilized in the actual repository, I comment it out.

The next issue is catalogbuilder... actually, really, a requirement of catalogbuilder's. intake-esm==2023.7.7, which seems to desire a python of 3.9 or less, but 3.9 is nearing EOL.

Given that this quick python version bump was supposed to be a quick-sidequest, i'll be restoring the dependencies and barking up that tree some other time.

@ilaflott
restore deps with the exception of the pip-installed gitpython and le…
ff8d8ce 
…aving cmor>=3.9.0 because i know we should not bother with anything less than 3.9.0 for cmor
@ilaflott
syntax error
32abb9b
@ilaflott
Copy link
Member Author

ilaflott commented Dec 24, 2024 •
edited
Loading

every now and then, a random conda build fails... here, it claims import git is causing the failure. I did remove that module, but i also removed that line

edit: same failure, same job. sanity demanding i ignore the conda_build workflow for a minute apparently

edit2: same failure, different job. xmas nightmare! moving on...

ilaflott and others added 5 commits December 24, 2024 13:38
@ilaflott ilaflott mentioned this pull request Jan 3, 2025
Merged
4 tasks
ilaflott and others added 10 commits January 6, 2025 10:48
@ilaflott
Merge branch 'main' into 298-cmor-process-ocean-tripolar-grid
0c0b45d
@ilaflott
remove a sys.exit or two. restore os.environ[HOME] back to what it wa…
fe0a656 
…s pre- fre make cli tests.
@ilaflott
Merge branch '298-cmor-process-ocean-tripolar-grid' of https://github…
9122ef3 
….com/noaa-gfdl/fre-cli into 298-cmor-process-ocean-tripolar-grid
@ilaflott
oops
3260539
@ilaflott
update click scaffolding to be much more lean and mean. approach cour…
50498e2 
…tesy of Ray M., thank you ray.
@ilaflott
removing now-redundant click-scaffoldding from fre app, fre check, fr…
c92e4e8 
…e list, fre pp, fre run, and fre test
@ilaflott
fix an oops, apply same approach to fre/catalog
58c83c1
@ilaflott
apply click-redundancy-removal logic to fre make
3e0096e
@ilaflott
Revert "fix an oops, apply same approach to fre/catalog"
19bc7a1 
This reverts commit 58c83c1.
@ilaflott
remove needless shebang, put fre catalog click scaffolding back where…
aef76ee 
… it was. catalogbuilder may need context forwarding.
ilaflott added 14 commits January 6, 2025 16:32
@ilaflott
fix the silly colon-oops in fre.pp again
e03e098
@ilaflott
fix another syntax oops. change sys.exit to return
e832807
@ilaflott
more getting tests etc to clean up after themselves
1a01de9
@ilaflott
#no more shebangs! (getit?)
ada0dc7
@ilaflott
edits addressing pylint complains about no value for arguments in calls
ab2708e
@ilaflott
further edits addressing pylint complains about no value for argument…
bc4a39b 
…s in calls
@ilaflott
notimplementederrors raised on functions not-yet-used. uncomment the …
67b67ee 
…help function test in meta.yaml because we DO want those to work
@ilaflott
remove useless __init__ files in test directories. more pylint edits.
a074922
@ilaflott
why last failure? unconvinced. fail again, conda_build, i dare you.
fb62498
@ilaflott
adjust import statement in fre/cmor/tests
21e22b4
@ilaflott
carefully tidy print statements, carefully changing order of certain …
54e02e3 
…things
@ilaflott
new cmor_helpers file holds helper functions
d6013a6 
cmor_mixer now creates xh and yh bnds. this was not supposed to take so long... why? see note-to-self...

...note to self, if you need to write data to a netcdf file... don't use [x][y][z] style indexing. you HAVE to use [x,y,z] for the assignment or the assignment fails.
@ilaflott
SO CLOSE
5911351
@ilaflott
pretty sure i got it!
dfe9ccb
@ilaflott ilaflott marked this pull request as ready for review January 9, 2025 23:27
@ilaflott ilaflott mentioned this pull request Jan 9, 2025
Open
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@ilaflott ilaflott

Labels
bug Something isn't working new functioning New feature or request priority: HIGH tricky Likely to encounter significant friction to solution
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

CMORmeal: successfully process ocean variables on tripolar grid with cmor
1 participant
@ilaflott