Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataset from zarr #1071

Merged
merged 2 commits into from
May 22, 2024
Merged

Dataset from zarr #1071

merged 2 commits into from
May 22, 2024

Conversation

ocefpaf
Copy link
Member

@ocefpaf ocefpaf commented May 21, 2024

Closes #884 and #672.

@benjwadams I had to squash the original 17 commits to make this manageable but I believe I got everything in.

PS: This implements checker for nczarr! Not pure zarr datasets. The latter would be easier to implement when we move away from netCDF files as the base for compliance checking.

Copy link

codecov bot commented May 21, 2024

Codecov Report

Attention: Patch coverage is 60.60606% with 26 lines in your changes are missing coverage. Please review.

Project coverage is 81.75%. Comparing base (6a006e4) to head (7704919).
Report is 1 commits behind head on develop.

Files Patch % Lines
compliance_checker/protocols/zarr.py 71.79% 6 Missing and 5 partials ⚠️
compliance_checker/protocols/netcdf.py 30.76% 4 Missing and 5 partials ⚠️
compliance_checker/protocols/opendap.py 50.00% 3 Missing and 1 partial ⚠️
compliance_checker/suite.py 66.66% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1071      +/-   ##
===========================================
- Coverage    81.92%   81.75%   -0.17%     
===========================================
  Files           24       25       +1     
  Lines         5172     5224      +52     
  Branches      1237     1253      +16     
===========================================
+ Hits          4237     4271      +34     
- Misses         634      644      +10     
- Partials       301      309       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ocefpaf ocefpaf force-pushed the zarrSupport_take_2 branch 2 times, most recently from c7e0836 to a504802 Compare May 21, 2024 19:54
@ocefpaf
Copy link
Member Author

ocefpaf commented May 21, 2024

@benjwadams only 1 test is failing, TestCLI.test_nczarr_pass_through[local_file], with:

                                     Errors                                     
--------------------------------------------------------------------------------
§2.1 Filename
* Dataset path file:///home/filipe/Dropbox/pymodules/01-forks/IOOS/compliance-checker/compliance_checker/tests/data/trajectory.zarr#mode=nczarr,file must end with ".nc"

That is something we need to skip b/c zarr files are, in a way, against that rule.

BTW, I could not make the skip option work. Not sure that is going on...

I skipped two tests: check_filename b/c .zarr won't pass a name test for .nc, and check_coordinate_variables_strict_monotonicity b/c I believe that openSourcerer9000 original dataset has a real issue that we need to fix to run the tests.

With that said, this PR works 🎉 and is read for a review!

@ocefpaf ocefpaf requested a review from benjwadams May 22, 2024 07:40
@ocefpaf ocefpaf force-pushed the zarrSupport_take_2 branch 3 times, most recently from ec2c618 to 06b1820 Compare May 22, 2024 10:00
@ocefpaf ocefpaf force-pushed the zarrSupport_take_2 branch from 06b1820 to 7704919 Compare May 22, 2024 10:27
@benjwadams benjwadams merged commit b67b1a5 into ioos:develop May 22, 2024
25 of 27 checks passed
@ocefpaf ocefpaf deleted the zarrSupport_take_2 branch May 22, 2024 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants