Skip to content

Commit

Permalink
Update requirements and fix test gallery failure on pandas pyarrow wa…
Browse files Browse the repository at this point in the history
…rning (#157)
  • Loading branch information
rly authored Jan 27, 2024
1 parent abf7a02 commit f5f09c8
Show file tree
Hide file tree
Showing 6 changed files with 48 additions and 35 deletions.
23 changes: 12 additions & 11 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

### Bug Fixes
* Fixed bug when converting HDF5 datasets with unlimited dimensions @oruebel [#155](https://github.com/hdmf-dev/hdmf-zarr/pull/155)
* Adjust gallery tests to not fail on deprecation warnings from pandas. @rly [#157](https://github.com/hdmf-dev/hdmf-zarr/pull/157)

## 0.5.0 (December 8, 2023)

Expand Down Expand Up @@ -40,11 +41,11 @@

### New Features
* Added support, tests, and docs for using ``DirectoryStore``, ``TempStore``, and
``NestedDirectoryStore`` Zarr storage backends with ``ZarrIO`` and ``NWBZarrIO``.
``NestedDirectoryStore`` Zarr storage backends with ``ZarrIO`` and ``NWBZarrIO``.
@oruebel [#62](https://github.com/hdmf-dev/hdmf-zarr/pull/62)

### Minor enhancements
* Updated handling of references on read to simplify future integration of file-based Zarr
* Updated handling of references on read to simplify future integration of file-based Zarr
stores (e.g., ZipStore or database stores). @oruebel [#62](https://github.com/hdmf-dev/hdmf-zarr/pull/62)
* Added ``can_read`` classmethod to ``ZarrIO``. @bendichter [#97](https://github.com/hdmf-dev/hdmf-zarr/pull/97)

Expand All @@ -53,7 +54,7 @@
@oruebel [#62](https://github.com/hdmf-dev/hdmf-zarr/pull/62)
* Fixed CI testing of minimum and optional installation requirement. @rly
[#99](https://github.com/hdmf-dev/hdmf-zarr/pull/99)
* Updated tests to handle upcoming changes to ``HDMFIO``. @rly
* Updated tests to handle upcoming changes to ``HDMFIO``. @rly
[#102](https://github.com/hdmf-dev/hdmf-zarr/pull/102)


Expand All @@ -74,25 +75,25 @@
links/reference when moving Zarr files @oruebel [#46](https://github.com/hdmf-dev/hdmf-zarr/pull/46)
* Fixed bugs in requirements defined in setup.py @oruebel [#46](https://github.com/hdmf-dev/hdmf-zarr/pull/46)
* Fixed bug regarding Sphinx external links @mavaylon1 [#53](https://github.com/hdmf-dev/hdmf-zarr/pull/53)
* Updated gallery tests to use test_gallery.py and necessary package dependencies
* Updated gallery tests to use test_gallery.py and necessary package dependencies
@mavaylon1 [#53](https://github.com/hdmf-dev/hdmf-zarr/pull/53)
* Updated dateset used in conversion tutorial, which caused warnings
* Updated dateset used in conversion tutorial, which caused warnings
@oruebel [#56](https://github.com/hdmf-dev/hdmf-zarr/pull/56)

### Docs
* Added tutorial illustrating how to create a new NWB file with NWBZarrIO
* Added tutorial illustrating how to create a new NWB file with NWBZarrIO
@oruebel [#46](https://github.com/hdmf-dev/hdmf-zarr/pull/46)
* Added docs for describing the mapping of HDMF schema to Zarr storage
* Added docs for describing the mapping of HDMF schema to Zarr storage
@oruebel [#48](https://github.com/hdmf-dev/hdmf-zarr/pull/48)
* Added ``docs/gallery/resources`` for storing local files used by the tutorial galleries
* Added ``docs/gallery/resources`` for storing local files used by the tutorial galleries
@oruebel [#61](https://github.com/hdmf-dev/hdmf-zarr/pull/61)
* Removed dependency on ``dandi`` library for data download in the conversion tutorial by storing the NWB files as
* Removed dependency on ``dandi`` library for data download in the conversion tutorial by storing the NWB files as
local resources @oruebel [#61](https://github.com/hdmf-dev/hdmf-zarr/pull/61)

## 0.1.0 (August 23, 2022)

### New features

* Created new optional Zarr-based I/O backend for writing files using Zarr's `zarr.store.DirectoryStore` backend,
including support for iterative write, chunking, compression, simple and compound data types, links, object
* Created new optional Zarr-based I/O backend for writing files using Zarr's `zarr.store.DirectoryStore` backend,
including support for iterative write, chunking, compression, simple and compound data types, links, object
references, namespace and spec I/O.
24 changes: 14 additions & 10 deletions requirements-dev.txt
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
# pinned dependencies to reproduce an entire development environment to use HDMF, run HDMF tests, check code style,
# compute coverage, and create test environments
coverage==6.4.2
flake8==5.0.4
flake8-debugger==4.1.2
flake8-print==5.0.0
pytest==7.1.2
pytest-cov==3.0.0
python-dateutil==2.8.2
tox==3.25.1
# pinned dependencies to reproduce an entire development environment to use HDMF-Zarr,
# run HDMF-Zarr tests, check code style,
# compute coverage, and create test environments. note that depending on the version of python installed, different
# versions of requirements may be installed due to package incompatibilities.
#
black==23.10.1
codespell==2.2.6
coverage==7.3.2
hdf5plugin==4.3.0 # hdf5plugin is used to test conversion of plugin filters
pre-commit==3.5.0
pytest==7.4.3
pytest-cov==4.1.0
python-dateutil==2.8.2
ruff==0.1.3
tox==4.11.3
2 changes: 1 addition & 1 deletion requirements-doc.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# dependencies to generate the documentation for HDMF
# dependencies to generate the documentation for HDMF-Zarr
matplotlib
sphinx>=4 # improved support for docutils>=0.17
sphinx_rtd_theme>=1 # <1 does not work with docutils>=0.17
Expand Down
6 changes: 3 additions & 3 deletions requirements-opt.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
tqdm==4.65.0
fsspec==2023.10.0
s3fs==2023.10.0
tqdm==4.66.1
fsspec==2023.12.2
s3fs==2023.12.2
8 changes: 4 additions & 4 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# pinned dependencies to reproduce an entire development environment to use HDMF-ZARR
hdmf==3.9.0
zarr==2.11.0
hdmf==3.12.0
zarr==2.16.1
pynwb==2.5.0
numpy==1.24.0
numcodecs==0.11.0
numpy==1.26.3
numcodecs==0.12.1
threadpoolctl==3.2.0
20 changes: 14 additions & 6 deletions test_gallery.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,26 +49,30 @@ def _import_from_file(script):
"length of electrodes. Your data may be transposed."
)

_deprication_warning_map = (
_deprecation_warning_map = (
'Classes in map.py should be imported from hdmf.build. Importing from hdmf.build.map will be removed '
'in HDMF 3.0.'
)

_deprication_warning_fmt_docval_args = (
_deprecation_warning_fmt_docval_args = (
"fmt_docval_args will be deprecated in a future version of HDMF. Instead of using fmt_docval_args, "
"call the function directly with the kwargs. Please note that fmt_docval_args "
"removes all arguments not accepted by the function's docval, so if you are passing kwargs that "
"includes extra arguments and the function's docval does not allow extra arguments (allow_extra=True "
"is set), then you will need to pop the extra arguments out of kwargs before calling the function."
)

_deprication_warning_call_docval_func = (
_deprecation_warning_call_docval_func = (
"call the function directly with the kwargs. Please note that call_docval_func "
"removes all arguments not accepted by the function's docval, so if you are passing kwargs that "
"includes extra arguments and the function's docval does not allow extra arguments (allow_extra=True "
"is set), then you will need to pop the extra arguments out of kwargs before calling the function."
)

_deprecation_warning_pandas_pyarrow_re = (
r"\nPyarrow will become a required dependency of pandas.*"
)


def run_gallery_tests():
global TOTAL, FAILURES, ERRORS
Expand Down Expand Up @@ -96,13 +100,13 @@ def run_gallery_tests():
try:
with warnings.catch_warnings(record=True):
warnings.filterwarnings(
"ignore", message=_deprication_warning_map, category=DeprecationWarning
"ignore", message=_deprecation_warning_map, category=DeprecationWarning
)
warnings.filterwarnings(
"ignore", message=_deprication_warning_fmt_docval_args, category=PendingDeprecationWarning
"ignore", message=_deprecation_warning_fmt_docval_args, category=PendingDeprecationWarning
)
warnings.filterwarnings(
"ignore", message=_deprication_warning_call_docval_func, category=PendingDeprecationWarning
"ignore", message=_deprecation_warning_call_docval_func, category=PendingDeprecationWarning
)
warnings.filterwarnings(
"ignore", message=_experimental_warning_re, category=UserWarning
Expand All @@ -127,6 +131,10 @@ def run_gallery_tests():
# this warning is triggered when downstream code such as pynwb uses pkg_resources>=5.13
"ignore", message=_pkg_resources_declare_warning_re, category=DeprecationWarning
)
warnings.filterwarnings(
# this warning is triggered from pandas
"ignore", message=_deprecation_warning_pandas_pyarrow_re, category=DeprecationWarning
)
_import_from_file(script_abs)
except Exception:
print(traceback.format_exc())
Expand Down

0 comments on commit f5f09c8

Please sign in to comment.