Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slight refactor to Vector Index get #1248

Merged
merged 9 commits into from
Feb 20, 2025
Merged

Slight refactor to Vector Index get #1248

merged 9 commits into from
Feb 20, 2025

Conversation

mavaylon1
Copy link
Contributor

@mavaylon1 mavaylon1 commented Feb 16, 2025

Motivation

hdmf-dev/hdmf-zarr#237
Using to_dataframe in Zarr is slower than HDF5, most likely due to how we handle references. The change is gathering all the data at once to resolve, rather than multiple I/O reads iteratively.

Checklist

  • Did you update CHANGELOG.md with your changes?
  • Does the PR clearly describe the problem and the solution?
  • Have you reviewed our Contributing Guide?
  • Does the PR use "Fix #XXX" notation to tell GitHub to close the relevant issue numbered XXX when the PR is merged?

Copy link

codecov bot commented Feb 16, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.56%. Comparing base (bcde246) to head (f68969c).
Report is 1 commits behind head on dev.

Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #1248      +/-   ##
==========================================
+ Coverage   91.55%   91.56%   +0.01%     
==========================================
  Files          42       42              
  Lines        9539     9552      +13     
  Branches     1926     1928       +2     
==========================================
+ Hits         8733     8746      +13     
  Misses        523      523              
  Partials      283      283              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@alejoe91
Copy link

@mavaylon1 tested again on my end an this dramatically speeds things up :)

@mavaylon1 mavaylon1 marked this pull request as ready for review February 19, 2025 18:57
@mavaylon1 mavaylon1 requested a review from rly February 19, 2025 18:57
@rly
Copy link
Contributor

rly commented Feb 19, 2025

This looks good to me, but see comment above

@mavaylon1 mavaylon1 merged commit 5858ee0 into dev Feb 20, 2025
27 checks passed
@mavaylon1 mavaylon1 deleted the opt_get branch February 20, 2025 15:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants