Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize Query::between for integers and timestamps #7785

Merged
merged 4 commits into from
Jun 11, 2024
Merged

Conversation

jedelbo
Copy link
Contributor

@jedelbo jedelbo commented Jun 6, 2024

What, How & Why?

☑️ ToDos

  • 📝 Changelog update
  • 🚦 Tests (or not relevant)
  • C-API, if public C++ API changed
  • bindgen/spec.yml, if public C++ API changed

@jedelbo jedelbo requested a review from ironage June 6, 2024 15:54
@cla-bot cla-bot bot added the cla: yes label Jun 6, 2024
Copy link
Contributor

@ironage ironage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. Check the compile errors though.

src/realm/array_integer.cpp Show resolved Hide resolved
src/realm/array_timestamp.cpp Show resolved Hide resolved
@@ -406,6 +406,64 @@ class IntegerNodeBase : public ColumnNodeBase {
};


template <class LeafType>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I've been wanting to do this as well. A great improvement for time series data.

@jedelbo jedelbo force-pushed the je/query-between branch from ea39d92 to 37e34df Compare June 7, 2024 09:32
Copy link

coveralls-official bot commented Jun 7, 2024

Pull Request Test Coverage Report for Build jorgen.edelbo_289

Details

  • 122 of 123 (99.19%) changed or added relevant lines in 9 files are covered.
  • 1260 unchanged lines in 42 files lost coverage.
  • Overall coverage decreased (-0.07%) to 90.891%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/realm/query_engine.hpp 29 30 96.67%
Files with Coverage Reduction New Missed Lines %
src/realm/array_mixed.cpp 1 91.91%
src/realm/index_string.hpp 1 93.48%
src/realm/sync/network/http.hpp 1 83.25%
src/realm/sync/network/websocket.cpp 1 71.79%
src/realm/util/flat_map.hpp 1 98.89%
test/test_compaction.cpp 1 98.35%
test/test_dictionary.cpp 1 99.83%
src/realm/array_integer_tpl.hpp 2 72.6%
src/realm/node_header.hpp 2 92.86%
test/object-store/sync/sync_manager.cpp 2 98.77%
Totals Coverage Status
Change from base Build 2397: -0.07%
Covered Lines: 214986
Relevant Lines: 236531

💛 - Coveralls

@jedelbo jedelbo requested a review from ironage June 7, 2024 11:17
Copy link
Contributor

@ironage ironage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you post the speedup of range queries? I think there is an existing benchmark and it would be easy to add one for timestamps as well because it is templated.
Add a changelog entry and this should be good to go! 👍

@jedelbo
Copy link
Contributor Author

jedelbo commented Jun 10, 2024

Performance improvement:

Req runs:   21  QueryRange<int><NonNullable><NonIndexed> (MemOnly, EncryptionOff):     min  22.20ms (-69.89%)           max  23.61ms (-68.61%)           med  22.41ms (-69.67%)           avg  22.62ms (-69.53%)           stddev   391us (-36.07%)      

Req runs:   13  QueryRange<timestamp><NonNullable><NonIndexed> (MemOnly, EncryptionOff):     min  33.68ms (-38.32%)           max  34.95ms (-38.67%)           med  34.33ms (-38.02%)           avg  34.26ms (-38.35%)           stddev   389us (-48.21%)      

Copy link

coveralls-official bot commented Jun 10, 2024

Pull Request Test Coverage Report for Build jorgen.edelbo_292

Details

  • 122 of 123 (99.19%) changed or added relevant lines in 9 files are covered.
  • 76 unchanged lines in 15 files lost coverage.
  • Overall coverage decreased (-0.008%) to 90.967%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/realm/query_engine.hpp 29 30 96.67%
Files with Coverage Reduction New Missed Lines %
src/realm/util/file.cpp 1 84.84%
test/fuzz_tester.hpp 1 57.73%
test/test_dictionary.cpp 1 99.83%
test/test_shared.cpp 1 96.73%
src/realm/array_blobs_big.cpp 2 98.58%
src/realm/cluster.cpp 2 75.6%
src/realm/mixed.cpp 2 86.46%
src/realm/sync/instruction_applier.cpp 3 68.01%
src/realm/unicode.cpp 3 83.83%
src/realm/bplustree.cpp 6 72.55%
Totals Coverage Status
Change from base Build 2405: -0.008%
Covered Lines: 214704
Relevant Lines: 236023

💛 - Coveralls

@nicola-cab nicola-cab mentioned this pull request Jun 10, 2024
4 tasks
@jedelbo jedelbo merged commit d318021 into master Jun 11, 2024
38 of 39 checks passed
@jedelbo jedelbo deleted the je/query-between branch June 11, 2024 12:22
@github-actions github-actions bot mentioned this pull request Jun 17, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jul 11, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants