-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ntuple] Make RNTupleChainProcessor
composable
#17393
[ntuple] Make RNTupleChainProcessor
composable
#17393
Conversation
Test Results 18 files 18 suites 4d 13h 53m 5s ⏱️ For more details on these failures, see this check. Results for commit 3da39b3. ♻️ This comment has been updated with latest results. |
8d818f5
to
75745f8
Compare
bf4b538
to
4b69b26
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I left a couple of, mostly minor, comments.
In general lgtm but there are some changes introduced in one commit that are changed by later commits, so it's a bit hard to review commit-wise.
4b69b26
to
b538fcc
Compare
lgtm but better wait for other people's approval as well |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's great! I have some questions on details.
Prevents the page sources corresponding to the processor to be openened upon creation. Instead, defer opening them until the first `Advance` call.
b538fcc
to
e70afa1
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
In anticipation of the composition of the different processors (in particular, the join processor), entries loading should offer the possibility for random access. This will not change anything about the way users can interact with the processor, which is exclusively through the (linear) iterator.
With this change, the `RNTupleChainProcessor` iterates over other `RNTupleProcessor` objects instead of individual ntuples. This allows us to chain, for example, ntuples that have previously been joined using the `RNTupleJoinProcessor`.
When no model is provided, it needs to be inferred from the page source, which first has to be created. This model inference previously happened at the `RNTupleProcessor::Create` factory method. Deferring this to the `RNTupleSingleProcessor` constructor instead removes the need to create the page source twice. In addition, when a model is provided, the page source only gets created when the processor is actually connected.
It is currently not used, and can be re-added when we actually start using it.
e70afa1
to
3da39b3
Compare
This PR introduces the possibility to create
RNTupleChainProcessor
s from other processor objects. In turn, this makes it possible to, for example, create a chain of joined ntuples.This PR is part of a bigger set of (foreseen) changes, collected and tracked in #17132.