Improve UX Rename FileScanConfig::new_exec to FileScanConfig::build
#14670
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
core
alamb
commented
Feb 14, 2025
| @@ -504,7 +504,7 @@ impl TableProvider for IndexTableProvider { | |||
| .with_file(partitioned_file); | |||
|
|
|||
| // Finally, put it all together into a DataSourceExec | |||
| Ok(file_scan_config.new_exec()) | |||
| Ok(file_scan_config.build()) | |||
There was a problem hiding this comment.
This is basically the change -- use build rather than new_exec which I think is more consistent with the style of code in the rest of this crate (and the Rust builder style in general)
| _filters: Option<&Arc<dyn PhysicalExpr>>, | ||
| ) -> Result<Arc<dyn ExecutionPlan>> { | ||
| conf = conf.with_source(Arc::new(ArrowSource::default())); | ||
| Ok(conf.new_exec()) | ||
| Ok(conf.with_source(Arc::new(ArrowSource::default())).build()) |
There was a problem hiding this comment.
dive by cleanup to avoid mut and make it more concise
| @@ -252,19 +257,20 @@ impl DataSource for FileScanConfig { | |||
| // If there is any non-column or alias-carrier expression, Projection should not be removed. | |||
| // This process can be moved into CsvExec, but it would be an overlap of their responsibility. | |||
| Ok(all_alias_free_columns(projection.expr()).then(|| { | |||
| let mut file_scan = self.clone(); | |||
| let file_scan = self.clone(); | |||
There was a problem hiding this comment.
Previously calling new_exec always cloned. Now it is only cloned when necessary
| pub fn new_exec(&self) -> Arc<DataSourceExec> { | ||
| Arc::new(DataSourceExec::new(Arc::new(self.clone()))) | ||
| /// Returns a new [`DataSourceExec`] to scan the files specified by this config | ||
| pub fn build(self) -> Arc<DataSourceExec> { |
There was a problem hiding this comment.
renamed and changed to take self rather than &self
| @@ -93,6 +96,8 @@ pub fn wrap_partition_value_in_dict(val: ScalarValue) -> ScalarValue { | |||
| /// PartitionedFile::new("file2.parquet", 56), | |||
| /// PartitionedFile::new("file3.parquet", 78), | |||
| /// ]); | |||
| /// // create an execution plan from the config | |||
There was a problem hiding this comment.
I also tried to add some docs that could make build easier to discover
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
DataSourceExecfor provided datasources, removeParquetExec,CsvExec, etc #14224 from @mertak-synnada ❤️Rationale for this change
This was something I noticed while trying to upgrade InfluxDB IOx to use the new APIs introduced in #14224
I would like to make it easier when someone using these APIs upgrades
What changes are included in this PR?
->build`selfrather than force a cloneAre these changes tested?
Are there any user-facing changes?
There are some changes to APIs that are not yet released