Skip to content

Commit

Permalink
Add more examples
Browse files Browse the repository at this point in the history
  • Loading branch information
alamb committed Feb 26, 2025
1 parent 0e20d8b commit b26ad16
Showing 1 changed file with 30 additions and 0 deletions.
30 changes: 30 additions & 0 deletions docs/source/library-user-guide/upgrading.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,38 @@ See more information

DataFusion 46 has a major change to how the built in DataSources are organized. The

### Cookbook: Changes to `ParquetExecBuilder`
#### Old pattern:

When writing optimizer passes, some code treats ParquetExec specially like this:

```rust
if let Some(parquet_exec) = plan.as_any().downcast_ref::<ParquetExec>() {
// Do something with ParquetExec here
}
}
```

#### New Pattern
With the new DataSource exec, most information is now on `FileScanConfig` and `ParquetSource`

```rust

if let Some(datasource_exec) = plan.as_any().downcast_ref::<DataSourceExec>() {
if let Some(scan_config) = datasource_exec.source().as_any().downcast_ref::<FileScanConfig>() {
// FileGroups, and other information is on the FileScanConfig
// parquet
if let Some(parquet_source) = scan_config.source.as_any().downcast_ref::<ParquetSource>()
{
// Information on PruningPredicates and parquet options are here
}
}
```

### Cookbook: Changes to `ParquetExecBuilder`

#### Old pattern:

```rust
let mut exec_plan_builder = ParquetExecBuilder::new(
FileScanConfig::new(self.log_store.object_store_url(), file_schema)
Expand Down Expand Up @@ -124,3 +153,4 @@ DataFusion 46 has a major change to how the built in DataSources are organized.
parquet_scan: file_scan_config.build(),

```

0 comments on commit b26ad16

Please sign in to comment.