Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 0.7.0 #716

Merged
merged 5 commits into from
Feb 25, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,95 @@
# Changelog

## [v0.7.0](https://github.com/delta-io/delta-kernel-rs/tree/v0.7.0/) (2025-02-24)

[Full Changelog](https://github.com/delta-io/delta-kernel-rs/compare/v0.6.1...v0.7.0)

### 🏗️ Breaking changes
1. Read transforms are now communicated via expressions ([#607], [#612], [#613], [#614]) This includes:
- `ScanData` now includes a third tuple field: a row-indexed vector of transforms to apply to the `EngineData`.
- Adds a new `scan::state::transform_to_logical` function that encapsulates the boilerplate of applying the transform expression
- Removes `scan_action_iter` API and `logical_to_physical` API
- Removes `column_mapping_mode` from `GlobalScanState`
- ffi: exposes methods to get an expression evaluator and evaluate an expression from c
- read-table example: Removes `add_partition_columns` in arrow.c
- read-table example: adds an `apply_transform` function in arrow.c
2. ffi: support field nullability in schema visitor ([#656])
3. ffi: expose metadata in SchemaEngineVisitor ffi api ([#659])
4. ffi: new `visit_schema` FFI now operates on a `Schema` instead of a `Snapshot` ([#683], [#709])
5. Introduced feature flags (`arrow_54` and `arrow_53`) to select major arrow versions ([#654], [#708], [#717])

### 🚀 Features / new APIs

1. Read `partition_values` in `RemoveVisitor` and remove `break` in `RowVisitor` for `RemoveVisitor` ([#633])
2. Add the in-commit timestamp field to CommitInfo ([#581])
3. Support NOT and column expressions in eval_sql_where ([#653])
4. Add check for schema read compatibility ([#554])
5. Introduce `TableConfiguration` to jointly manage metadata, protocol, and table properties ([#644])
6. Add visitor `SidecarVisitor` and `Sidecar` action struct ([#673])
7. Add in-commit timestamps table properties ([#558])
8. Support writing to writer version 1 ([#693])
9. ffi: new `logical_schema` FFI to get the logical schema of a snapshot ([#709])

### 🐛 Bug Fixes

1. Incomplete multi-part checkpoint handling when no hint is provided ([#641])
2. Consistent PartialEq for Scalar ([#677])
3. Cargo fmt does not handle mods defined in macros ([#676])
4. Ensure properly nested null masks for parquet reads ([#692])
5. Handle predicates on non-nullable columns without stats ([#700])

### 📚 Documentation

1. Update readme to reflect tracing feature is needed for read-table ([#619])
2. Clarify `JsonHandler` semantics on EngineData ordering ([#635])

### 🚜 Refactor

1. Make [non] nullable struct fields easier to create ([#646])
2. Make eval_sql_where available to DefaultPredicateEvaluator ([#627])

### 🧪 Testing

1. Port cdf tests from delta-spark to kernel ([#611])

### ⚙️ Chores/CI

1. Fix some typos ([#643])
2. Release script publishing fixes ([#638])

[#638]: https://github.com/delta-io/delta-kernel-rs/pull/638
[#643]: https://github.com/delta-io/delta-kernel-rs/pull/643
[#619]: https://github.com/delta-io/delta-kernel-rs/pull/619
[#635]: https://github.com/delta-io/delta-kernel-rs/pull/635
[#633]: https://github.com/delta-io/delta-kernel-rs/pull/633
[#611]: https://github.com/delta-io/delta-kernel-rs/pull/611
[#581]: https://github.com/delta-io/delta-kernel-rs/pull/581
[#646]: https://github.com/delta-io/delta-kernel-rs/pull/646
[#627]: https://github.com/delta-io/delta-kernel-rs/pull/627
[#641]: https://github.com/delta-io/delta-kernel-rs/pull/641
[#653]: https://github.com/delta-io/delta-kernel-rs/pull/653
[#607]: https://github.com/delta-io/delta-kernel-rs/pull/607
[#656]: https://github.com/delta-io/delta-kernel-rs/pull/656
[#554]: https://github.com/delta-io/delta-kernel-rs/pull/554
[#644]: https://github.com/delta-io/delta-kernel-rs/pull/644
[#659]: https://github.com/delta-io/delta-kernel-rs/pull/659
[#612]: https://github.com/delta-io/delta-kernel-rs/pull/612
[#677]: https://github.com/delta-io/delta-kernel-rs/pull/677
[#676]: https://github.com/delta-io/delta-kernel-rs/pull/676
[#673]: https://github.com/delta-io/delta-kernel-rs/pull/673
[#613]: https://github.com/delta-io/delta-kernel-rs/pull/613
[#558]: https://github.com/delta-io/delta-kernel-rs/pull/558
[#692]: https://github.com/delta-io/delta-kernel-rs/pull/692
[#700]: https://github.com/delta-io/delta-kernel-rs/pull/700
[#683]: https://github.com/delta-io/delta-kernel-rs/pull/683
[#654]: https://github.com/delta-io/delta-kernel-rs/pull/654
[#693]: https://github.com/delta-io/delta-kernel-rs/pull/693
[#614]: https://github.com/delta-io/delta-kernel-rs/pull/614
[#709]: https://github.com/delta-io/delta-kernel-rs/pull/709
[#708]: https://github.com/delta-io/delta-kernel-rs/pull/708
[#717]: https://github.com/delta-io/delta-kernel-rs/pull/717


## [v0.6.1](https://github.com/delta-io/delta-kernel-rs/tree/v0.6.1/) (2025-01-10)

[Full Changelog](https://github.com/delta-io/delta-kernel-rs/compare/v0.6.0...v0.6.1)
Expand Down
2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ license = "Apache-2.0"
repository = "https://github.com/delta-io/delta-kernel-rs"
readme = "README.md"
rust-version = "1.80"
version = "0.6.1"
version = "0.7.0"

[workspace.dependencies]
object_store = { version = ">=0.11, <0.12" }
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,10 @@ consumer's own `Engine` trait, the kernel has a feature flag to enable a default
```toml
# fewer dependencies, requires consumer to implement Engine trait.
# allows consumers to implement their own in-memory format
delta_kernel = "0.6.1"
delta_kernel = "0.7.0"

# or turn on the default engine, based on arrow
delta_kernel = { version = "0.6.1", features = ["default-engine"] }
delta_kernel = { version = "0.7.0", features = ["default-engine"] }
```

### Feature flags
Expand Down
2 changes: 1 addition & 1 deletion ffi/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ delta_kernel = { path = "../kernel", default-features = false, features = [
"arrow",
"developer-visibility",
] }
delta_kernel_ffi_macros = { path = "../ffi-proc-macros", version = "0.6.1" }
delta_kernel_ffi_macros = { path = "../ffi-proc-macros", version = "0.7.0" }

[build-dependencies]
cbindgen = "0.28"
Expand Down
2 changes: 1 addition & 1 deletion kernel/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ uuid = "1.10.0"
z85 = "3.0.5"

# bring in our derive macros
delta_kernel_derive = { path = "../derive-macros", version = "0.6.1" }
delta_kernel_derive = { path = "../derive-macros", version = "0.7.0" }

# used for developer-visibility
visibility = "0.1.1"
Expand Down
Loading