Merge remote-tracking branch 'origin/branch-22.06' into main #5782
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fix merge of branch-22.04 to branch-22.06
* Unshim Arrow Signed-off-by: Jason Lowe <[email protected]> * Unshim Alluxio Signed-off-by: Jason Lowe <[email protected]> * Unshim Kryo Signed-off-by: Jason Lowe <[email protected]> * Unshim getGpuColumnarToRowTransition Signed-off-by: Jason Lowe <[email protected]> * Unshim createTable Signed-off-by: Jason Lowe <[email protected]> * Unshim hasSeparateINT96RebaseConf Signed-off-by: Jason Lowe <[email protected]> * Unshim getScalaUDFAsExpression Signed-off-by: Jason Lowe <[email protected]> * Unshim getMapSizesByExecutorId Signed-off-by: Jason Lowe <[email protected]> * Unshim getFileSourceMaxMetadataValueLength Signed-off-by: Jason Lowe <[email protected]> * Unshim getGpuShuffleExchangeExec Signed-off-by: Jason Lowe <[email protected]> * Unshim SortOrder Signed-off-by: Jason Lowe <[email protected]> * Remove HadoopFSUtils shim Signed-off-by: Jason Lowe <[email protected]> * Unshim getLegacyComplexTypeToString Signed-off-by: Jason Lowe <[email protected]> * Unshim shouldFailDivByZero Signed-off-by: Jason Lowe <[email protected]> * Unshim shouldFailOnElementNotExists Signed-off-by: Jason Lowe <[email protected]> * Remove registerKryo from Spark31XdbShims * Unshim shouldFallbackOnAnsiTimestamp
[auto-merge] branch-22.04 to branch-22.06 [skip ci] [bot]
[auto-merge] branch-22.04 to branch-22.06 [skip ci] [bot]
[auto-merge] branch-22.04 to branch-22.06 [skip ci] [bot]
…VIDIA#4993) Signed-off-by: Chong Gao <[email protected]>
[auto-merge] branch-22.04 to branch-22.06 [skip ci] [bot]
* Unshim GpuRegExpReplace Signed-off-by: Jason Lowe <[email protected]> * Unshim Lead and Lag Signed-off-by: Jason Lowe <[email protected]> * Unshim GpuGetArrayItem Signed-off-by: Jason Lowe <[email protected]> * Unshim GetMapValue Signed-off-by: Jason Lowe <[email protected]> * Unshim ElementAt Signed-off-by: Jason Lowe <[email protected]> * Remove GpuTimeSub * Update generated docs * Fix unused import in Spark31XdbShims
Fix merge from 22.04 to 22.06
[auto-merge] branch-22.04 to branch-22.06 [skip ci] [bot]
Signed-off-by: Peixin Li <[email protected]>
Fix merge from 22.04 to 22.06
…ala style guide calls for (NVIDIA#5077) Signed-off-by: Thomas Graves <[email protected]>
Signed-off-by: Robert (Bobby) Evans <[email protected]>
Signed-off-by: Alessandro Bellina <[email protected]>
…5076) * Unshim tests-spark310+ Signed-off-by: Jason Lowe <[email protected]> * Unshim ParquetCachedBatchSerializer * Commonize GpuColumnarToRowExec and GpuColumnarToRowTransitionExec * Move GpuJoinUtils out of shims * Remove 311+-all directory * Remove unshimmed-spark311.txt * Add missing dependency on Databricks for tests build * Add mockito-inline dependency to tests
[auto-merge] branch-22.04 to branch-22.06 [skip ci] [bot]
[auto-merge] branch-22.04 to branch-22.06 [skip ci] [bot]
Signed-off-by: Jason Lowe <[email protected]>
* Patterns to auto-trigger databricks build in premerge Signed-off-by: Peixin Li <[email protected]> * also remove non-existed shims sourcePattern * remove dup unstash * aborted status change * avoid list all changed files
Signed-off-by: Jason Lowe <[email protected]>
* Remove single-buffer compression codec APIs Signed-off-by: Jason Lowe <[email protected]> * Fix copyrights
[auto-merge] branch-22.04 to branch-22.06 [skip ci] [bot]
[auto-merge] branch-22.04 to branch-22.06 [skip ci] [bot]
[auto-merge] branch-22.04 to branch-22.06 [skip ci] [bot]
…IDIA#5090) Signed-off-by: Alessandro Bellina <[email protected]>
* Write Scala test for ORC encrypted write Signed-off-by: Raza Jafri <[email protected]> * add a defensive piece of code to make sure we don't run this code for 3.1.x Signed-off-by: Raza Jafri <[email protected]> * park Signed-off-by: Raza Jafri <[email protected]> * remove the redundant test Signed-off-by: Raza Jafri <[email protected]> Co-authored-by: Raza Jafri <[email protected]>
…ssloader [databricks] (NVIDIA#5716) * use the shim to load the avro class Signed-off-by: Raza Jafri <[email protected]> * Use ShimLoader to load avro class and to check Signed-off-by: Raza Jafri <[email protected]> * addressed review comments Signed-off-by: Raza Jafri <[email protected]> * Review suggestions Signed-off-by: Gera Shegalov <[email protected]> * added documentation Signed-off-by: Raza Jafri <[email protected]> * modified documentation Signed-off-by: Raza Jafri <[email protected]> * addressed build failure Signed-off-by: Raza Jafri <[email protected]> * addressed comments Signed-off-by: Raza Jafri <[email protected]> * added the exact error and warning Signed-off-by: Raza Jafri <[email protected]> Co-authored-by: Raza Jafri <[email protected]> Co-authored-by: Gera Shegalov <[email protected]>
* Init 22.06 changelog Signed-off-by: Peixin Li <[email protected]> * generate based on latest issues and PRs
* enable avrotest in nightly tests Signed-off-by: Peixin Li <[email protected]> * remove space
* Write Scala test for ORC encrypted write Signed-off-by: Raza Jafri <[email protected]> * add a defensive piece of code to make sure we don't run this code for 3.1.x Signed-off-by: Raza Jafri <[email protected]> * park Signed-off-by: Raza Jafri <[email protected]> * added test for ORC write fallback Signed-off-by: Raza Jafri <[email protected]> * add comment Signed-off-by: Raza Jafri <[email protected]> * addressed revied comments Signed-off-by: Raza Jafri <[email protected]> Co-authored-by: Raza Jafri <[email protected]>
Signed-off-by: Tim Liu <[email protected]>
|
build |
Signed-off-by: Firestarman <[email protected]>
…ricks] (NVIDIA#5784) * Update the error string for test_cast_neg_to_decimal_err on 330 Signed-off-by: Firestarman <[email protected]> * address comments Signed-off-by: Firestarman <[email protected]>
|
there are still a couple PRs to be merged so we will have to do this again if it gets merged |
… files on the GPU [databricks] (NVIDIA#5761) * Throw useful message when parquet columnar encryption enabled * update message * fix message * handle native encrypted * move variable * cleanup * fix native check * cleanup imports * fix import order * Sign off Signed-off-by: Thomas Graves <[email protected]> * Shim the parquet crypto exception check Signed-off-by: Thomas Graves <[email protected]> * shim 320cdh * Add test for parquet encryption Signed-off-by: Thomas Graves <[email protected]>
Passes the datatype of round-like functions directly to GPU overrides, so as to adapt different Spark versions. Signed-off-by: sperlingxx <[email protected]>
…IDIA#5766) Fixes NVIDIA#5765 Fix the potential overflow when casting float/double to decimal. The overflow occurs on the container decimal for HALF_UP round. Signed-off-by: sperlingxx <[email protected]>
* remove ORC encryption test from 321cdh Signed-off-by: Raza Jafri <[email protected]> * addressed review comments Signed-off-by: Raza Jafri <[email protected]> Co-authored-by: Raza Jafri <[email protected]>
* Fall back to CPU for RoundCeil and RoundFloor expressions Signed-off-by: Niranjan Artal <[email protected]>
Right, I'll update this PR to pick all the v22.06.0 PRs to main after they get merged |
tgravescs
approved these changes
Jun 14, 2022
|
build |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Merge remote-tracking branch 'origin/branch-22.06' into main
Change version to 22.06.0
Note, merge this PR via Create a merge commit
Signed-off-by: Tim Liu [email protected]