How can I save partitioned Parquet ? #3737
Answered
by
andygrove
Miyake-Diogo
asked this question in
Q&A
Replies: 2 comments 2 replies
-
Have you tried calling write_parquet on a DataFrame? https://arrow.apache.org/datafusion/user-guide/dataframe.html |
Beta Was this translation helpful? Give feedback.
1 reply
-
let ctx = SessionContext::new();
ctx
.read_csv("data.csv", CsvReadOptions::default().has_header(true)).await?
.repartition(Partitioning::Hash(vec![col("id")], 2))?
.write_parquet("data.parquet", None).await?; |
Beta Was this translation helpful? Give feedback.
1 reply
Answer selected by
Miyake-Diogo
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
How can I save partitioned parquet like apache spark?
Beta Was this translation helpful? Give feedback.
All reactions