GH-45156: [Python][Packaging] Refactor Python Windows wheel images to use newer base image #45442
Conversation
|
This is failing when the image is built with a thrift checksum mismatch from vcpkg but otherwise what's here in the PR is working. |
|
I let a build of the base image run overnight and it ran out of disk space on my small hard disk but it looks like it built. So next steps here are:
|
amoeba
force-pushed
the
feature/GH-45156--python-windows-wheel-docker-image
branch
from
9c311c3 to
0c18bee
Compare
amoeba
force-pushed
the
feature/GH-45156--python-windows-wheel-docker-image
branch
from
0c18bee to
ed31241
Compare
|
Good progress today. I was able to build the base image with a combination of increasing the memory limit to 4GB and tweaking the storage limit. I propagated the changes to the other dockerfiles and elsewhere in the repo. Now this just needs testing. |
|
Working on this today. I have the vs2022 base image building cleanly now so this runs: archery docker build python-wheel-windows-vs2022(other related builds also run since they use the same base now). |
|
@github-actions crossbow submit wheel-windows-* |
|
Revision: ffa8d79 Submitted crossbow builds: ursacomputing/crossbow @ actions-ad4009d4a3
|
|
Testing this on crossbow now. |
|
@github-actions crossbow submit wheel-windows-* |
|
Revision: bb517c4 Submitted crossbow builds: ursacomputing/crossbow @ actions-4a3296b911
|
|
@github-actions crossbow submit wheel-windows-* |
|
Revision: b420014 Submitted crossbow builds: ursacomputing/crossbow @ actions-1b8ee8dbc2
|
amoeba
requested review from
assignUser,
jonkeane,
kou and
raulcd
as code owners
| RUN curl https://dl.min.io/server/minio/release/windows-amd64/archive/minio.RELEASE.2024-09-13T20-26-02Z ` | ||
| --output "C:\Windows\Minio.exe" | ||
|
|
||
| # Install archiver to extract xz archives (for timezone database). |
There was a problem hiding this comment.
We use a package in PyPI for timezone database now.
Can we remove using archiver?
github-actions
bot
added
awaiting merge
|
This is excellent @amoeba ! Out of curiosity, what is the size of the new Docker images? |
|
@pitrou The new images are a bit smaller which is nice. For example,
|
|
I built all the images and it looks like we generally shave off about 5GiB off each,
|
github-actions
bot
added
awaiting changes
Co-authored-by: Sutou Kouhei <[email protected]>
|
@github-actions crossbow submit wheel-windows-* |
github-actions
bot
added
awaiting change review
|
Revision: 9651440 Submitted crossbow builds: ursacomputing/crossbow @ actions-ef7fa8e459
|
|
It worked. Can we merge this? |
|
Yes, I'll merge it now. |
…ges to use newer base image (apache#45442) ### Rationale for this change See apache#45156. Refactors our previous Windows Docker CI images from an unofficial image to the official Microsoft one based on Windows Server 2022 and adds VS2022 BuildTools. ### What changes are included in this PR? - New Windows base Dockerfiles - Updated Dockerfiles which build on top of that base - Updated Docker Compose services to use new images ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: apache#45156 Lead-authored-by: Bryce Mecum <[email protected]> Co-authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Bryce Mecum <[email protected]>
|
Thanks a lot @amoeba for taking this up, this is really great! |
|
After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit a94f860. There were no benchmark performance regressions. 🎉 The full Conbench report has more details. It also includes information about 9 possible false positives for unstable benchmarks that are known to sometimes produce them. |
…ges to use newer base image (apache#45442) ### Rationale for this change See apache#45156. Refactors our previous Windows Docker CI images from an unofficial image to the official Microsoft one based on Windows Server 2022 and adds VS2022 BuildTools. ### What changes are included in this PR? - New Windows base Dockerfiles - Updated Dockerfiles which build on top of that base - Updated Docker Compose services to use new images ### Are these changes tested? Yes. ### Are there any user-facing changes? No. * GitHub Issue: apache#45156 Lead-authored-by: Bryce Mecum <[email protected]> Co-authored-by: Sutou Kouhei <[email protected]> Signed-off-by: Bryce Mecum <[email protected]>
Rationale for this change
See #45156. Refactors our previous Windows Docker CI images from an unofficial image to the official Microsoft one based on Windows Server 2022 and adds VS2022 BuildTools.
What changes are included in this PR?
Are these changes tested?
Yes.
Are there any user-facing changes?
No.