Skip to content

Commit

Permalink
[windows] Documentation: Add use case how to build a Python package
Browse files Browse the repository at this point in the history
This aims to build Python wheels for PyTables in a DIY manner.

It uses Microsoft Visual C++ Build Tools 2015 and Anaconda, both
installed using Chocolatey, and `cibuildwheel`.
  • Loading branch information
amotl committed Jun 12, 2022
1 parent 33817da commit 857a261
Show file tree
Hide file tree
Showing 4 changed files with 135 additions and 2 deletions.
3 changes: 3 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,9 @@ in progress
and ``RACKER_WDM_MACHINE``.
- Add environment variable ``RACKER_WDM_PROVIDER`` to reconfigure the
Vagrant virtualization backend differently than VirtualBox.
- Documentation: Add use case how to build a Python package within a
Windows environment, using Microsoft Visual C++ Build Tools 2015 and
Anaconda, both installed using Chocolatey, and ``cibuildwheel``.


2022-05-20 0.2.0
Expand Down
67 changes: 67 additions & 0 deletions doc/use-cases/python-on-windows.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
###############################
Use cases for Python on Windows
###############################


*************************
Build wheels for PyTables
*************************

DIY, without a hosted CI provider.

References
==========

- https://github.com/PyTables/PyTables/pull/872#issuecomment-773535041
- https://github.com/PyTables/PyTables/blob/master/.github/workflows/wheels.yml

Synopsis
========

.. note::

The ``windows-pytables-wheel.sh`` program is part of this repository. You
will only find it at the designated location when running ``racker`` from
the working tree of its Git repository.

You still can get hold of the program and invoke it, by downloading it from
`windows-pytables-wheel.sh`_.

So, let's start by defining the download URL to that file::

export PAYLOAD_URL=https://raw.githubusercontent.com/cicerops/racker/windows/doc/use-cases/windows-pytables-wheel.sh

Unattended::

time racker --verbose run --rm --platform=windows/amd64 python:3.9 -- \
"sh -c 'wget ${PAYLOAD_URL}; sh -c windows-pytables-wheel.sh'"

Or, interactively::

racker --verbose run -it --rm --platform=windows/amd64 python:3.9 -- bash
wget ${PAYLOAD_URL}
sh -c windows-pytables-wheel.sh


Future
======

See https://github.com/cicerops/racker/issues/8.

When working on the code base, you can invoke the program directly from
the repository, after the ``--volume`` option got implemented::

# Unattended.
time racker --verbose run --rm \
--volume=C:/Users/amo/dev/cicerops-foss/sources/postroj:C:/racker \
--platform=windows/amd64 python:3.9 -- \
sh -c /c/racker/doc/use-cases/windows-pytables-wheel.sh

# Interactively.
racker --verbose run -it --rm \
--volume=C:/Users/amo/dev/cicerops-foss/sources/postroj:C:/racker \
--platform=windows/amd64 python:3.9 -- bash
/c/racker/doc/use-cases/windows-pytables-wheel.sh


.. _windows-pytables-wheel.sh: https://raw.githubusercontent.com/cicerops/racker/main/doc/use-cases/windows-pytables-wheel.sh
59 changes: 59 additions & 0 deletions doc/use-cases/windows-pytables-wheel.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
#!/bin/bash
#
# Build wheels for PyTables on Windows. DIY, without a hosted CI provider.
# https://github.com/cicerops/racker/blob/main/doc/use-cases/python-on-windows.rst
#
# Synopsis::
#
# racker --verbose run -it --rm --platform=windows/amd64 python:3.9 -- bash
# /c/racker/doc/use-cases/windows-pytables-wheel.sh
#
set -e

# Install prerequisites.

# Microsoft Visual C++ Build Tools 2015 14.0.25420.1
# https://community.chocolatey.org/packages/microsoft-visual-cpp-build-tools
choco install --yes microsoft-visual-cpp-build-tools --install-arguments="'/InstallSelectableItems Win81SDK_CppBuildSKUV1;VisualCppBuildTools_ATLMFC_SDK'"

# Miniconda - A minimal installer for Anaconda.
# https://conda.io/miniconda.html
# https://community.chocolatey.org/packages/miniconda3
choco install --yes miniconda3 --package-parameters="'/AddToPath:1'"
export PATH="$PATH:/c/Tools/miniconda3/condabin"
export conda="conda.bat"

# cibuildwheel - Build Python wheels for all the platforms on CI with minimal configuration.
# https://cibuildwheel.readthedocs.io/
pip install --upgrade cibuildwheel

# Check prerequisites.
echo $PATH
$conda --version
# cibuildwheel --version


# Acquire sources.
mkdir -p /c/src
cd /c/src
test ! -d PyTables && git clone https://github.com/PyTables/PyTables --recursive --depth=1
cd PyTables

# Pretend to be on a build matrix.
export MATRIX_ARCH=win_amd64 # win32
export MATRIX_ARCH_SUBDIR=win-64 # win-32


# Configure cibuildwheel.
export CIBW_BUILD="cp36-${MATRIX_ARCH} cp37-${MATRIX_ARCH} cp38-${MATRIX_ARCH} cp39-${MATRIX_ARCH} cp310-${MATRIX_ARCH}"
export CIBW_BEFORE_ALL_WINDOWS="$conda create --yes --name=build && $conda activate build && $conda config --env --set subdir ${MATRIX_ARCH_SUBDIR} && $conda install --yes blosc bzip2 hdf5 lz4 lzo snappy zstd zlib"
export CIBW_ENVIRONMENT_WINDOWS='CONDA_PREFIX="C:\\Miniconda\\envs\\build" PATH="$PATH;C:\\Miniconda\\envs\\build\\Library\\bin"'
export CIBW_ENVIRONMENT="PYTABLES_NO_EMBEDDED_LIBS=true DISABLE_AVX2=true"
export CIBW_BEFORE_BUILD="pip install -r requirements.txt cython>=0.29.21 delvewheel"
export CIBW_REPAIR_WHEEL_COMMAND_WINDOWS="delvewheel repair -w {dest_dir} {wheel}"

# Debugging.
# env

# Build wheel.
cibuildwheel --platform=windows --output-dir=wheelhouse
8 changes: 6 additions & 2 deletions postroj/winrunner.py
Original file line number Diff line number Diff line change
Expand Up @@ -189,8 +189,12 @@ def run(self, command, interactive: bool = False, tty: bool = False):
if interactive or tty:
option_interactive = "-it"

# TODO: Propagate ``--rm`` appropriately.
command = f"docker --context={self.wdm_machine} run {option_interactive} --rm {self.image_real} {command}"
# TODO: Propagate ``--rm`` option appropriately.
# TODO: Propagate ``--volume`` option.
# option_volume = "--volume=C:/Users/amo/dev/cicerops-foss/sources/postroj:C:/racker"
# https://github.com/cicerops/racker/issues/8
option_volume = ""
command = f"docker --context={self.wdm_machine} run {option_interactive} --rm {option_volume} {self.image_real} {command}"

# When an interactive prompt is requested, spawn a shell without further ado.
if interactive or tty:
Expand Down

0 comments on commit 857a261

Please sign in to comment.