Transfer cibuildwheel action (rism-digital#1893)

* check with policy * ste cmake flag * enforce flag * try alias (https://github.com/actions/virtual-environments/blob/main/images/macos/macos-10.15-Readme.md#language-and-runtime) * test with brew (https://github.com/ltcmelo/psychec/blob/master/.github/workflows/generator-build.yml) * test * test * test * test * test * test * fix indentation * test * fix symbol * removing options * remove unneeded stuff * should run fine * check if apt install is needed on Ubuntu * test * check available compilers * check for specific compiler * check for prebuilt compilers * use g++ * test * test * test * test * fix symbol * test with condensed logic * test win * test * missing if * try win * fix indentation * fix indentation again * test * get cmake * fix version * order * test * test * test * use XCode Developer * test * test * fix syntax * test * test * test * fix path * check * remove win * test * test * fix option * remove * test * test * test * test * get version * test * test * check * test * mute verify * test * test * test * test with 11.7 * test * test tests * test with full test suite * cache build * upload artifacts * git status * fix path * fix path * get changes * test * test * catch changed files * parallel build * test * test * test * add missing needs * test order * remove cache * test * Create python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update test-suite.yml * Update test-suite.yml * Test with win path * Test without nuget * Text with choco * check swig install * Update python-build.yml * Update setup.py * Update setup.py * Update python-build.yml * Update setup.py * Update python-build.yml * Update setup.py * Update python-build.yml * Update README.md * Update python-build.yml * Update python-build.yml * Update python-build.yml * Create python-ci-wheel.yml * Update python-ci-wheel.yml * Update python-ci-wheel.yml * Update setup.py * Update python-ci-wheel.yml * Update python-build.yml * Update python-ci-wheel.yml * Update python-ci-wheel.yml * Update python-ci-wheel.yml * Update setup.py * Update python-ci-wheel.yml * Update setup.py * Update python-build.yml * Update python-build.yml * Update setup.py * Update python-ci-wheel.yml * Update python-build.yml * Update setup.py * Update setup.py * Update setup.py * Create MANIFEST.in * Update MANIFEST.in * Update setup.py * Test with TestPyPi * Update setup.py * Update python-build.yml * Update python-build.yml * Run build_ext first to generate verovio.py * Update setup.py * Update setup.py * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update setup.py * Update setup.py * Delete MANIFEST.in * Test with data * Update setup.py * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update python-build.yml * Update setup.py * Test with top level setup * Update setup.py * Move complete python bindings to top level * Update setup.py * Only copy setup.py * copy complete python bindings to top level again * check with cibuildwheel * adapt swig to put verovio.py on top level * include linux for wheels * add swig to manylinux docker env * fix path in verovio.i * check toolkit installation * check with specific distros * update ci-wheel * specify architecture * test * test * wildcard not working with Windows PowerShell * use PowerShell specific command on Windows * remove module option * test * GetChildItem on Win * rename * update wheel.yml * no message * test * test with data_files * fix paths * Adding namespace-level function, see lpugin/verovio#0bc90d4fff985597e93245a73ccbba288f13ba65 * patch * adapt path * Add patches from https://github.com/lpugin/verovio/pull/17 * fix path * fix path to verovio.i * no message * no message * use python -m * fix python 3.9 * check installation from subfolder * no message * adjust os.system sub-shell command * check commit * verify version * run build_ext before sdist * run build_ext before sdist * include upload * force bash for build_ext to run shell script from Win * check with cibuild * fix install * move verovio module to bindings/python * test to trigger swig before cibuild * try to get correct swig on linux * try to get correct siwg on linux (2) * use manylinux2014 for cibuildwheel * use yum to install swig on linux (docker) * use specific ubuntu version * fix typo * build swig on linux * use yum again * run swig before build * include ubuntu x86 * there is no x86 ubuntu, use ubuntu-20.4 x64 instead * exlucde x86 for mac and linux * move everything into one file * test * Update setup.py * remove bindings/python/setup.py * cosmetics * fix test version * switch back module name and version * remove obsolete files * reset analytical.cpp (where is this coming from?) * indent in analytical * fix branch Co-authored-by: Laurent Pugin <[email protected]>
DDMAL · Dec 19, 2020 · b0099f6 · b0099f6
1 parent 8a0c256
commit b0099f6
Show file tree

Hide file tree

Showing 9 changed files with 437 additions and 73 deletions.
diff --git a/.github/workflows/python-ci-wheel.yml b/.github/workflows/python-ci-wheel.yml
@@ -0,0 +1,299 @@
+name: Python Wheel Build
+
+on:
+  workflow_dispatch: # manual trigger
+  push:
+    # Trigger the workflow on push,
+    # but only for the branches specified
+    branches:
+      # Push events on specified branch
+      - develop-pypi
+
+
+# globals
+env:
+  PACKAGE_NAME: verovio
+  CHECK_INSTALL_DIR: check_install_dir
+  CIBW_BUILD_IDENTIFIER: ''
+
+
+jobs:
+  #===============================================#
+  #         BUILD WHEEL DISTRIBUTION              #
+  #===============================================#
+  build_wheels:
+    name: Build wheels (${{ matrix.python-version }}, ${{ matrix.os }}-${{ matrix.architecture }})
+    runs-on: ${{ matrix.os }}
+    strategy:
+      # Ensure that a wheel builder finishes even if another fails
+      fail-fast: false
+      # Build the wheels for Linux, Windows and macOS for Python 3.9
+      matrix:
+        os: [ macos-latest, windows-latest, ubuntu-18.04, ubuntu-20.04]
+        python-version: [ 3.9 ] # [3.6, 3.7, 3.8, 3.9]
+        architecture: [ x86, x64 ]
+        include:
+          - os: macos-latest
+            architecture: x64
+            platform_id: macosx_x86_64
+          - os: windows-latest
+            architecture: x64
+            platform_id: win_amd64
+          - os: windows-latest
+            architecture: x86
+            platform_id: win32
+          - os: ubuntu-18.04
+            architecture: x64
+            platform_id: manylinux_x86_64
+          - os: ubuntu-20.04
+            architecture: x64
+            platform_id: manylinux_x86_64
+        exclude:
+          - os: macos-latest
+            architecture: x86
+          - os: ubuntu-18.04
+            architecture: x86
+          - os: ubuntu-20.04
+            architecture: x86
+
+    steps:
+      #===============================================#
+      # Set up
+      - uses: actions/checkout@v2
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+          architecture: ${{ matrix.architecture }}
+
+      #===============================================#
+      # Dependencies
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install cibuildwheel==1.7.1 twine
+
+      #===============================================#
+      # Prepare identifier
+      - name: Set CIBW_BUILD_IDENTIFIER
+        shell: bash
+        run: |
+          # Remove dot from python-version for CIBW and prefix with cp
+          id=cp$(echo ${{ matrix.python-version }} | sed -e 's/\.//g')-${{ matrix.platform_id }}
+          echo CIBW_BUILD_IDENTIFIER=$id >> $GITHUB_ENV
+
+      # TODO: Can be removed when everything is set up
+      - name: Check CIBW_BUILD_IDENTIFIER
+        shell: bash
+        run: echo ${{ env.CIBW_BUILD_IDENTIFIER }}
+
+      # TODO: Can be removed when everything is set up
+      - name: Get Build Identifiers provided by cibuildwheel for ${{ matrix.os }}-${{ matrix.architecture }}
+        run: python -m cibuildwheel --print-build-identifiers
+
+      #===============================================#
+      # wheels
+      - name: Build wheels
+        env:
+          CIBW_SKIP: cp27-* # manylinux2014 not compatible with python 2.7
+          CIBW_BUILD: ${{ env.CIBW_BUILD_IDENTIFIER }}
+          CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014
+          CIBW_MANYLINUX_I686_IMAGE: manylinux2014
+          CIBW_BEFORE_ALL_LINUX: yum install swig -y
+          CIBW_BEFORE_ALL_MACOS: brew update && brew install swig
+          CIBW_BEFORE_ALL_WINDOWS: choco install swig -f -y
+          CIBW_BEFORE_BUILD: swig -version && bash -c 'cd tools; ./get_git_commit.sh' && swig -c++ -python ./bindings/python/verovio.i
+        run: python -m cibuildwheel --output-dir wheelhouse
+
+      #===============================================#
+      # Check build
+
+      # TODO: Can be removed when everything is set up
+      - name: "List result"
+        working-directory: wheelhouse
+        run: ls -R
+
+      - name: Check with Twine
+        working-directory: wheelhouse
+        run: twine check *.whl
+
+      - name: Install from wheel on Unix
+        if: always() && runner.os != 'Windows'
+        working-directory: wheelhouse
+        run: python -m pip install ./*.whl
+
+      # Wildcard use is different with PowerShell
+      # cf. https://stackoverflow.com/a/43900040
+      - name: Install from wheel on Windows
+        working-directory: wheelhouse
+        if: always() && runner.os == 'Windows'
+        run: python -m pip install (get-item .\*.whl).FullName
+
+      - name: Check wheel installation
+        working-directory: wheelhouse
+        run: python -c "import verovio; tk = verovio.toolkit(); v = tk.getVersion(); print('verovio.toolkit.getVersion():', v)"
+
+      #===============================================#
+      # Upload artifacts
+      - uses: actions/upload-artifact@v2
+        with:
+          name: cibuildwheel-${{ runner.os }}-python-${{ matrix.python-version }}
+          path: ./wheelhouse/*.whl
+
+
+  #===============================================#
+  #         BUILD SOURCE DISTRIBUTION             #
+  #===============================================#
+  build_sdist:
+    name: Build sdist (${{ matrix.python-version }}, ${{ matrix.os }})
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ubuntu-18.04]
+        python-version: [3.9]
+
+    steps:
+      #===============================================#
+      # Set up
+      - uses: actions/checkout@v2
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      #===============================================#
+      # Swig
+      - name: Install swig
+        if: runner.os == 'Linux'
+        run: |
+          sudo apt update -q
+          sudo apt install swig
+
+      - name: Verify swig
+        run: swig -version
+
+      #===============================================#
+      # Dependencies
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install setuptools wheel twine
+
+      #===============================================#
+      # sdist
+      - name: Build ext
+        run: python setup.py build_ext
+
+      - name: Build source distribution
+        run: python setup.py sdist
+
+      #===============================================#
+      # Check build
+
+      # TODO: Can be removed when everything is set up
+      - name: List result
+        working-directory: dist
+        run: ls -R
+
+      - name: Check with Twine
+        working-directory: dist
+        run: twine check *.tar.gz
+
+      - name: Create install dir
+        shell: bash
+        run: mkdir - p ${{ env.CHECK_INSTALL_DIR }}
+
+      - name: Install from source
+        working-directory: ${{ env.CHECK_INSTALL_DIR }}
+        run: python -m pip install ../dist/*.tar.gz
+
+      - name: Check source installation
+        working-directory: ${{ env.CHECK_INSTALL_DIR }}
+        run: python -c "import verovio; tk = verovio.toolkit(); v = tk.getVersion(); print('verovio.toolkit.getVersion():', v)"
+
+      #===============================================#
+      # Upload artifact
+      - uses: actions/upload-artifact@v2
+        with:
+          name: sdist-${{ runner.os }}-python-${{ matrix.python-version }}
+          path: dist/*.tar.gz
+
+
+  #===============================================#
+  #         UPLOAD TO PACKAGE INDEX               #
+  #===============================================#
+  upload_testpypi:
+    name: Upload to TestPyPi
+    needs: [build_wheels, build_sdist]
+    runs-on: ${{ matrix.os }}
+    strategy:
+      matrix:
+        os: [ubuntu-18.04]
+        python-version: [3.9]
+        architecture: [ x64 ]
+
+    steps:
+      #===============================================#
+      # Set up
+      - uses: actions/checkout@v2
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+          architecture: ${{ matrix.architecture }}
+
+      #===============================================#
+      # Dependencies
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install twine
+
+      #===============================================#
+      # Prepare artifacts
+      - name: Download artifacts
+        uses: actions/download-artifact@v2
+        with:
+          path: bindings/python/artifacts/
+
+      # TODO: Can be removed when everything is set up
+      - name: Display structure of downloaded files
+        working-directory: bindings/python/artifacts/
+        run: ls -R
+
+      - name: Copy artifacts to dist folder
+        working-directory: bindings/python/
+        run: |
+          mkdir -p ./dist
+          cp artifacts/*-${{ matrix.python-version }}/* dist/
+
+      # TODO: Can be removed when everything is set up
+      - name: Verify dist folder
+        working-directory: bindings/python/dist/
+        run: ls -R
+
+      #===============================================#
+      # Upload to package index
+      - name: Upload to TestPyPi
+        working-directory: bindings/python
+        env:
+          TWINE_USERNAME: ${{ secrets.TESTPYPI_USERNAME }}
+          TWINE_PASSWORD: ${{ secrets.TESTPYPI_TOKEN }}
+        run: twine upload --repository testpypi dist/*
+
+      #===============================================#
+      # Check build
+      - name: Create install dir
+        shell: bash
+        run: mkdir - p ${{ env.CHECK_INSTALL_DIR }}
+
+      - name: Install from package index
+        working-directory: ${{ env.CHECK_INSTALL_DIR }}
+        run: python -m pip install --index-url https://test.pypi.org/simple/ ${{ env.PACKAGE_NAME }}
+
+      - name: Check installation
+        working-directory: ${{ env.CHECK_INSTALL_DIR }}
+        run: python -c "import verovio; tk = verovio.toolkit(); v = tk.getVersion(); print('verovio.toolkit.getVersion():', v)"
diff --git a/MANIFEST.in b/MANIFEST.in
@@ -0,0 +1,4 @@
+recursive-include data *
+recursive-include src *
+recursive-include include *
+recursive-include libmei *
diff --git a/README.md b/README.md
@@ -41,7 +41,7 @@ Major releases of Verovio and MEI versions:
 From Verovio 2.x.x, the plan is to have even version numbers for Verovio releases using a stable version of MEI, and odd version numbers for releases using a development version of MEI. It means that once MEI 5.0 will be released, Verovio will move to version 4.x.x. Older versions of MEI are still supported by newer versions of Verovio. MEI files are internally upgraded when loaded into Verovio. This applies only to the features supported by Verovio. We will try to maintain this in the future.
 
 Other libraries
---------------
+---------------
 
 The following libraries are embedded in Verovio:
 
@@ -66,7 +66,7 @@ The sample page of music shown below was generated with version 2.4.0-dev-2748fe
 ![Example page](https://raw.githubusercontent.com/rism-ch/verovio.org/gh-pages/images/verovio-exp-2.4.0-dev.png)
 
 Example resources using verovio
------------------------------------------
+-------------------------------
 
 | name | type  | description |
 | ---- | ----- | ----------- |
@@ -80,13 +80,13 @@ Example resources using verovio
 
 
 Digital score repositories on Github
--------------------------------------
+------------------------------------
 
 Here is a list of digital score repositories on Github that can be displayed with verovio:
 
 
 | link | encoding | description |
-| ---- | ------ | ----------- | 
+| ---- | -------- | ----------- | 
 |  [MEI complete examples](https://github.com/music-encoding/sample-encodings/tree/master/MEI_4.0/Music/Complete_examples) | MEI | 86 various works encoded in MEI |
 | [Mozart Piano Sonatas](https://github.com/craigsapp/mozart-piano-sonatas) | Humdrum | 17 Piano sonatas by W.A. Mozart from the Alte Mozart-Ausgabe ([in VHV](http://verovio.humdrum.org/?file=mozart/sonatas&k=e)) |
 | [Beethoven Piano Sonatas](https://github.com/craigsapp/beethoven-piano-sonatas) | Humdrum | 32 Piano sonatas by L. van Beethoven, edited by Paul Dukas ([in VHV](http://verovio.humdrum.org/?file=beethoven/sonatas&k=e)) |

diff --git a/bindings/python/__init__.py b/bindings/python/__init__.py
@@ -0,0 +1,5 @@
+from pkg_resources import resource_filename
+
+from .verovio import *
+
+verovio.setDefaultResourcePath(resource_filename('verovio', 'data'))