Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI][Update] Update CI Actions for releasing a brand new release package. #1219

Merged
merged 23 commits into from
Apr 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
File renamed without changes.
42 changes: 42 additions & 0 deletions .github/workflows/release_linux_x86_64.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: Build Package

on:
release:
types: [published]

permissions:
contents: read

jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [ '3.8', '3.9', '3.10', '3.11', '3.12']
architecture: [ 'x64' ]

# For the sake of simplicity in testing, the Paddle2ONNX packaging program will temporarily only package executable files for Python 3.8.
# In the future, we will need to extend support to cover Python 3.8 through Python 3.10.
steps:
# Checkout the latest branch of Paddle2ONNX.
- name: Checkout Paddle2ONNX
uses: actions/checkout@v4
with:
submodules: true

- name: Build on manylinux2014_x86_64
uses: docker://quay.io/pypa/manylinux2014_x86_64:latest
with:
entrypoint: bash
args: .github/workflows/scripts/entrypoint.sh ${{ matrix.python-version }} manylinux2014_x86_64

- uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32
with:
name: wheels
path: dist

- name: Publish package
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
user: __token__
password: ${{ secrets.ZHENG_BICHENG_PYPI_TOKEN }}
22 changes: 22 additions & 0 deletions .github/workflows/scripts/build_protobuf_unix.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#!/bin/bash

# Copyright (c) ONNX Project Contributors
#
# SPDX-License-Identifier: Apache-2.0

export CORE_NUMBER=$1

if [[ -z "$CORE_NUMBER" ]]; then
export CORE_NUMBER=1
fi

# Build protobuf from source with -fPIC on Unix-like system
ORIGINAL_PATH=$(pwd)
cd ..
git clone https://github.com/protocolbuffers/protobuf.git -b v3.16.0
cd protobuf
mkdir build_source && cd build_source
cmake ../cmake -DCMAKE_INSTALL_PREFIX=${PWD}/installed_protobuf_lib -Dprotobuf_BUILD_SHARED_LIBS=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON -Dprotobuf_BUILD_TESTS=OFF -DCMAKE_BUILD_TYPE=Release
make -j$CORE_NUMBER && make install
export PATH=${PWD}/installed_protobuf_lib/bin:${PATH}
cd $ORIGINAL_PATH
54 changes: 54 additions & 0 deletions .github/workflows/scripts/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
#!/bin/bash

# Copyright (c) ONNX Project Contributors
#
# SPDX-License-Identifier: Apache-2.0

set -e -x

# CLI arguments
PY_VERSION=$1
PLAT=$2
GITHUB_EVENT_NAME=$3

export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/lib

# Compile wheels
# Need to be updated if there is a new Python Version
if [ "$(uname -m)" == "aarch64" ]; then
PIP_INSTALL_COMMAND="$PY_VERSION -m pip install --no-cache-dir -q"
PYTHON_COMMAND="$PY_VERSION"
else
declare -A python_map=( ["3.8"]="cp38-cp38" ["3.9"]="cp39-cp39" ["3.10"]="cp310-cp310" ["3.11"]="cp311-cp311" ["3.12"]="cp312-cp312")
PY_VER=${python_map[$PY_VERSION]}
PIP_INSTALL_COMMAND="/opt/python/${PY_VER}/bin/pip install --no-cache-dir -q"
PYTHON_COMMAND="/opt/python/${PY_VER}/bin/python"
fi

# Update pip and install cmake
$PIP_INSTALL_COMMAND --upgrade pip
$PIP_INSTALL_COMMAND cmake

# Build protobuf from source
source .github/workflows/scripts/build_protobuf_unix.sh "$(nproc)"

# Build Paddle2ONNX wheels
$PYTHON_COMMAND -m build --wheel || { echo "Building wheels failed."; exit 1; }

# Bundle external shared libraries into the wheels
# find -exec does not preserve failed exit codes, so use an output file for failures
failed_wheels=$PWD/failed-wheels
rm -f "$failed_wheels"
find . -type f -iname "*-linux*.whl" -exec sh -c "auditwheel repair '{}' -w \$(dirname '{}') --plat '${PLAT}' || { echo 'Repairing wheels failed.'; auditwheel show '{}' >> '$failed_wheels'; }" \;

if [[ -f "$failed_wheels" ]]; then
echo "Repairing wheels failed:"
cat failed-wheels
exit 1
fi

# Remove useless *-linux*.whl; only keep manylinux*.whl
rm -f dist/*-linux*.whl

echo "Successfully build wheels:"
find . -type f -iname "*manylinux*.whl"
4 changes: 1 addition & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,11 @@ TensorRT/OpenVINO/MNN/TNN/NCNN,以及其它对 ONNX 开源格式进行支持

# 3 安装

针对PaddlePaddle2.5.2的用户可以直接运行以下命令行代码来安装P2O

```
pip install paddle2onnx
```

由于没有自动发包机制,针对PaddlePaddle2.6.0的用户,请按照[Github 源码安装方式](docs/zh/compile.md)编译Paddle2ONNX。
开发用户,请按照[Github 源码安装方式](docs/zh/compile_local.md)编译Paddle2ONNX。

# 4 使用

Expand Down
2 changes: 1 addition & 1 deletion VERSION_NUMBER
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.1.0
1.2.0
19 changes: 0 additions & 19 deletions docs/zh/change_log.md

This file was deleted.

26 changes: 26 additions & 0 deletions docs/zh/compile_docker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Docker编译安装Paddle2ONNX

Paddle2ONNX编译安装需要确保环境满足以下需求
- cmake >= 3.18.0
- protobuf == 3.16.0

注意:Paddle2ONNX产出的模型,在使用ONNX Runtime推理时,要求使用最新版本(1.10.0版本以及上),如若需要使用低版本(1.6~1.10之间),则需要将ONNX版本降至1.8.2,在执行完`git submodule update`后,执行如下命令,然后再进行编译
```
cd Paddle2ONNX/third/onnx
git checkout v1.8.1
```

拉取manylinux镜像并创建容器

```bash
docker pull quay.io/pypa/manylinux2014_x86_64
docker run --name p2o_build -d quay.io/pypa/manylinux2014_x86_64
```

创建容器并运行

```bash
docker start p2o_build
docker exec -it p2o_build bash
```

13 changes: 9 additions & 4 deletions docs/zh/compile.md → docs/zh/compile_local.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# 编译安装Paddle2ONNX
# 本地编译安装Paddle2ONNX

Paddle2ONNX编译安装需要确保环境满足以下需求
- cmake >= 3.18.0
- protobuf >= 3.16.0
- protobuf == 3.16.0

注意:Paddle2ONNX产出的模型,在使用ONNX Runtime推理时,要求使用最新版本(1.10.0版本以及上),如若需要使用低版本(1.6~1.10之间),则需要将ONNX版本降至1.8.2,在执行完`git submodule update`后,执行如下命令,然后再进行编译
```
Expand All @@ -13,6 +13,7 @@ git checkout v1.8.1
## Linux/Mac编译安装

### 安装Protobuf

```
git clone https://github.com/protocolbuffers/protobuf.git
cd protobuf
Expand All @@ -25,26 +26,30 @@ make install
# 将编译目录加入环境变量
export PATH=${PWD}/installed_protobuf_lib/bin:${PATH}
```

### 安装Paddle2ONNX

```
git clone https://github.com/PaddlePaddle/Paddle2ONNX.git
cd Paddle2ONNX
git submodule init
git submodule update

python -m build
python -m pip install dist/*.whl
```

## Windows编译安装

注意Windows编译安装先验条件是系统中已安装好Visual Studio 2019

### 打开VS命令行工具

系统菜单中,找到**x64 Native Tools Command Prompt for VS 2019**打开

### 安装Protobuf

注意下面cmake命令中`-DCMAKE_INSTALL_PREFIX`指定为你实际设定的路径

```
git clone https://github.com/protocolbuffers/protobuf.git
cd protobuf
Expand All @@ -59,12 +64,12 @@ set PATH=D:\Paddle\installed_protobuf_lib\bin;%PATH%
```

### 安装Paddle2ONNX

```
git clone https://github.com/PaddlePaddle/Paddle2ONNX.git
cd Paddle2ONNX
git submodule init
git submodule update

python -m build
python -m pip install dist/*.whl
```
Empty file removed docs/zh/custom_op_mapper.md
Empty file.