cpu-inference #20
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| name: cpu-inference | |
| on: | |
| workflow_dispatch: | |
| pull_request: | |
| paths: | |
| - '.github/workflows/cpu-inference.yml' | |
| - 'requirements/**' | |
| - 'deepspeed/__init__.py' | |
| - 'deepspeed/inference/**' | |
| - '!deepspeed/inference/v2/**' # exclude v2 dir | |
| - 'tests/unit/inference/**' | |
| - '!tests/unit/inference/v2/**' # exclude v2 tests dir | |
| merge_group: | |
| branches: [ master ] | |
| schedule: | |
| - cron: "0 0 * * 0" | |
| concurrency: | |
| group: ${{ github.workflow }}-${{ github.ref }} | |
| cancel-in-progress: true | |
| jobs: | |
| unit-tests: | |
| runs-on: [self-hosted, cpu] | |
| steps: | |
| - uses: actions/checkout@v3 | |
| - id: setup-venv | |
| uses: ./.github/workflows/setup-venv | |
| - name: Install gcc-9 | |
| run: | | |
| sudo add-apt-repository -u ppa:ubuntu-toolchain-r/test | |
| sudo apt install -y gcc-9 g++-9 | |
| # set gcc-9 and g++9 to default | |
| sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 99 | |
| sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-9 99 | |
| - name: Check gcc version | |
| run: | | |
| # Get gcc version | |
| gcc --version | |
| g++ --version | |
| - name: Detect instruction sets on instance | |
| run: | | |
| lscpu | |
| pip install cmake | |
| git clone https://github.com/intel/intel-extension-for-pytorch | |
| cd intel-extension-for-pytorch/tests/cpu/isa | |
| cmake . | |
| make | |
| ./cpu_features | |
| - name: Install numactl | |
| run: | | |
| sudo apt-get install -y numactl | |
| - name: Install oneCCL Bindings for PyTorch | |
| run: | | |
| pip install torch | |
| python -m pip install intel_extension_for_pytorch | |
| # the curl line is for troubleshooting | |
| curl -L https://pytorch-extension.intel.com/release-whl/stable/cpu/us/ | |
| python -m pip install oneccl_bind_pt --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/cpu/us/ | |
| pip install py-cpuinfo | |
| # check installed version | |
| pip list |grep \\\<torch\\\> | |
| pip list |grep intel-extension-for-pytorch | |
| pip list |grep oneccl-bind-pt | |
| - name: Install oneCCL | |
| run: | | |
| git clone https://github.com/oneapi-src/oneCCL | |
| cd oneCCL | |
| mkdir build | |
| cd build | |
| cmake .. | |
| make | |
| make install | |
| #source ./_install/env/setvars.sh | |
| # test whether oneCCL is correctly installed | |
| #mpirun -n 2 ./examples/benchmark/benchmark | |
| - name: Install transformers | |
| run: | | |
| git clone https://github.com/huggingface/transformers | |
| cd transformers | |
| git rev-parse --short HEAD | |
| pip install . | |
| - name: Install deepspeed | |
| run: | | |
| # check why the host does not have AVX2 support | |
| pip install .[dev,1bit,autotuning,inf] | |
| ds_report | |
| - name: Python environment check | |
| run: | | |
| pip list | |
| source oneCCL/build/_install/env/setvars.sh | |
| export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6 | |
| # check whether the environment is properly setup | |
| python -c "import torch;import intel_extension_for_pytorch as ipex;import oneccl_bindings_for_pytorch;print('done')" | |
| python -c "import deepspeed;from deepspeed.accelerator import get_accelerator;print(get_accelerator().device_name());print(get_accelerator().is_available())" | |
| - name: Unit tests | |
| run: | | |
| # prep oneCCL for CCLBackend comm ops building | |
| source oneCCL/build/_install/env/setvars.sh | |
| export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libstdc++.so.6 | |
| unset TORCH_CUDA_ARCH_LIST # only jit compile for current arch | |
| cd tests | |
| # LOCAL_SIZE=2 enforce CPU to report 2 devices, this helps run the test on github default runner | |
| LOCAL_SIZE=2 COLUMNS=240 TRANSFORMERS_CACHE=~/tmp/transformers_cache/ TORCH_EXTENSIONS_DIR=./torch-extensions pytest -m 'seq_inference' unit/ | |
| LOCAL_SIZE=2 COLUMNS=240 TRANSFORMERS_CACHE=~/tmp/transformers_cache/ TORCH_EXTENSIONS_DIR=./torch-extensions pytest -m 'inference_ops' -m 'inference' unit/ |