Skip to content

Commit

Permalink
Update release.yml
Browse files Browse the repository at this point in the history
@mrwyattii
mrwyattii authored Jan 19, 2024
1 parent 9144b17 commit 1c8b8f3
Showing 1 changed file with 3 additions and 0 deletions.
3 changes: 3 additions & 0 deletions .github/workflows/release.yml
View file
Original file line number Diff line number Diff line change
@@ -16,6 +16,9 @@ jobs:
ref: "master"
- id: setup-venv
uses: ./.github/workflows/setup-venv
- name: Install torch
run: |
pip install torch
- name: Get release version from tag
run: |
echo "RELEASE_VERSION=${GITHUB_REF#refs/*/v}" >> $GITHUB_ENV

2 comments on commit 1c8b8f3

@apoorvkh
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @mrwyattii, I see that you are actively working on the release for deepspeed 0.13.

I think the import torch issue you had with the build in the previous commit is a problem with the package itself, not the release workflow.

For example, the following fails on my end, when trying to install deepspeed==0.13.0:

conda create -p ./.venv python=3.9.18
conda activate ./.venv
pip install deepspeed==0.13.0

which results in

Collecting deepspeed==0.13.0
  Downloading deepspeed-0.13.0.tar.gz (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 22.4 MB/s eta 0:00:00
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [27 lines of output]
      Traceback (most recent call last):
        File "/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/op_builder/xpu/builder.py", line 14, in <module>
          from op_builder.builder import OpBuilder, TORCH_MAJOR, TORCH_MINOR
      ImportError: cannot import name 'TORCH_MAJOR' from 'op_builder.builder' (/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/op_builder/builder.py)
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/setup.py", line 37, in <module>
          from op_builder import get_default_compute_capabilities, OpBuilder
        File "/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/op_builder/__init__.py", line 48, in <module>
          module = importlib.import_module(f".{module_name}", package=op_builder_dir)
        File ".venv/lib/python3.9/importlib/__init__.py", line 127, in import_module
          return _bootstrap._gcd_import(name[level:], package, level)
        File "/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/op_builder/xpu/__init__.py", line 6, in <module>
          from .cpu_adam import CPUAdamBuilder
        File "/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/op_builder/xpu/cpu_adam.py", line 6, in <module>
          from .builder import SYCLOpBuilder
        File "/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/op_builder/xpu/builder.py", line 16, in <module>
          from deepspeed.ops.op_builder.builder import OpBuilder, TORCH_MAJOR, TORCH_MINOR
        File "/tmp/pip-install-3_cqq7tl/deepspeed_405eafe5b6ee43f49e3b9272d170d8ba/deepspeed/__init__.py", line 10, in <module>
          import torch
      ModuleNotFoundError: No module named 'torch'
      [WARNING] Unable to import torch, pre-compiling ops will be disabled. Please visit https://pytorch.org/ to see how to properly install torch on your system.
       [WARNING]  unable to import torch, please install it if you want to pre-compile any deepspeed ops.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

@apoorvkh
Copy link

@apoorvkh apoorvkh commented on 1c8b8f3 Jan 19, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the other hand, running pip install torch before pip install deepspeed==0.13.0 works.

But if torch is a build dependency for DeepSpeed, then I think the correct thing is to add torch to build-system.requires in setup.cfg.

Do you agree?

Please sign in to comment.