Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pip install has issues detecting dependencies on conda #6

Open
masadcv opened this issue Jun 25, 2022 · 3 comments
Open

pip install has issues detecting dependencies on conda #6

masadcv opened this issue Jun 25, 2022 · 3 comments

Comments

@masadcv
Copy link
Owner

masadcv commented Jun 25, 2022

If a system has pytorch pre-installed using conda - then poetry (pyproject.toml) does not detect it and installs pytorch using pip

This causes issues especially if there is a mismatch between CUDA version for pytorch from pip and the CUDA version available on system

To reproduce:

  • Start a docker imagedocker run --gpus all -it --rm nvcr.io/nvidia/pytorch:21.10-py3
  • Install FastGeodis using pip: pip install FastGeodis

The above gives the following error:

{rocessing FastGeodis-1.0.0rc2.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: torch>=1.8.0 in /opt/conda/lib/python3.8/site-packages (from FastGeodis==1.0.0rc2) (1.9.0a0+2ecb2c7)
Requirement already satisfied: typing-extensions in /opt/conda/lib/python3.8/site-packages (from torch>=1.8.0->FastGeodis==1.0.0rc2) (3.7.4.3)
Building wheels for collected packages: FastGeodis
  Building wheel for FastGeodis (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Building wheel for FastGeodis (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [69 lines of output]
      setup.py with torch 1.11.0+cu102
      BUILD_CPP=True, BUILD_CUDA=True, TORCH_VERSION=11100.
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-38
      creating build/lib.linux-x86_64-cpython-38/FastGeodis
      copying FastGeodis/__init__.py -> build/lib.linux-x86_64-cpython-38/FastGeodis
      running build_ext
      /tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/torch/_masked/__init__.py:223: UserWarning: Failed to initialize NumPy: numpy.core.multiarray failed to import (Triggered internally at  ../torch/csrc/utils/tensor_numpy.cpp:68.)
        example_input = torch.tensor([[-3, -2, -1], [0, 1, 2]])
      Traceback (most recent call last):
        File "/opt/conda/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/opt/conda/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/opt/conda/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
          return _build_backend().build_wheel(wheel_directory, config_settings,
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 244, in build_wheel
          return self._build_with_temp_dir(['bdist_wheel'], '.whl',
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 229, in _build_with_temp_dir
          self.run_setup()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 174, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 123, in <module>
          setup(
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 87, in setup
          return distutils.core.setup(**attrs)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 177, in setup
          return run_commands(dist)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 193, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 968, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 1229, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 299, in run
          self.run_command('build')
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 317, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 1229, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/command/build.py", line 24, in run
          super().run()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 131, in run
          self.run_command(cmd_name)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 317, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/dist.py", line 1229, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 987, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 79, in run
          _build_ext.run(self)
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 339, in run
          self.build_extensions()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 410, in build_extensions
          self._check_cuda_version()
        File "/tmp/pip-build-env-9or841db/overlay/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 787, in _check_cuda_version
          raise RuntimeError(CUDA_MISMATCH_MESSAGE.format(cuda_str_version, torch.version.cuda))
      RuntimeError:
      The detected CUDA version (11.3) mismatches the version that was used to compile
      PyTorch (10.2). Please make sure to use the same CUDA versions.
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for FastGeodis
Failed to build FastGeodis
ERROR: Could not build wheels for FastGeodis, which is required to install pyproject.toml-based projects
@masadcv
Copy link
Owner Author

masadcv commented Jun 25, 2022

This is a known issue with poetry pyproject.toml and conda - see: cupy/cupy#4727

A possible workaround can be found here: cupy/cupy#4727 (comment)
is to install on conda systems where pytorch is pre-installed (e.g. docker images) using --no-build-isolation:

pip install FastGeodis --no-build-isolation

@masadcv masadcv mentioned this issue Jun 26, 2022
Open
@masadcv masadcv mentioned this issue Jul 21, 2022
Closed
@masadcv masadcv mentioned this issue May 5, 2024
Closed
@masadcv masadcv mentioned this issue Aug 2, 2024
Closed
@Kin-Zhang
Copy link

Not sure why pip install --no-build-isolation have following issue:

(seflow) ➜  DeFlow git:(feature/benchmarks) ✗ pip install FastGeodis --no-build-isolation          
Collecting FastGeodis
  Using cached FastGeodis-1.0.5-cp38-cp38-linux_x86_64.whl
Requirement already satisfied: torch>=1.5.0 in /home/kin/mambaforge/envs/seflow/lib/python3.8/site-packages (from FastGeodis) (2.0.0)
Requirement already satisfied: filelock in /home/kin/mambaforge/envs/seflow/lib/python3.8/site-packages (from torch>=1.5.0->FastGeodis) (3.15.4)
Requirement already satisfied: typing-extensions in /home/kin/.local/lib/python3.8/site-packages (from torch>=1.5.0->FastGeodis) (4.12.2)
Requirement already satisfied: sympy in /home/kin/mambaforge/envs/seflow/lib/python3.8/site-packages (from torch>=1.5.0->FastGeodis) (1.12.1)
Requirement already satisfied: networkx in /home/kin/mambaforge/envs/seflow/lib/python3.8/site-packages (from torch>=1.5.0->FastGeodis) (3.1)
Requirement already satisfied: jinja2 in /home/kin/.local/lib/python3.8/site-packages (from torch>=1.5.0->FastGeodis) (3.1.2)
Requirement already satisfied: MarkupSafe>=2.0 in /home/kin/.local/lib/python3.8/site-packages (from jinja2->torch>=1.5.0->FastGeodis) (2.1.1)
Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /home/kin/mambaforge/envs/seflow/lib/python3.8/site-packages (from sympy->torch>=1.5.0->FastGeodis) (1.3.0)
Installing collected packages: FastGeodis
Successfully installed FastGeodis-1.0.5
(seflow) ➜  DeFlow git:(feature/benchmarks) ✗ python -c "import FastGeodis"              
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/kin/mambaforge/envs/seflow/lib/python3.8/site-packages/FastGeodis/__init__.py", line 34, in <module>
    import FastGeodisCpp
ImportError: /home/kin/mambaforge/envs/seflow/lib/python3.8/site-packages/FastGeodisCpp.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN3c107WarningC1ENS_7variantIJNS0_11UserWarningENS0_18DeprecationWarningEEEERKNS_14SourceLocationENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEb

But when I install 1.0.4 no issue above.

@masadcv
Copy link
Owner Author

masadcv commented Aug 7, 2024

Hi @Kin-Zhang,
Thanks for reporting. This seems to me like a setup issue similar to reported here: #60

It seems like you are installing FastGeodis from cache, which means it is using a previously built version from the cache that possibly was built with another setup on your machine.

Could you try to force a rebuild:

pip install FastGeodis --no-build-isolation --no-cache-dir

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@masadcv @Kin-Zhang