Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problems installing #3

Closed
Nora-Go opened this issue Feb 2, 2024 · 8 comments
Closed

Problems installing #3

Nora-Go opened this issue Feb 2, 2024 · 8 comments

Comments

@Nora-Go
Copy link

Nora-Go commented Feb 2, 2024

Hi :) and moin!

I fear I face a problem installing your package on a cluster (locally it works fine).
I create a venv with a python version (tried 3.7, 3.8, 3.9, 3.10, 3.11), make cuda available (tried 11.3.1, 11.6.2, 11.7.1, 12.1.1) and install torch (tried 1.13.0+cu116, 2.2.0+cu118).
Then I tried pip install bilateralfilter_torch (I also tried it via the setup.py).

The error I get is:

Collecting bilateralfilter_torch
Using cached bilateralfilter_torch-1.1.0.tar.gz (17 kB)
Preparing metadata (setup.py) ... done
Building wheels for collected packages: bilateralfilter_torch
Building wheel for bilateralfilter_torch (setup.py) ... error
error: subprocess-exited-with-error

× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [84 lines of output]
    No CUDA runtime is found, using CUDA_HOME='/apps/SPACK/0.17.1/opt/linux-almalinux8-zen/gcc-8.4.1/cuda-11.3.1-rxdhgaje46fxemfejzpdm3gzz7y2tf3m'
    /home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/__init__.py:80: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
    !!

            ********************************************************************************
            Requirements should be satisfied by a PEP 517 installer.
            If you are using pip, you can try `pip install --use-pep517`.
            ********************************************************************************

    !!
      dist.fetch_build_eggs(dist.setup_requires)
    running bdist_wheel
    /home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/torch/utils/cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
      warnings.warn(msg.format('we could not find ninja.'))
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-cpython-38
    copying bilateral_filter_layer.py -> build/lib.linux-x86_64-cpython-38
    copying example_filter.py -> build/lib.linux-x86_64-cpython-38
    copying example_optimization.py -> build/lib.linux-x86_64-cpython-38
    copying gradcheck.py -> build/lib.linux-x86_64-cpython-38
    running build_ext
    /home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/torch/utils/cpp_extension.py:387: UserWarning: The detected CUDA version (11.3) has a minor version mismatch with the version that was used to compile PyTorch (11.6). Most likely this shouldn't be a problem.
      warnings.warn(CUDA_MISMATCH_WARN.format(cuda_str_version, torch.version.cuda))
    building 'bilateralfilter_gpu_lib' extension
    creating build/temp.linux-x86_64-cpython-38
    creating build/temp.linux-x86_64-cpython-38/csrc
    Traceback (most recent call last):
      File "<string>", line 2, in <module>
      File "<pip-setuptools-caller>", line 34, in <module>
      File "/tmp/pip-install-wbrzl1fz/bilateralfilter-torch_b486764707934d4c84652bdbd2f9b85d/setup.py", line 7, in <module>
        setup(
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/__init__.py", line 103, in setup
        return distutils.core.setup(**attrs)
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 185, in setup
        return run_commands(dist)
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
        dist.run_commands()
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
        self.run_command(cmd)
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
        super().run_command(command)
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
        cmd_obj.run()
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 364, in run
        self.run_command("build")
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
        self.distribution.run_command(command)
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
        super().run_command(command)
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
        cmd_obj.run()
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 131, in run
        self.run_command(cmd_name)
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
        self.distribution.run_command(command)
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/dist.py", line 989, in run_command
        super().run_command(command)
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
        cmd_obj.run()
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 88, in run
        _build_ext.run(self)
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
        self.build_extensions()
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 843, in build_extensions
        build_ext.build_extensions(self)
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 467, in build_extensions
        self._build_extensions_serial()
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 493, in _build_extensions_serial
        self.build_extension(ext)
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/command/build_ext.py", line 249, in build_extension
        _build_ext.build_extension(self, ext)
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/_distutils/command/build_ext.py", line 548, in build_extension
        objects = self.compiler.compile(
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/setuptools/_distutils/ccompiler.py", line 600, in compile
        self._compile(obj, src, ext, cc_args, extra_postargs, pp_opts)
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 581, in unix_wrap_single_compile
        cflags = unix_cuda_flags(cflags)
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 548, in unix_cuda_flags
        cflags + _get_cuda_arch_flags(cflags))
      File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1780, in _get_cuda_arch_flags
        arch_list[-1] += '+PTX'
    IndexError: list index out of range
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for bilateralfilter_torch
Running setup.py clean for bilateralfilter_torch
Failed to build bilateralfilter_torch

Trying it with --use-pep517 leads to:

Collecting bilateralfilter_torch
  Using cached bilateralfilter_torch-1.1.0.tar.gz (17 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      Traceback (most recent call last):
        File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/home/atuin/b110dc/b110dc10/software/privat/conda/envs/DenoisedHookFormer_env3/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
        File "/tmp/pip-build-env-4z8rjffc/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
        File "/tmp/pip-build-env-4z8rjffc/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-4z8rjffc/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 480, in run_setup
          super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
        File "/tmp/pip-build-env-4z8rjffc/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 2, in <module>
      ModuleNotFoundError: No module named 'torch'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

I'm stuck on both errors, as all possible solutions would require sudo rights (which I don't have) as far as I see it.

Could you help me with that? Already telling me the versions you used, might help me :)
@faebstn96
Copy link
Owner

Hi @Nora-Go,
The easiest workaround is probably installing the filter layer from the MONAI repo: https://docs.monai.io/en/stable/networks.html#trainablebilateralfilter

Their pre-compiled library is less sensitive to different Cuda/PyTorch versions. We contributed the trainable bilateral filter for better usability and platform support.

Let me know if that worked for your cluster. If not, I can provide my running configuration. Note that the layer input layout is slightly modified in the MONAI layer to adapt MONAI conventions.

Hope that helped!

@Nora-Go
Copy link
Author

Nora-Go commented Feb 2, 2024

Hi,

I fear it is not working for me:
AttributeError: module 'monai.networks.layers' has no attribute 'TrainableBilateralFilter'
ImportError: cannot import name 'TrainableBilateralFilter' from 'monai.networks.layers'

I have monai version 1.1.0 installed.
Should I open an issue at monai's?

@faebstn96
Copy link
Owner

I just checked, the layers were first part of release 1.2.0 (https://github.com/Project-MONAI/MONAI/releases/tag/1.2.0). So, upgrading MONAI should do the job :)

@Nora-Go
Copy link
Author

Nora-Go commented Feb 6, 2024

Ah! :D I used a virtual environment with python 3.7 so pip installed version 1.1.0. With python 3.9 it installed version 1.2.0.
The problem with installing is now solved! Thank you @faebstn96 !

However, I have a problem with dimensions: It seems that the layer does not take batches and the channel dimension? But only single images? At least the check in line 247 "ValueError(f"Spatial dimension ({len_input}) must match initialized len(spatial_sigma).")" does not make sense to me. Should I make a new issue? And here or at monai? :)

@faebstn96
Copy link
Owner

Did you consider the slightly changed layer input specification for the MONAI layer? There, the initial spatial sigmas are input as a vector with two or three entries depending on how many dimensions your filtered image has. This is due to adapting MONAI conventions.

The check just makes sure that the number of provided spatial sigmas matches the number of spatial input dimensions.

Currently, batch dimensions >1 are supported. However, the channel dimension is restricted to a single channel due to open questions on the handling/mixing of multiple image channels.

Hope that helped :)

@Nora-Go
Copy link
Author

Nora-Go commented Feb 14, 2024 •
edited
Loading

I input the spatial_sigmas as tuple with two entries, as I have 2D images.
I raised an issue at MONAI's GitHub: Project-MONAI/MONAI#7457 (which seems to be a duplicate of Project-MONAI/MONAI#7444)
The error arises at the check (line 246 in filtering.py):
if self.len_spatial_sigma != len_input: raise ValueError(f"Spatial dimension ({len_input}) must match initialized len(spatial_sigma).")
len_input is 4, as I have a batch, and a channel dimension and 2D images.
self.len_spatial_sigma is 2, as I input 2 spatial_sigmas.

I already tried simply commenting the check out - but then I get issues with the C++ extensions: Project-MONAI/MONAI#7464
This problem is probably nothing you can help me with, but it makes me unable to use MONAI at the moment. Still, if you have any idea how to proceed (either with MONAI or your Git), I'd be very glad to hear it!

Thanks a lot for your help! It is much appreciated!

@lucasdevries
Copy link

@Nora-Go Did you manage to get the C++ extension installation working?

@Nora-Go
Copy link
Author

Nora-Go commented Apr 18, 2024

@lucasdevries No :) I don't use MONAI now. I made it work by installing this package here on the cluster. My problem was that I was trying to install it on a node that did not have a GPU. So solution was to go onto a node that has a GPU, allocate the GPU and then install ist.
Hope this helps!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@lucasdevries @faebstn96 @Nora-Go