fails to install Python modules when user pip configuration has a custom site-packages path

[BenWibking]

Running on Setonix with the default modules, I get:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'CppHeaderParser'
Running command pip3 install --prefix=/scratch/pawsey0807/bwibking/rocm-build-gcc-nolibc/build-deps/pypackages cppheaderparser argparse virtualenv wheel lit
Collecting cppheaderparser
  Using cached CppHeaderParser-2.7.4.tar.gz (54 kB)
  Preparing metadata (setup.py) ... done
Collecting argparse
  Using cached argparse-1.4.0-py2.py3-none-any.whl (23 kB)
Collecting virtualenv
  Using cached virtualenv-20.24.5-py3-none-any.whl (3.7 MB)
Collecting wheel
  Using cached wheel-0.41.2-py3-none-any.whl (64 kB)
Collecting lit
  Using cached lit-16.0.6.tar.gz (153 kB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Collecting setuptools
        Using cached setuptools-68.2.2-py3-none-any.whl (807 kB)
      Installing collected packages: setuptools
      ERROR: Cannot set --home and --prefix together
      WARNING: You are using pip version 22.0.4; however, version 23.2.1 is available.
      You should consider upgrading via the '/opt/cray/pe/python/3.9.13.1/bin/python -m pip install --upgrade pip' command.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
WARNING: You are using pip version 22.0.4; however, version 23.2.1 is available.
You should consider upgrading via the '/opt/cray/pe/python/3.9.13.1/bin/python -m pip install --upgrade pip' command.
Error running a command: pip3 install --prefix=/scratch/pawsey0807/bwibking/rocm-build-gcc-nolibc/build-deps/pypackages cppheaderparser argparse virtualenv wheel lit

[dipietrantonio]

Hi @BenWibking, I cannot reproduce this error. As the message says, it seems that the --home flag is passed somehow. Is it possible that it is set in some of your custom configurations? Also notice that @pelahi has attempted building later versions of ROCm, but it has been proving harder than expected..

@pelahi have you observed this issue recently?

[dipietrantonio]

Hi @BenWibking any updates on this?

[BenWibking]

I stopped working on this, since rocm/5.4.3 is now available for PrgEnv-cray.

[BenWibking]

This issue appears when I set a different user site-packages directory using pip.conf:

bwibking@setonix-03:~> cat .config/pip/pip.conf
[global]
target = /software/projects/pawsey0807/bwibking/site-packages

The build scripts should create a separate virtual env for the Python packages needed for the build to avoid this issue.