Normal pip install of pixl CLI results in export directory being wrongly determined

[jeremyestein]

Definition of Done / Acceptance Criteria

• The system test should pass whether the pixl CLI packages are installed with or without the -e option to pip install

Details and Comments

See main.yml workflow file:

  system-test:
    steps:
      - name: Install Python dependencies
        # The -e option is required here due to the way the
        # code determines the export directory. See issue #318.
        run: |
          pip install -e pixl_core/
          pip install -e cli/[test]
          pip install -e pytest-pixl/

If the -e options are removed, the system test will fail.

For reference, the root docker-compose.yml:

services:
  ehr-api:
    volumes:
      - ${PWD}/exports:/run/exports

This defines the export directory as in the root of the source code working tree on the GAE, as is intended.

The problem comes with how this directory is determined in the CLI code (cli/src/pixl_cli/_io.py):

HOST_EXPORT_ROOT_DIR = Path(__file__).parents[3] / "exports"

If you have done a "normal" install (pip install cli/), this location will be somewhere in your python libs directory, NOT in your source tree as expected. It's only when installed in editable mode, which is intended for dev use only, that these directories match up.

I guess we can keep installing with -e for now, it just seems kind of broken if your packages can't be installed in the standard way?

The most obvious way I see to fix it is to make it a config option, so it can be passed to docker compose so it can be mounted correctly and the CLI can pick up on it.

[jstutters]

See here for a way to determine the location of within-package files that should work whether or not a package is installed as editable: https://github.com/UCLH-Foundry/PIXL/blob/027f794c90046ef7f1ed3c3a5be75619b992ebbf/pytest-pixl/src/pytest_pixl/dicom.py#L36

[jeremyestein]

See here for a way to determine the location of within-package files that should work whether or not a package is installed as editable:

https://github.com/UCLH-Foundry/PIXL/blob/027f794c90046ef7f1ed3c3a5be75619b992ebbf/pytest-pixl/src/pytest_pixl/dicom.py#L36

It will determine the location correctly (as does __file__ I think?), but that still isn't the right thing to do.
Example of editable vs non-editable installs, for reference:

>>> importlib.resources.files("importlib")
PosixPath('/Users/jeremystein/miniconda3/envs/pixl/lib/python3.10/importlib')
>>> importlib.resources.files("pytest_pixl")
PosixPath('/Users/jeremystein/PIXL/PIXL/pytest-pixl/src/pytest_pixl')

We always want to define the exports dir relative to the source directory, regardless of install type of our packages.

(I don't like putting application data into the source tree, but that's a separate discussion)