build: unpin numpy + use Python 3.9 in CI

[anakin87]

Related Issues

• fixes numpy 2 compatible #8474
• in chore: pin numpy<2; tenacity!=8.4.0 #7876, we pinned numpy<2 to avoid failures in several core integrations.

Proposed Changes:

• unpin numpy: its usage in Haystack is compatible with both 1.x and 2.x
• test haystack with numpy>=2 in CI
• use Python 3.9 in the CI: Python 3.8 already reached End Of Life and we need 3.9 to use numpy>=2

How did you test it?

CI

Core integrations

In general, I think that if some of the core integrations break we will find out because on the nightly tests of core integrations we also test with Haystack main branch. In that case, we can pin numpy specifically where required.

To be on the safe side, I ran the following script locally (with Python 3.10, Ubuntu 22.04) from core integrations root directory and got no failures.

#!/bin/bash

set -e

for dir in ./integrations/*/; do
  if [ -d "$dir" ]; then
    echo "Processing $(basename "$dir")"
    
    cd "$dir" || exit
    
    hatch env prune
    hatch run pip install git+https://github.com/deepset-ai/haystack.git@unpin-numpy
    hatch run cov-retry -m "not integration"
    
    cd - > /dev/null
  fi
done

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
• I documented my code
• I ran pre-commit hooks and fixed any issue

[coveralls]

Pull Request Test Coverage Report for Build 11502851488

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 2 unchanged lines in 2 files lost coverage.
• Overall coverage increased (+0.02%) to 90.548%

---

Files with Coverage Reduction	New Missed Lines	%
core/component/component.py	1	97.58%
core/type_utils.py	1	95.12%

Totals
Change from base Build 11501290533:	0.02%
Covered Lines:	7558
Relevant Lines:	8347

---

💛 - Coveralls

[anakin87]

The previous spacy dependencies required numpy<2, while these require numpy>=2

[anakin87]

this pin is not strictly required (see the comment on spacy), but I have included it for clarity

[silvanocerza]

Nice 👍