Unpin setproctitle, fixes #37727

[timkpaine]

Why are these changes needed?

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[mattip]

@richardliaw this is needed for ray + python 3.11 on conda-forge. Could we get some eyes on it?

[richardliaw]

could you merge master back in?

[mattip]

master was merged back in, there are still a number of failing CI jobs. Is that normal?

[mattip]

Once again this is out of date with master.

[mattip]

FWIW, the conda-forge 2.7.1 release incorporates this PR, which enables python3.11 packaging.

[mattip]

Merging with master is an ongoing process. Until this can get some review it probably is not worth the effort. FWIW, this is working well on conda-forge, where it has already been adopted. Is there a reason not to do this?

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

• If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[anyscalesam]

@mattip is this specific to conda-forge only and not pip?

[mattip]

The version 1.2.2 in setup.py has wheels for python<3.10. pip install works currently by installing from source, which requires compiling the c-extension on each install. It would be much cleaner to merge this PR and use wheels.

[mattip]

I rebased the PR off master to make this mergable.

[timkpaine]

Thanks @mattip!

[anyscalesam]

this part of the stack sort of in no mans land; doesn't map directly to any eng team.... @aslonnie @can-anyscale thoughts on merging this?

[can-anyscale]

accept to unblock, i think the change is safe;

if the code owners do not accept in the next few days I can help merge it

[timkpaine]

rebased on latest master just to make sure its still ok

[anyscalesam]

@can-anyscale please just merge it

[can-anyscale]

yes, merging, sorry for the delay

[can-anyscale]

Could be breaking this test https://github.com/orgs/anyscale/projects/76/views/1?pane=issue&itemId=64249695 and is blocking the release. I'll revert if confirmed in #45539. Thankks.

[mattip]

I don't have access to that project page. What test is failing? Why isn't it failing in the CI here?

[can-anyscale]

@mattip ah yes sorry, it is this issue #45537

it's a macos test darwin://python/ray/tests:test_job so it doesn't run on CI

[timkpaine]

I'll leave it to the anyscale team to ensure this eventually makes it in

[mattip]

The problem is here

ray/python/ray/tests/test_job.py

Line 291 in c94140a

assert submission_job["Entrypoint"] == list2cmdline(commands)

At this point list2cmdline(commands) is

'python /var/folders/4r/67pc2qss79q97y_7jpftm8tw0000gn/T/tmpei8y07tm --flag'

and job["Entrypoint"] is

'"/opt/homebrew/Cellar/[email protected]/3.11.2_1/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python \
/var/folders/4r/67pc2qss79q97y_7jpftm8tw0000gn/T/tmpei8y07tm --fla" "" LC_PAPER=en_AU.UTF-8'

The long name for python has something to do with me running the tests in a virtual environment, I have a fix to allow tests to run locally, but that is not the problem.
Note --flag is missing a g. This somehow has something to do with setproctitle: the older version does not cut off the g.

[mattip]

This is probable dvarrazzo/py-setproctitle#118, which is still open.