Update PyO3 to version 0.18.0 #811

lahwaacz · 2025-02-02T12:51:42Z

I could actually compile this on Arch Linux with Python 3.13 😄 Now going to test it...

This is not the newest pyo3 version, but the latest supported by the latest pyo3-asyncio. We also need to bump the abi3 version, let's go with abi3-py39 since Python 3.8 is already end-of-life.

Otherwise there is a conflict between the `hyperqueue` crate and the module defined by the `#[pymodule]` attribute.

Since [pyo3 0.16.0](https://pyo3.rs/main/changelog): PyType::is_subclass, PyErr::is_instance and PyAny::is_instance now operate run-time type object instead of a type known at compile-time. The old behavior is still available as PyType::is_subclass_of, PyErr::is_instance_of and PyAny::is_instance_of. PyO3/pyo3#1985

Since [pyo3 0.16.0](https://pyo3.rs/main/changelog): Rename some methods on PyErr (the old names are just marked deprecated for now): - pytype -> get_type - pvalue -> value (and deprecate equivalent instance) - ptraceback -> traceback - from_instance -> from_value - into_instance -> into_value PyO3/pyo3#2026

lahwaacz · 2025-02-02T14:18:05Z

Ok so I have some failed tests with Python 3.13 on Arch Linux:

=================================== FAILURES ===================================
_______________________________ test_empty_stdio _______________________________
[gw10] linux -- Python 3.13.1 /build/hyperqueue-git/src/hyperqueue/test-env/bin/python

env = <tests.conftest.HqEnv object at 0x7845cb71d8c0>, ids = {'1'}
target_states = {'finished'}, commands = ['job', 'list', '--all']
state_index = 2, kwargs = {}
check = <function wait_for_state.<locals>.check at 0x7845cb7e6c00>

    def wait_for_state(
        env,
        ids: Union[int, List[int]],
        target_states: Union[str, List[str]],
        commands: List[str],
        state_index: int,
        **kwargs,
    ):
        if isinstance(ids, int):
            ids = {str(ids)}
        else:
            ids = set(str(id) for id in ids)

        if isinstance(target_states, str):
            target_states = {target_states.lower()}
        else:
            target_states = set(state.lower() for state in target_states)

        last_table = None

        def check():
            nonlocal last_table

            table = env.command(commands, as_table=True)
            last_table = table
            items = [row for row in table if row[0].lstrip("*") in ids]
            return len(items) >= len(ids) and all(j[state_index].lower() in target_states for j in items)

        try:
>           wait_until(check, **kwargs)

/build/hyperqueue-git/src/hyperqueue/tests/utils/wait.py:53:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

fn = <function wait_for_state.<locals>.check at 0x7845cb7e6c00>, sleep_s = 0.2
timeout_s = 10

    def wait_until(fn, sleep_s=0.2, timeout_s=DEFAULT_TIMEOUT):
        end = time.time() + timeout_s

        while time.time() < end:
            value = fn()
            if value is not None and value is not False:
                return value
            time.sleep(sleep_s)
>       raise TimeoutException(f"Wait timeouted after {timeout_s} seconds")
E       tests.utils.wait.TimeoutException: Wait timeouted after 10 seconds

/build/hyperqueue-git/src/hyperqueue/tests/utils/wait.py:21: TimeoutException

During handling of the above exception, another exception occurred:

hq_env = <tests.conftest.HqEnv object at 0x7845cb71d8c0>

    def test_empty_stdio(hq_env: HqEnv):
        (job, client) = prepare_job_client(hq_env)

        def foo():
            print("Hello")
            print("Hello", file=sys.stderr)

        job.function(
            foo,
            stdout=None,
            stderr=None,
        )
        submitted_job = client.submit(job)
>       wait_for_job_state(hq_env, submitted_job.id, "FINISHED")

/build/hyperqueue-git/src/hyperqueue/tests/pyapi/test_job.py:92:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/build/hyperqueue-git/src/hyperqueue/tests/utils/wait.py:61: in wait_for_job_state
    wait_for_state(env, ids, target_states, ["job", "list", "--all"], 2, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

env = <tests.conftest.HqEnv object at 0x7845cb71d8c0>, ids = {'1'}
target_states = {'finished'}, commands = ['job', 'list', '--all']
state_index = 2, kwargs = {}
check = <function wait_for_state.<locals>.check at 0x7845cb7e6c00>

    def wait_for_state(
        env,
        ids: Union[int, List[int]],
        target_states: Union[str, List[str]],
        commands: List[str],
        state_index: int,
        **kwargs,
    ):
        if isinstance(ids, int):
            ids = {str(ids)}
        else:
            ids = set(str(id) for id in ids)

        if isinstance(target_states, str):
            target_states = {target_states.lower()}
        else:
            target_states = set(state.lower() for state in target_states)

        last_table = None

        def check():
            nonlocal last_table

            table = env.command(commands, as_table=True)
            last_table = table
            items = [row for row in table if row[0].lstrip("*") in ids]
            return len(items) >= len(ids) and all(j[state_index].lower() in target_states for j in items)

        try:
            wait_until(check, **kwargs)
        except TimeoutException as e:
            if last_table is not None:
>               raise Exception(f"{e}, most recent table:\n{last_table}")
E               Exception: Wait timeouted after 10 seconds, most recent table:
E               ID | Name | State | Tasks
E               1 |  | FAILED | 1

/build/hyperqueue-git/src/hyperqueue/tests/utils/wait.py:56: Exception
---------------------------- Captured stdout setup -----------------------------

Working dir /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_empty_stdio0
----------------------------- Captured stdout call -----------------------------
Starting process server with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_empty_stdio0/server.out
Starting process worker1 with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_empty_stdio0/worker1.out
True
ID | State | Hostname | Resources | Manager | Manager Job ID
ID | State | Hostname | Resources | Manager | Manager Job ID
1 | RUNNING | worker1 | cpus 1 | None | N/A
_________________________ test_submit_python_prologue __________________________
[gw6] linux -- Python 3.13.1 /build/hyperqueue-git/src/hyperqueue/test-env/bin/python

env = <tests.conftest.HqEnv object at 0x78d4f540b350>, ids = {'1'}
target_states = {'finished'}, commands = ['job', 'list', '--all']
state_index = 2, kwargs = {}
check = <function wait_for_state.<locals>.check at 0x78d4f42bbba0>

    def wait_for_state(
        env,
        ids: Union[int, List[int]],
        target_states: Union[str, List[str]],
        commands: List[str],
        state_index: int,
        **kwargs,
    ):
        if isinstance(ids, int):
            ids = {str(ids)}
        else:
            ids = set(str(id) for id in ids)

        if isinstance(target_states, str):
            target_states = {target_states.lower()}
        else:
            target_states = set(state.lower() for state in target_states)

        last_table = None

        def check():
            nonlocal last_table

            table = env.command(commands, as_table=True)
            last_table = table
            items = [row for row in table if row[0].lstrip("*") in ids]
            return len(items) >= len(ids) and all(j[state_index].lower() in target_states for j in items)

        try:
>           wait_until(check, **kwargs)

/build/hyperqueue-git/src/hyperqueue/tests/utils/wait.py:53:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

fn = <function wait_for_state.<locals>.check at 0x78d4f42bbba0>, sleep_s = 0.2
timeout_s = 10

    def wait_until(fn, sleep_s=0.2, timeout_s=DEFAULT_TIMEOUT):
        end = time.time() + timeout_s

        while time.time() < end:
            value = fn()
            if value is not None and value is not False:
                return value
            time.sleep(sleep_s)
>       raise TimeoutException(f"Wait timeouted after {timeout_s} seconds")
E       tests.utils.wait.TimeoutException: Wait timeouted after 10 seconds

/build/hyperqueue-git/src/hyperqueue/tests/utils/wait.py:21: TimeoutException

During handling of the above exception, another exception occurred:

hq_env = <tests.conftest.HqEnv object at 0x78d4f540b350>

    def test_submit_python_prologue(hq_env: HqEnv):
        init_sh = Path("init.sh")
        init_sh.write_text("""export ABC=xyz""")

        hq_env.start_server()
        hq_env.start_worker()
        client = Client(hq_env.server_dir, python_env=PythonEnv(prologue=f"source {init_sh.resolve()}"))

        def body():
            print(os.environ.get("ABC"))

        job = Job()
        job.function(body, stdout="out")
        submitted_job = client.submit(job)
>       wait_for_job_state(hq_env, submitted_job.id, "FINISHED")

/build/hyperqueue-git/src/hyperqueue/tests/pyapi/test_function.py:43:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/build/hyperqueue-git/src/hyperqueue/tests/utils/wait.py:61: in wait_for_job_state
    wait_for_state(env, ids, target_states, ["job", "list", "--all"], 2, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

env = <tests.conftest.HqEnv object at 0x78d4f540b350>, ids = {'1'}
target_states = {'finished'}, commands = ['job', 'list', '--all']
state_index = 2, kwargs = {}
check = <function wait_for_state.<locals>.check at 0x78d4f42bbba0>

    def wait_for_state(
        env,
        ids: Union[int, List[int]],
        target_states: Union[str, List[str]],
        commands: List[str],
        state_index: int,
        **kwargs,
    ):
        if isinstance(ids, int):
            ids = {str(ids)}
        else:
            ids = set(str(id) for id in ids)

        if isinstance(target_states, str):
            target_states = {target_states.lower()}
        else:
            target_states = set(state.lower() for state in target_states)

        last_table = None

        def check():
            nonlocal last_table

            table = env.command(commands, as_table=True)
            last_table = table
            items = [row for row in table if row[0].lstrip("*") in ids]
            return len(items) >= len(ids) and all(j[state_index].lower() in target_states for j in items)

        try:
            wait_until(check, **kwargs)
        except TimeoutException as e:
            if last_table is not None:
>               raise Exception(f"{e}, most recent table:\n{last_table}")
E               Exception: Wait timeouted after 10 seconds, most recent table:
E               ID | Name | State | Tasks
E               1 |  | FAILED | 1

/build/hyperqueue-git/src/hyperqueue/tests/utils/wait.py:56: Exception
---------------------------- Captured stdout setup -----------------------------

Working dir /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_submit_python_prologue0
----------------------------- Captured stdout call -----------------------------
Starting process server with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_submit_python_prologue0/server.out
Starting process worker1 with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_submit_python_prologue0/worker1.out
True
ID | State | Hostname | Resources | Manager | Manager Job ID
1 | RUNNING | worker1 | cpus 1 | None | N/A
_________________________ test_submit_pyfunction_fail __________________________
[gw6] linux -- Python 3.13.1 /build/hyperqueue-git/src/hyperqueue/test-env/bin/python

hq_env = <tests.conftest.HqEnv object at 0x78d4f4308050>

    def test_submit_pyfunction_fail(hq_env: HqEnv):
        (job, client) = prepare_job_client(hq_env)

        def body():
            raise Exception("MyException")

        job.function(body, stderr="err")
        job_id = client.submit(job)
        client.wait_for_jobs([job_id], raise_on_error=False)
        errors = client.get_failed_tasks(job_id)
        assert list(errors.keys()) == [0]
>       assert errors[0].error.endswith('    raise Exception("MyException")\nException: MyException\n')
E       assert False
E        +  where False = <built-in method endswith of str object at 0x78d4f4269830>('    raise Exception("MyException")\nException: MyException\n')
E        +    where <built-in method endswith of str object at 0x78d4f4269830> = 'Error: Program terminated with exit code 1'.endswith
E        +      where 'Error: Program terminated with exit code 1' = FailedTaskContext(error='Error: Program terminated with exit code 1', cwd='/tmp/pytest-of-builduser/pytest-0/popen-gw6/test_submit_pyfunction_fail0', stdout='/tmp/pytest-of-builduser/pytest-0/popen-gw6/test_submit_pyfunction_fail0/0.stdout', stderr='/tmp/pytest-of-builduser/pytest-0/popen-gw6/test_submit_pyfunction_fail0/err').error

/build/hyperqueue-git/src/hyperqueue/tests/pyapi/test_function.py:58: AssertionError
---------------------------- Captured stdout setup -----------------------------

Working dir /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_submit_pyfunction_fail0
----------------------------- Captured stdout call -----------------------------
Starting process server with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_submit_pyfunction_fail0/server.out
Starting process worker1 with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_submit_pyfunction_fail0/worker1.out
True
ID | State | Hostname | Resources | Manager | Manager Job ID
1 | RUNNING | worker1 | cpus 1 | None | N/A
----------------------------- Captured stderr call -----------------------------
  0%|                               | 0/1 [00:01<?, ?task/s, 1/1 job completed]
___________________________ test_function_resources ____________________________
[gw6] linux -- Python 3.13.1 /build/hyperqueue-git/src/hyperqueue/test-env/bin/python

hq_env = <tests.conftest.HqEnv object at 0x78d4f430d650>

    def test_function_resources(hq_env: HqEnv):
        (job, client) = prepare_job_client(hq_env)

        job.function(lambda: 1, resources=ResourceRequest(cpus="1"))
        job.function(lambda: 1, resources=ResourceRequest(cpus="2"))
        job.function(lambda: 1, resources=ResourceRequest(cpus="all"))
        submitted_job = client.submit(job)
        time.sleep(2.0)

        table = hq_env.command(["task", "list", str(submitted_job.id)], as_table=True)
>       assert table.get_column_value("State") == ["FINISHED", "WAITING", "FINISHED"]
E       AssertionError: assert ['FAILED', 'CANCELED', 'FAILED'] == ['FINISHED', 'WAITING', 'FINISHED']
E
E         At index 0 diff: 'FAILED' != 'FINISHED'
E
E         Full diff:
E           [
E         -     'FINISHED',
E         ?        ^^^^
E         +     'FAILED',
E         ?       + ^
E         -     'WAITING',
E         +     'CANCELED',
E         -     'FINISHED',
E         ?        ^^^^
E         +     'FAILED',
E         ?       + ^
E           ]

/build/hyperqueue-git/src/hyperqueue/tests/pyapi/test_function.py:72: AssertionError
---------------------------- Captured stdout setup -----------------------------

Working dir /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_function_resources0
----------------------------- Captured stdout call -----------------------------
Starting process server with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_function_resources0/server.out
Starting process worker1 with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_function_resources0/worker1.out
True
ID | State | Hostname | Resources | Manager | Manager Job ID
ID | State | Hostname | Resources | Manager | Manager Job ID
1 | RUNNING | worker1 | cpus 1 | None | N/A
_____________________________ test_default_workdir _____________________________
[gw6] linux -- Python 3.13.1 /build/hyperqueue-git/src/hyperqueue/test-env/bin/python

hq_env = <tests.conftest.HqEnv object at 0x78d4f430de50>

    def test_default_workdir(hq_env: HqEnv):
        workdir = Path("foo").resolve()
        (job, client) = prepare_job_client(hq_env, default_workdir=workdir)

        def fn():
            assert os.getcwd() == str(workdir)

        job.function(fn)
        job_id = client.submit(job)
>       client.wait_for_jobs([job_id])

/build/hyperqueue-git/src/hyperqueue/tests/pyapi/test_function.py:84:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <hyperqueue.client.Client object at 0x78d4f4e16fd0>
jobs = [SubmittedJob(job=<hyperqueue.job.Job object at 0x78d4f42cc510>, id=1)]
raise_on_error = True

    def wait_for_jobs(self, jobs: Sequence[SubmittedJob], raise_on_error=True) -> bool:
        """Returns True if all tasks were successfully finished"""

        job_ids = tuple(job.id for job in jobs)
        job_ids_str = ",".join(str(id) for id in job_ids)
        if len(jobs) > 1:
            job_ids_str = "{" + job_ids_str + "}"
        logging.info(f"Waiting for {pluralize('job', len(jobs))} {job_ids_str} to finish")

        callback = create_progress_callback()

        failed_jobs = self.connection.wait_for_jobs(job_ids, callback)
        if failed_jobs and raise_on_error:
            failed_tasks = self.connection.get_failed_tasks(failed_jobs)
            job_map = {job.id: job.job for job in jobs}
>           raise FailedJobsException(failed_tasks, job_map)
E           hyperqueue.client.FailedJobsException: The following tasks of job `1` have failed:
E           Task fn/0 (id=0):
E           Error: Program terminated with exit code 1
E           You can find more information here:
E           Working directory: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_workdir0/foo
E           Stdout: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_workdir0/foo/0.stdout
E           Stderr: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_workdir0/foo/0.stderr

/build/hyperqueue-git/src/hyperqueue/test-env/lib/python3.13/site-packages/hyperqueue/client.py:104: FailedJobsException
---------------------------- Captured stdout setup -----------------------------

Working dir /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_workdir0
----------------------------- Captured stdout call -----------------------------
Starting process server with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_workdir0/server.out
Starting process worker1 with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_workdir0/worker1.out
True
ID | State | Hostname | Resources | Manager | Manager Job ID
ID | State | Hostname | Resources | Manager | Manager Job ID
1 | RUNNING | worker1 | cpus 1 | None | N/A
----------------------------- Captured stderr call -----------------------------
  0%|                               | 0/1 [00:01<?, ?task/s, 1/1 job completed]
____________________________ test_submit_pyfunction ____________________________
[gw9] linux -- Python 3.13.1 /build/hyperqueue-git/src/hyperqueue/test-env/bin/python

env = <tests.conftest.HqEnv object at 0x7175c5c10140>, ids = {'1'}
target_states = {'finished'}, commands = ['job', 'list', '--all']
state_index = 2, kwargs = {}
check = <function wait_for_state.<locals>.check at 0x7175c5a48fe0>

    def wait_for_state(
        env,
        ids: Union[int, List[int]],
        target_states: Union[str, List[str]],
        commands: List[str],
        state_index: int,
        **kwargs,
    ):
        if isinstance(ids, int):
            ids = {str(ids)}
        else:
            ids = set(str(id) for id in ids)

        if isinstance(target_states, str):
            target_states = {target_states.lower()}
        else:
            target_states = set(state.lower() for state in target_states)

        last_table = None

        def check():
            nonlocal last_table

            table = env.command(commands, as_table=True)
            last_table = table
            items = [row for row in table if row[0].lstrip("*") in ids]
            return len(items) >= len(ids) and all(j[state_index].lower() in target_states for j in items)

        try:
>           wait_until(check, **kwargs)

/build/hyperqueue-git/src/hyperqueue/tests/utils/wait.py:53:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

fn = <function wait_for_state.<locals>.check at 0x7175c5a48fe0>, sleep_s = 0.2
timeout_s = 10

    def wait_until(fn, sleep_s=0.2, timeout_s=DEFAULT_TIMEOUT):
        end = time.time() + timeout_s

        while time.time() < end:
            value = fn()
            if value is not None and value is not False:
                return value
            time.sleep(sleep_s)
>       raise TimeoutException(f"Wait timeouted after {timeout_s} seconds")
E       tests.utils.wait.TimeoutException: Wait timeouted after 10 seconds

/build/hyperqueue-git/src/hyperqueue/tests/utils/wait.py:21: TimeoutException

During handling of the above exception, another exception occurred:

hq_env = <tests.conftest.HqEnv object at 0x7175c5c10140>

    def test_submit_pyfunction(hq_env: HqEnv):
        (job, client) = prepare_job_client(hq_env)

        def body(a, b, c):
            print(f"stdout {a} {b} {c}")

        job.function(body, args=(1, 2), kwargs={"c": 3}, stdout="out", stderr="err")
        submitted_job = client.submit(job)
>       wait_for_job_state(hq_env, submitted_job.id, "FINISHED")

/build/hyperqueue-git/src/hyperqueue/tests/pyapi/test_function.py:24:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/build/hyperqueue-git/src/hyperqueue/tests/utils/wait.py:61: in wait_for_job_state
    wait_for_state(env, ids, target_states, ["job", "list", "--all"], 2, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

env = <tests.conftest.HqEnv object at 0x7175c5c10140>, ids = {'1'}
target_states = {'finished'}, commands = ['job', 'list', '--all']
state_index = 2, kwargs = {}
check = <function wait_for_state.<locals>.check at 0x7175c5a48fe0>

    def wait_for_state(
        env,
        ids: Union[int, List[int]],
        target_states: Union[str, List[str]],
        commands: List[str],
        state_index: int,
        **kwargs,
    ):
        if isinstance(ids, int):
            ids = {str(ids)}
        else:
            ids = set(str(id) for id in ids)

        if isinstance(target_states, str):
            target_states = {target_states.lower()}
        else:
            target_states = set(state.lower() for state in target_states)

        last_table = None

        def check():
            nonlocal last_table

            table = env.command(commands, as_table=True)
            last_table = table
            items = [row for row in table if row[0].lstrip("*") in ids]
            return len(items) >= len(ids) and all(j[state_index].lower() in target_states for j in items)

        try:
            wait_until(check, **kwargs)
        except TimeoutException as e:
            if last_table is not None:
>               raise Exception(f"{e}, most recent table:\n{last_table}")
E               Exception: Wait timeouted after 10 seconds, most recent table:
E               ID | Name | State | Tasks
E               1 |  | FAILED | 1

/build/hyperqueue-git/src/hyperqueue/tests/utils/wait.py:56: Exception
---------------------------- Captured stdout setup -----------------------------

Working dir /tmp/pytest-of-builduser/pytest-0/popen-gw9/test_submit_pyfunction0
----------------------------- Captured stdout call -----------------------------
Starting process server with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw9/test_submit_pyfunction0/server.out
Starting process worker1 with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw9/test_submit_pyfunction0/worker1.out
True
ID | State | Hostname | Resources | Manager | Manager Job ID
ID | State | Hostname | Resources | Manager | Manager Job ID
1 | RUNNING | worker1 | cpus 1 | None | N/A
_______________________________ test_default_env _______________________________
[gw6] linux -- Python 3.13.1 /build/hyperqueue-git/src/hyperqueue/test-env/bin/python

hq_env = <tests.conftest.HqEnv object at 0x78d4f42d16d0>

    def test_default_env(hq_env: HqEnv):
        env = {"FOO": "1"}
        (job, client) = prepare_job_client(hq_env, default_env=env)

        def fn():
            assert os.environ["FOO"] == "1"

        job.function(fn)
        job_id = client.submit(job)
>       client.wait_for_jobs([job_id])

/build/hyperqueue-git/src/hyperqueue/tests/pyapi/test_function.py:96:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <hyperqueue.client.Client object at 0x78d4f4e17a80>
jobs = [SubmittedJob(job=<hyperqueue.job.Job object at 0x78d4f42ccfc0>, id=1)]
raise_on_error = True

    def wait_for_jobs(self, jobs: Sequence[SubmittedJob], raise_on_error=True) -> bool:
        """Returns True if all tasks were successfully finished"""

        job_ids = tuple(job.id for job in jobs)
        job_ids_str = ",".join(str(id) for id in job_ids)
        if len(jobs) > 1:
            job_ids_str = "{" + job_ids_str + "}"
        logging.info(f"Waiting for {pluralize('job', len(jobs))} {job_ids_str} to finish")

        callback = create_progress_callback()

        failed_jobs = self.connection.wait_for_jobs(job_ids, callback)
        if failed_jobs and raise_on_error:
            failed_tasks = self.connection.get_failed_tasks(failed_jobs)
            job_map = {job.id: job.job for job in jobs}
>           raise FailedJobsException(failed_tasks, job_map)
E           hyperqueue.client.FailedJobsException: The following tasks of job `1` have failed:
E           Task fn/0 (id=0):
E           Error: Program terminated with exit code 1
E           You can find more information here:
E           Working directory: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env0
E           Stdout: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env0/0.stdout
E           Stderr: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env0/0.stderr

/build/hyperqueue-git/src/hyperqueue/test-env/lib/python3.13/site-packages/hyperqueue/client.py:104: FailedJobsException
---------------------------- Captured stdout setup -----------------------------

Working dir /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env0
----------------------------- Captured stdout call -----------------------------
Starting process server with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env0/server.out
Starting process worker1 with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env0/worker1.out
True
ID | State | Hostname | Resources | Manager | Manager Job ID
1 | RUNNING | worker1 | cpus 1 | None | N/A
----------------------------- Captured stderr call -----------------------------

______________________ test_default_env_overwrite_default ______________________
[gw6] linux -- Python 3.13.1 /build/hyperqueue-git/src/hyperqueue/test-env/bin/python

hq_env = <tests.conftest.HqEnv object at 0x78d4f42d3020>

    def test_default_env_overwrite_default(hq_env: HqEnv):
        env = {"FOO": "1"}
        (job, client) = prepare_job_client(hq_env, default_env=env)

        def fn():
            assert os.environ["FOO"] == "2"

        job.function(fn, env=dict(FOO="2"))
        job_id = client.submit(job)
>       client.wait_for_jobs([job_id])

/build/hyperqueue-git/src/hyperqueue/tests/pyapi/test_function.py:108:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <hyperqueue.client.Client object at 0x78d4f54d57f0>
jobs = [SubmittedJob(job=<hyperqueue.job.Job object at 0x78d4f54d4290>, id=1)]
raise_on_error = True

    def wait_for_jobs(self, jobs: Sequence[SubmittedJob], raise_on_error=True) -> bool:
        """Returns True if all tasks were successfully finished"""

        job_ids = tuple(job.id for job in jobs)
        job_ids_str = ",".join(str(id) for id in job_ids)
        if len(jobs) > 1:
            job_ids_str = "{" + job_ids_str + "}"
        logging.info(f"Waiting for {pluralize('job', len(jobs))} {job_ids_str} to finish")

        callback = create_progress_callback()

        failed_jobs = self.connection.wait_for_jobs(job_ids, callback)
        if failed_jobs and raise_on_error:
            failed_tasks = self.connection.get_failed_tasks(failed_jobs)
            job_map = {job.id: job.job for job in jobs}
>           raise FailedJobsException(failed_tasks, job_map)
E           hyperqueue.client.FailedJobsException: The following tasks of job `1` have failed:
E           Task fn/0 (id=0):
E           Error: Program terminated with exit code 1
E           You can find more information here:
E           Working directory: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env_overwrite_def0
E           Stdout: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env_overwrite_def0/0.stdout
E           Stderr: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env_overwrite_def0/0.stderr

/build/hyperqueue-git/src/hyperqueue/test-env/lib/python3.13/site-packages/hyperqueue/client.py:104: FailedJobsException
---------------------------- Captured stdout setup -----------------------------

Working dir /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env_overwrite_def0
----------------------------- Captured stdout call -----------------------------
Starting process server with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env_overwrite_def0/server.out
Starting process worker1 with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env_overwrite_def0/worker1.out
True
ID | State | Hostname | Resources | Manager | Manager Job ID
ID | State | Hostname | Resources | Manager | Manager Job ID
1 | RUNNING | worker1 | cpus 1 | None | N/A
----------------------------- Captured stderr call -----------------------------

_____________________________ test_task_priorities _____________________________
[gw10] linux -- Python 3.13.1 /build/hyperqueue-git/src/hyperqueue/test-env/bin/python

hq_env = <tests.conftest.HqEnv object at 0x7845cb796350>

    def test_task_priorities(hq_env: HqEnv):
        """Submits tasks with different randomly shuffled priorities and
        checks that tasks are executed in this order
        """
        (job, client) = prepare_job_client(hq_env)

        priorities = list(range(-20, 20))
        random.seed(123123)
        random.shuffle(priorities)
        for p in priorities:
            if random.randint(0, 1) == 0:
                job.program(bash("echo Hello"), priority=p)
            else:
                job.function(lambda: 0, priority=p)
        job_id = client.submit(job)
>       client.wait_for_jobs([job_id])

/build/hyperqueue-git/src/hyperqueue/tests/pyapi/test_job.py:210:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <hyperqueue.client.Client object at 0x7845cb71ee00>
jobs = [SubmittedJob(job=<hyperqueue.job.Job object at 0x7845cb71e690>, id=1)]
raise_on_error = True

    def wait_for_jobs(self, jobs: Sequence[SubmittedJob], raise_on_error=True) -> bool:
        """Returns True if all tasks were successfully finished"""

        job_ids = tuple(job.id for job in jobs)
        job_ids_str = ",".join(str(id) for id in job_ids)
        if len(jobs) > 1:
            job_ids_str = "{" + job_ids_str + "}"
        logging.info(f"Waiting for {pluralize('job', len(jobs))} {job_ids_str} to finish")

        callback = create_progress_callback()

        failed_jobs = self.connection.wait_for_jobs(job_ids, callback)
        if failed_jobs and raise_on_error:
            failed_tasks = self.connection.get_failed_tasks(failed_jobs)
            job_map = {job.id: job.job for job in jobs}
>           raise FailedJobsException(failed_tasks, job_map)
E           hyperqueue.client.FailedJobsException: The following tasks of job `1` have failed:
E           Task <lambda>/27 (id=27):
E           Error: Program terminated with exit code 1
E           You can find more information here:
E           Working directory: /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_task_priorities0
E           Stdout: /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_task_priorities0/27.stdout
E           Stderr: /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_task_priorities0/27.stderr
E           Task <lambda>/9 (id=9):
E           Error: Program terminated with exit code 1
E           You can find more information here:
E           Working directory: /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_task_priorities0
E           Stdout: /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_task_priorities0/9.stdout
E           Stderr: /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_task_priorities0/9.stderr

/build/hyperqueue-git/src/hyperqueue/test-env/lib/python3.13/site-packages/hyperqueue/client.py:104: FailedJobsException
---------------------------- Captured stdout setup -----------------------------

Working dir /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_task_priorities0
----------------------------- Captured stdout call -----------------------------
Starting process server with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_task_priorities0/server.out
Starting process worker1 with logfile /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_task_priorities0/worker1.out
True
ID | State | Hostname | Resources | Manager | Manager Job ID
ID | State | Hostname | Resources | Manager | Manager Job ID
1 | RUNNING | worker1 | cpus 1 | None | N/A
----------------------------- Captured stderr call -----------------------------
  0%|                              | 0/40 [00:01<?, ?task/s, 1/1 job completed]
=============================== inline snapshot ================================
INFO: inline-snapshot was disabled because you used xdist
=========================== short test summary info ============================
FAILED tests/pyapi/test_job.py::test_empty_stdio - Exception: Wait timeouted after 10 seconds, most recent table:
ID | Name | State | Tasks
1 |  | FAILED | 1
FAILED tests/pyapi/test_function.py::test_submit_python_prologue - Exception: Wait timeouted after 10 seconds, most recent table:
ID | Name | State | Tasks
1 |  | FAILED | 1
FAILED tests/pyapi/test_function.py::test_submit_pyfunction_fail - assert False
 +  where False = <built-in method endswith of str object at 0x78d4f4269830>('    raise Exception("MyException")\nException: MyException\n')
 +    where <built-in method endswith of str object at 0x78d4f4269830> = 'Error: Program terminated with exit code 1'.endswith
 +      where 'Error: Program terminated with exit code 1' = FailedTaskContext(error='Error: Program terminated with exit code 1', cwd='/tmp/pytest-of-builduser/pytest-0/popen-gw6/test_submit_pyfunction_fail0', stdout='/tmp/pytest-of-builduser/pytest-0/popen-gw6/test_submit_pyfunction_fail0/0.stdout', stderr='/tmp/pytest-of-builduser/pytest-0/popen-gw6/test_submit_pyfunction_fail0/err').error
FAILED tests/pyapi/test_function.py::test_function_resources - AssertionError: assert ['FAILED', 'CANCELED', 'FAILED'] == ['FINISHED', 'WAITING', 'FINISHED']

  At index 0 diff: 'FAILED' != 'FINISHED'

  Full diff:
    [
  -     'FINISHED',
  ?        ^^^^
  +     'FAILED',
  ?       + ^
  -     'WAITING',
  +     'CANCELED',
  -     'FINISHED',
  ?        ^^^^
  +     'FAILED',
  ?       + ^
    ]
FAILED tests/pyapi/test_function.py::test_default_workdir - hyperqueue.client.FailedJobsException: The following tasks of job `1` have failed:
Task fn/0 (id=0):
Error: Program terminated with exit code 1
You can find more information here:
Working directory: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_workdir0/foo
Stdout: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_workdir0/foo/0.stdout
Stderr: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_workdir0/foo/0.stderr
FAILED tests/pyapi/test_function.py::test_submit_pyfunction - Exception: Wait timeouted after 10 seconds, most recent table:
ID | Name | State | Tasks
1 |  | FAILED | 1
FAILED tests/pyapi/test_function.py::test_default_env - hyperqueue.client.FailedJobsException: The following tasks of job `1` have failed:
Task fn/0 (id=0):
Error: Program terminated with exit code 1
You can find more information here:
Working directory: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env0
Stdout: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env0/0.stdout
Stderr: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env0/0.stderr
FAILED tests/pyapi/test_function.py::test_default_env_overwrite_default - hyperqueue.client.FailedJobsException: The following tasks of job `1` have failed:
Task fn/0 (id=0):
Error: Program terminated with exit code 1
You can find more information here:
Working directory: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env_overwrite_def0
Stdout: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env_overwrite_def0/0.stdout
Stderr: /tmp/pytest-of-builduser/pytest-0/popen-gw6/test_default_env_overwrite_def0/0.stderr
FAILED tests/pyapi/test_job.py::test_task_priorities - hyperqueue.client.FailedJobsException: The following tasks of job `1` have failed:
Task <lambda>/27 (id=27):
Error: Program terminated with exit code 1
You can find more information here:
Working directory: /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_task_priorities0
Stdout: /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_task_priorities0/27.stdout
Stderr: /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_task_priorities0/27.stderr
Task <lambda>/9 (id=9):
Error: Program terminated with exit code 1
You can find more information here:
Working directory: /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_task_priorities0
Stdout: /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_task_priorities0/9.stdout
Stderr: /tmp/pytest-of-builduser/pytest-0/popen-gw10/test_task_priorities0/9.stderr
============= 9 failed, 378 passed, 4 skipped, 1 xfailed in 30.91s =============

None of the /tmp/ paths exist when pytest is done, any ideas?

lahwaacz · 2025-02-02T17:42:00Z

Actually, the test failures were caused by me not activating the venv where I installed the pyhq bindings, so the server used a different Python environment than the client. When I did it correctly, all tests passed 🎉

Kobzol · 2025-02-03T17:20:53Z

Thanks! I wanted to keep Python 3.8 for as long as possible, because HPC clusters quite often have ancient Python versions. But 3.8 is old enough now that I think that we can just let it go.

I pushed one more commit that updated docs & the changelog. Before, we claimed compatibility with Python 3.6, but I think that 3.8 was more realistic.

lahwaacz added 5 commits February 2, 2025 14:03

bump pyo3 version to 0.18.0

aa65567

This is not the newest pyo3 version, but the latest supported by the latest pyo3-asyncio. We also need to bump the abi3 version, let's go with abi3-py39 since Python 3.8 is already end-of-life.

update Cargo.lock

ed262cd

use ::hyperqueue to refer to the external crate unambiguously

f4031e7

Otherwise there is a conflict between the `hyperqueue` crate and the module defined by the `#[pymodule]` attribute.

lahwaacz force-pushed the pyhq-update branch from 41969f3 to 9b8ba1c Compare February 2, 2025 13:04

Update minimum required Python version and modify changelog

65cce5c

Kobzol approved these changes Feb 3, 2025

View reviewed changes

Kobzol enabled auto-merge (rebase) February 3, 2025 17:21

Kobzol merged commit f83f892 into It4innovations:main Feb 3, 2025
9 checks passed

lahwaacz deleted the pyhq-update branch February 4, 2025 06:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update PyO3 to version 0.18.0 #811

Update PyO3 to version 0.18.0 #811

lahwaacz commented Feb 2, 2025

lahwaacz commented Feb 2, 2025 •

edited

Loading

lahwaacz commented Feb 2, 2025 •

edited

Loading

Kobzol commented Feb 3, 2025 •

edited

Loading

Update PyO3 to version 0.18.0 #811

Update PyO3 to version 0.18.0 #811

Conversation

lahwaacz commented Feb 2, 2025

lahwaacz commented Feb 2, 2025 • edited Loading

lahwaacz commented Feb 2, 2025 • edited Loading

Kobzol commented Feb 3, 2025 • edited Loading

lahwaacz commented Feb 2, 2025 •

edited

Loading

lahwaacz commented Feb 2, 2025 •

edited

Loading

Kobzol commented Feb 3, 2025 •

edited

Loading