test_packed_selection_nminusone_dak is flaky (FileNotFoundError)

[GaetanLepage]

Describe the bug
The test_packed_selection_nminusone_dak test appears to be flaky on all platforms.
It unreliably fails with:

>           os.remove("nminusone.npz")
E           FileNotFoundError: [Errno 2] No such file or directory: 'nminusone.npz'

Maybe this is caused by some specificity of the nix builder. In this case we can surely skip this specific test.
Yet, I wanted to mention this observation here.

To Reproduce
Steps to reproduce the behavior:

1. pytest

Expected behavior
All tests succeed.

Output

=================================== FAILURES ===================================
__________________ test_packed_selection_nminusone_dak[True] ___________________
[gw2] darwin -- Python 3.12.8 /nix/store/fpmkmdzgd1q7kqadc7czcjdhjj7bsc0i-python3-3.12.8/bin/python3.12

optimization_enabled = True

    @pytest.mark.parametrize("optimization_enabled", [True, False])
    def test_packed_selection_nminusone_dak(optimization_enabled):
        import dask
        import dask_awkward as dak

        from coffea.analysis_tools import PackedSelection

        events = dakevents

        selection = PackedSelection()

        with dask.config.set({"awkward.optimization.enabled": optimization_enabled}):
            twoelectron = dak.num(events.Electron) == 2
            nomuon = dak.num(events.Muon) == 0
            leadpt20 = dak.any(events.Electron.pt >= 20.0, axis=1) | dak.any(
                events.Muon.pt >= 20.0, axis=1
            )

            selection.add_multiple(
                {
                    "twoElectron": twoelectron,
                    "noMuon": nomuon,
                    "leadPt20": leadpt20,
                }
            )

            assert selection.names == ["twoElectron", "noMuon", "leadPt20"]

            with pytest.raises(
                ValueError,
                match="All arguments must be strings that refer to the names of existing selections",
            ):
                selection.nminusone("twoElectron", "nonexistent")
            nminusone = selection.nminusone("twoElectron", "noMuon", "leadPt20")

            labels, nev, masks = nminusone.result()

            assert labels == [
                "initial",
                "N - twoElectron",
                "N - noMuon",
                "N - leadPt20",
                "N",
            ]

            assert list(dask.compute(*nev)) == [
                dak.num(events, axis=0).compute(),
                dak.num(events[nomuon & leadpt20], axis=0).compute(),
                dak.num(events[twoelectron & leadpt20], axis=0).compute(),
                dak.num(events[twoelectron & nomuon], axis=0).compute(),
                dak.num(events[twoelectron & nomuon & leadpt20], axis=0).compute(),
            ]

            for mask, truth in zip(
                masks,
                [
                    nomuon & leadpt20,
                    twoelectron & leadpt20,
                    twoelectron & nomuon,
                    twoelectron & nomuon & leadpt20,
                ],
            ):
                assert np.all(mask.compute() == truth.compute())

            nminusone.to_npz("nminusone.npz", compressed=False).compute()
            with np.load("nminusone.npz") as file:
                assert np.all(file["labels"] == labels)
                assert np.all(file["nev"] == list(dask.compute(*nev)))
                assert np.all(file["masks"] == list(dask.compute(*masks)))
            os.remove("nminusone.npz")

            nminusone.to_npz("nminusone.npz", compressed=True).compute()
            with np.load("nminusone.npz") as file:
                assert np.all(file["labels"] == labels)
                assert np.all(file["nev"] == list(dask.compute(*nev)))
                assert np.all(file["masks"] == list(dask.compute(*masks)))
>           os.remove("nminusone.npz")
E           FileNotFoundError: [Errno 2] No such file or directory: 'nminusone.npz'

tests/test_analysis_tools.py:875: FileNotFoundError

Desktop (please complete the following information):

• OS: NixOS
• Version: 2025.1.0

Additional context
Maintenance of the coffea package in nixpkgs.

[NickCao]

The line right before os.remove("nminusone.npz") is np.load("nminusone.npz"), if numpy could load that file, how come os.remove cannot find it?

[GaetanLepage]

The line right before os.remove("nminusone.npz") is np.load("nminusone.npz"), if numpy could load that file, how come os.remove cannot find it?

Good question. And again, sometimes it passes fine, some others it fails...