Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always write indptr as int64 #1262

Closed
wants to merge 2 commits into from

Merge branch 'main' into indptr-int64

4f56751
Select commit
Loading
Failed to load commit list.
Closed

Always write indptr as int64 #1262

Merge branch 'main' into indptr-int64
4f56751
Select commit
Loading
Failed to load commit list.
Azure Pipelines / scverse.anndata failed Jan 29, 2024 in 9m 22s

Build #20240129.2 had test failures

3 errors / 1 warnings

Details

Tests

  • Failed: 630 (5.45%)
  • Passed: 9,000 (77.84%)
  • Other: 1,932 (16.71%)
  • Total: 11,562

Annotations

Check failure on line 16733 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / scverse.anndata

Build log #L16733 

Bash exited with code '1'.

Check failure on line 16731 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / scverse.anndata

Build log #L16731 

Bash exited with code '1'.

Check failure on line 16760 in Build log

See this annotation in the file changed.

@azure-pipelines azure-pipelines / scverse.anndata

Build log #L16760 

Bash exited with code '1'.

Check failure on line 1 in test_backed_indexing[array_subset-array_subset-h5ad]

See this annotation in the file changed.

@azure-pipelines azure-pipelines / scverse.anndata

test_backed_indexing[array_subset-array_subset-h5ad] 

ValueError: Output dtype not compatible with inputs.
Raw output 
ondisk_equivalent_adata = (AnnData object with n_obs × n_vars = 50 × 50, AnnData object with n_obs × n_vars = 50 × 50 backed at '/tmp/pytest-of-...ject with n_obs × n_vars = 50 × 50 backed at '/tmp/pytest-of-vsts/pytest-0/test_backed_indexing_array_sub0/dense.h5ad')
subset_func = <function array_subset at 0x7f48d7481800>
subset_func2 = <function array_subset at 0x7f48d7481800>

    def test_backed_indexing(
        ondisk_equivalent_adata: tuple[AnnData, AnnData, AnnData, AnnData],
        subset_func,
        subset_func2,
    ):
        csr_mem, csr_disk, csc_disk, dense_disk = ondisk_equivalent_adata
    
        obs_idx = subset_func(csr_mem.obs_names)
        var_idx = subset_func2(csr_mem.var_names)
    
>       assert_equal(csr_mem[obs_idx, var_idx].X, csr_disk[obs_idx, var_idx].X)

anndata/tests/test_backed_sparse.py:113: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
anndata/_core/anndata.py:678: in X
    X = _subset(X, (self._oidx, self._vidx))
/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/functools.py:909: in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
anndata/_core/sparse_dataset.py:597: in subset_sparsedataset
    return d[subset_idx]
anndata/_core/sparse_dataset.py:427: in __getitem__
    sub = mtx[row, col]
/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/scipy/sparse/_index.py:85: in __getitem__
    return self._get_columnXarray(row[:,0], col.ravel())
/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/scipy/sparse/_compressed.py:691: in _get_columnXarray
    return self._major_index_fancy(major)._minor_index_fancy(minor)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <50x50 sparse matrix of type '<class 'numpy.float64'>'
	with 250 stored elements in Compressed Sparse Row format>
idx = array([48, 49, 25, 33, 35, 41, 22, 31,  1, 17, 14, 16, 29, 21, 28, 11,  6,
       46,  3, 37,  5,  8, 19, 36,  4, 26, 20, 47, 13, 30, 44, 15, 38, 39,
       27, 45, 42, 32, 34, 23, 12])

    def _major_index_fancy(self, idx):
        """Index along the major axis where idx is an array of ints.
        """
        idx_dtype = self.indices.dtype
        indices = np.asarray(idx, dtype=idx_dtype).ravel()
    
        _, N = self._swap(self.shape)
        M = len(indices)
        new_shape = self._swap((M, N))
        if M == 0:
            return self.__class__(new_shape, dtype=self.dtype)
    
        row_nnz = self.indptr[indices + 1] - self.indptr[indices]
        idx_dtype = self.indices.dtype
        res_indptr = np.zeros(M+1, dtype=idx_dtype)
        np.cumsum(row_nnz, out=res_indptr[1:])
    
        nnz = res_indptr[-1]
        res_indices = np.empty(nnz, dtype=idx_dtype)
        res_data = np.empty(nnz, dtype=self.dtype)
>       csr_row_index(M, indices, self.indptr, self.indices, self.data,
                      res_indices, res_data)
E       ValueError: Output dtype not compatible with inputs.

/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/scipy/sparse/_compressed.py:713: ValueError

Check failure on line 1 in test_backed_indexing[array_subset-array_subset-zarr]

See this annotation in the file changed.

@azure-pipelines azure-pipelines / scverse.anndata

test_backed_indexing[array_subset-array_subset-zarr] 

ValueError: Output dtype not compatible with inputs.
Raw output 
ondisk_equivalent_adata = (AnnData object with n_obs × n_vars = 50 × 50, AnnData object with n_obs × n_vars = 50 × 50, AnnData object with n_obs × n_vars = 50 × 50, AnnData object with n_obs × n_vars = 50 × 50)
subset_func = <function array_subset at 0x7f48d7481800>
subset_func2 = <function array_subset at 0x7f48d7481800>

    def test_backed_indexing(
        ondisk_equivalent_adata: tuple[AnnData, AnnData, AnnData, AnnData],
        subset_func,
        subset_func2,
    ):
        csr_mem, csr_disk, csc_disk, dense_disk = ondisk_equivalent_adata
    
        obs_idx = subset_func(csr_mem.obs_names)
        var_idx = subset_func2(csr_mem.var_names)
    
>       assert_equal(csr_mem[obs_idx, var_idx].X, csr_disk[obs_idx, var_idx].X)

anndata/tests/test_backed_sparse.py:113: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
anndata/_core/anndata.py:683: in X
    _subset(self._adata_ref.X, (self._oidx, self._vidx)),
/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/functools.py:909: in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
anndata/_core/sparse_dataset.py:597: in subset_sparsedataset
    return d[subset_idx]
anndata/_core/sparse_dataset.py:427: in __getitem__
    sub = mtx[row, col]
/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/scipy/sparse/_index.py:85: in __getitem__
    return self._get_columnXarray(row[:,0], col.ravel())
/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/scipy/sparse/_compressed.py:691: in _get_columnXarray
    return self._major_index_fancy(major)._minor_index_fancy(minor)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <50x50 sparse matrix of type '<class 'numpy.float64'>'
	with 250 stored elements in Compressed Sparse Row format>
idx = array([ 9, 34, 23])

    def _major_index_fancy(self, idx):
        """Index along the major axis where idx is an array of ints.
        """
        idx_dtype = self.indices.dtype
        indices = np.asarray(idx, dtype=idx_dtype).ravel()
    
        _, N = self._swap(self.shape)
        M = len(indices)
        new_shape = self._swap((M, N))
        if M == 0:
            return self.__class__(new_shape, dtype=self.dtype)
    
        row_nnz = self.indptr[indices + 1] - self.indptr[indices]
        idx_dtype = self.indices.dtype
        res_indptr = np.zeros(M+1, dtype=idx_dtype)
        np.cumsum(row_nnz, out=res_indptr[1:])
    
        nnz = res_indptr[-1]
        res_indices = np.empty(nnz, dtype=idx_dtype)
        res_data = np.empty(nnz, dtype=self.dtype)
>       csr_row_index(M, indices, self.indptr, self.indices, self.data,
                      res_indices, res_data)
E       ValueError: Output dtype not compatible with inputs.

/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/scipy/sparse/_compressed.py:713: ValueError

Check failure on line 1 in test_backed_indexing[array_subset-array_int_subset-h5ad]

See this annotation in the file changed.

@azure-pipelines azure-pipelines / scverse.anndata

test_backed_indexing[array_subset-array_int_subset-h5ad] 

ValueError: Output dtype not compatible with inputs.
Raw output 
ondisk_equivalent_adata = (AnnData object with n_obs × n_vars = 50 × 50, AnnData object with n_obs × n_vars = 50 × 50 backed at '/tmp/pytest-of-...ject with n_obs × n_vars = 50 × 50 backed at '/tmp/pytest-of-vsts/pytest-0/test_backed_indexing_array_sub6/dense.h5ad')
subset_func = <function array_subset at 0x7f48d7481800>
subset_func2 = <function array_int_subset at 0x7f48d74818a0>

    def test_backed_indexing(
        ondisk_equivalent_adata: tuple[AnnData, AnnData, AnnData, AnnData],
        subset_func,
        subset_func2,
    ):
        csr_mem, csr_disk, csc_disk, dense_disk = ondisk_equivalent_adata
    
        obs_idx = subset_func(csr_mem.obs_names)
        var_idx = subset_func2(csr_mem.var_names)
    
>       assert_equal(csr_mem[obs_idx, var_idx].X, csr_disk[obs_idx, var_idx].X)

anndata/tests/test_backed_sparse.py:113: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
anndata/_core/anndata.py:678: in X
    X = _subset(X, (self._oidx, self._vidx))
/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/functools.py:909: in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
anndata/_core/sparse_dataset.py:597: in subset_sparsedataset
    return d[subset_idx]
anndata/_core/sparse_dataset.py:427: in __getitem__
    sub = mtx[row, col]
/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/scipy/sparse/_index.py:85: in __getitem__
    return self._get_columnXarray(row[:,0], col.ravel())
/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/scipy/sparse/_compressed.py:691: in _get_columnXarray
    return self._major_index_fancy(major)._minor_index_fancy(minor)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <50x50 sparse matrix of type '<class 'numpy.float64'>'
	with 250 stored elements in Compressed Sparse Row format>
idx = array([28, 23, 37, 30, 35, 10, 38, 18, 25, 26, 27, 32, 15,  7,  5, 49, 21,
       33, 42, 46, 40, 43, 11,  8, 20,  4, 44, 39])

    def _major_index_fancy(self, idx):
        """Index along the major axis where idx is an array of ints.
        """
        idx_dtype = self.indices.dtype
        indices = np.asarray(idx, dtype=idx_dtype).ravel()
    
        _, N = self._swap(self.shape)
        M = len(indices)
        new_shape = self._swap((M, N))
        if M == 0:
            return self.__class__(new_shape, dtype=self.dtype)
    
        row_nnz = self.indptr[indices + 1] - self.indptr[indices]
        idx_dtype = self.indices.dtype
        res_indptr = np.zeros(M+1, dtype=idx_dtype)
        np.cumsum(row_nnz, out=res_indptr[1:])
    
        nnz = res_indptr[-1]
        res_indices = np.empty(nnz, dtype=idx_dtype)
        res_data = np.empty(nnz, dtype=self.dtype)
>       csr_row_index(M, indices, self.indptr, self.indices, self.data,
                      res_indices, res_data)
E       ValueError: Output dtype not compatible with inputs.

/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/scipy/sparse/_compressed.py:713: ValueError

Check failure on line 1 in test_backed_indexing[array_subset-array_int_subset-zarr]

See this annotation in the file changed.

@azure-pipelines azure-pipelines / scverse.anndata

test_backed_indexing[array_subset-array_int_subset-zarr] 

ValueError: Output dtype not compatible with inputs.
Raw output 
ondisk_equivalent_adata = (AnnData object with n_obs × n_vars = 50 × 50, AnnData object with n_obs × n_vars = 50 × 50, AnnData object with n_obs × n_vars = 50 × 50, AnnData object with n_obs × n_vars = 50 × 50)
subset_func = <function array_subset at 0x7f48d7481800>
subset_func2 = <function array_int_subset at 0x7f48d74818a0>

    def test_backed_indexing(
        ondisk_equivalent_adata: tuple[AnnData, AnnData, AnnData, AnnData],
        subset_func,
        subset_func2,
    ):
        csr_mem, csr_disk, csc_disk, dense_disk = ondisk_equivalent_adata
    
        obs_idx = subset_func(csr_mem.obs_names)
        var_idx = subset_func2(csr_mem.var_names)
    
>       assert_equal(csr_mem[obs_idx, var_idx].X, csr_disk[obs_idx, var_idx].X)

anndata/tests/test_backed_sparse.py:113: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
anndata/_core/anndata.py:683: in X
    _subset(self._adata_ref.X, (self._oidx, self._vidx)),
/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/functools.py:909: in wrapper
    return dispatch(args[0].__class__)(*args, **kw)
anndata/_core/sparse_dataset.py:597: in subset_sparsedataset
    return d[subset_idx]
anndata/_core/sparse_dataset.py:427: in __getitem__
    sub = mtx[row, col]
/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/scipy/sparse/_index.py:85: in __getitem__
    return self._get_columnXarray(row[:,0], col.ravel())
/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/scipy/sparse/_compressed.py:691: in _get_columnXarray
    return self._major_index_fancy(major)._minor_index_fancy(minor)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <50x50 sparse matrix of type '<class 'numpy.float64'>'
	with 250 stored elements in Compressed Sparse Row format>
idx = array([47,  7, 21, 34, 42, 48, 15, 45, 40, 24, 43,  4, 49, 12])

    def _major_index_fancy(self, idx):
        """Index along the major axis where idx is an array of ints.
        """
        idx_dtype = self.indices.dtype
        indices = np.asarray(idx, dtype=idx_dtype).ravel()
    
        _, N = self._swap(self.shape)
        M = len(indices)
        new_shape = self._swap((M, N))
        if M == 0:
            return self.__class__(new_shape, dtype=self.dtype)
    
        row_nnz = self.indptr[indices + 1] - self.indptr[indices]
        idx_dtype = self.indices.dtype
        res_indptr = np.zeros(M+1, dtype=idx_dtype)
        np.cumsum(row_nnz, out=res_indptr[1:])
    
        nnz = res_indptr[-1]
        res_indices = np.empty(nnz, dtype=idx_dtype)
        res_data = np.empty(nnz, dtype=self.dtype)
>       csr_row_index(M, indices, self.indptr, self.indices, self.data,
                      res_indices, res_data)
E       ValueError: Output dtype not compatible with inputs.

/opt/hostedtoolcache/Python/3.11.7/x64/lib/python3.11/site-packages/scipy/sparse/_compressed.py:713: ValueError
View more details on Azure Pipelines