v1.11.0

Changes

R and Python package dependencies updated to use TileDB-SOMA version 1.8.0

What's Changed

• [misc] Fix CL_0002494 in geneformer pipeline by @pablo-gar in #1000
• [docs] Add embeddings API to docs by @martinkim0 in #1005
• [docs] Fix docstrings for various functions by @martinkim0 in #1002
• [builder] fix incorrect filtering of tissue type by @bkmartinjr in #1010
• [misc] add support for var embeddings to contrib curation tool by @bkmartinjr in #995
• [builder] add explicit validation check for primary tissue cells by @bkmartinjr in #1015
• [builder] update uberon and tissue_general mapping to schema 4 standard by @bkmartinjr in #1018
• [docs] Update copyright notice by @ebezzi in #978
• [docs] misc docsite adjustments by @ebezzi in #1020
• [profiler] Early version of profiler harness by @beroy in #959
• [docs] Geneformer docs errata by @mlin in #1003
• [python] [r] Upgrade to tiledbsoma 1.8.0 by @ebezzi in #1046

New Contributors

• @beroy made their first contribution in #959

Full Changelog: v1.10.2...v1.11.0

v1.10.2

What's Changed

• [python] use new indexer in highly variable genes algorithm by @bkmartinjr in #973
• [python] updates to the SCVI model training pipeline by @ebezzi in #939
• [python] remove upper bound on numpy version dependency by @martinkim0 in #980
• [builder] use tiledbsoma indexer for builder validation pass by @bkmartinjr in #988
• [python] update to use tiledbsoma indexer for get_embedding by @bkmartinjr in #991

Full Changelog: v1.10.1...v1.10.2

v1.10.1

What's Changed

Fixed an issue where the Python package name showed up as cellxgene_census instead of the intended cellxgene-census.

Full Changelog: v1.10.0...v1.10.1

v1.10.0

Census S3 H5AD bucket now is limited to data present in Census

Beginning with the 2024-01-29 Census weekly build, the Census build will only include H5AD files contributing data to companion SOMA objects. Previously, Census builds included all H5ADs in the CELLxGENE corpus. Put another way, if a dataset fails to contribute a cell to the Census, the dataset H5AD will not be present in the Census build h5ad directory.

Census uses CELLxGENE schema version 4.0.0

The Census is now using CELLxGENE schema version 4.0.0.

Additions

• [docs] Adds biological demonstration notebooks by @atarashansky in #875
• [docs] Add citation notebooks by @pablo-gar in #916
• [builder] schema 4.0 by @bkmartinjr in #872

Breaking changes

• [python] remove deprecated X_sparse_iter API by @bkmartinjr in #962

What’s Changed

• [python] [R] upgrade tiledbsoma to 1.7.0 by @bkmartinjr in #969
• [docs] add no-sign-request flag to model download in tutorials by @pablo-gar in #956
• [builder] remove unused H5AD files from build by @bkmartinjr in #946
• [python] Syntactic sugar for specifying tiledb_config options by @atolopko-czi in #902
• [python] Make default soma context available for customization by @atolopko-czi in #895

New Contributors

• @atarashansky made their first contribution in #875
• @seve made their first contribution in #846

Full Changelog: v1.9.1...v1.10.0

v1.9.1

Changes

Minor updates in preparation for releasing December LTS Census and contributed embeddings.

Deprecations

• [python] cellxgene_census.experimental.util.X_sparse_iter is deprecated and will be removed in a future release. Please use the new SOMA blockwise iterators available on the SparseNDMatrix class.

What's Changed

• [builder] Update release cleanup code to support relative URIs by @ebezzi in #888
• [python] Geneformer WDL: pin TileDB and Census versions by @mlin in #889
• [misc] contrib embedding curation fixes by @bkmartinjr in #890
• [r] Update R docs in preparation for API release 1.9.1 by @pablo-gar in #893

Full Changelog: v1.9.0...v1.9.1

v1.9.0

Changes

• R and Python package dependencies updated to use TileDB-SOMA version 1.6.0

Deprecations

• [python] cellxgene_census.experimental.util.X_sparse_iter is deprecated and will be removed in a future release. Please use the new SOMA blockwise iterators available on the SparseNDMatrix class.

What's Changed

• [misc] Bump aiohttp from 3.8.6 to 3.9.0 in /tools/cellxgene_census_builder by @dependabot in #868
• [builder] Absolute uris should use the default mirror by @ebezzi in #863
• [python] GeneformerTokenizer maintenance by @mlin in #861
• [python] migrate GeneformerTokenizer to blockwise iterator by @mlin in #866
• [python] [r] Fix tests that had the mirror name hardcoded by @ebezzi in #869
• [builder] Fix builder release manifest validation to not check for absolute uri by @ebezzi in #882
• [python] upgrade to tiledbsoma and add obsm to get_anndata by @ebezzi in #885
• [misc] Contrib embeddings by @bkmartinjr in #853
• [r] add obsm_layers to get_seurat() and get_single_cell_experiment() by @mlin in #887
• [python] Geneformer fine-tuning scripts by @mlin in #805
• [python] scVI pipeline by @ebezzi in #787

Full Changelog: v1.8.0...v1.9.0

v1.8.0

Changes

• R and Python package dependencies updated to use TileDB-SOMA version 1.5.1

Deprecations

• [python] cellxgene_census.experimental.util.X_sparse_iter is deprecated as of this release and will be removed in a future release. Please use the new SOMA blockwise iterators available on the SparseNDMatrix class.

What's Changed

• [misc] update GHA action and pre-commit versions by @bkmartinjr in #829
• [misc] fix installation of geneformer's transitive dependency in py-dependency-check.yml by @mlin in #833
• [misc] update actions/stale by @bkmartinjr in #834
• [misc] prep for 1.8 release by @bkmartinjr in #852
• [python] update hvg acceptance tests to account for numerical instability by @bkmartinjr in #838
• [builder] Sync whole bucket by @ebezzi in #819
• [builder] Acceptance/Builder tests misc fixes by @ebezzi in #855
• [docs] Update bucket name in AWS Open Data docpage by @ebezzi in #851
• [docs] DOC: Update cellxgene_census_aws_open_data.md typo by @kamilkrukowski in #858
• [r] add r-dependency-check.yml nightly test of current release package by @mlin in #769
• [r] Fix acceptance tests by @pablo-gar in #859

New Contributors

• @kamilkrukowski made their first contribution in #858

Full Changelog: v1.7.0...v1.8.0

v1.7.0

Additions

• [python] add support for distributed PyTorch by @atolopko-czi in #699
• [python] Add X_layers argument to cellxgene_census.get_anndata() by @pablo-
• [python] Geneformer Dataset builder by @mlin in #762
• [python][misc] GHA for Python dependency checks by @bkmartinjr in #763
• [python] GeneformerTokenizer speed optimizations by @mlin in #784
• [python] add multi-batch-key support to highly-variable gene finder by @bkmartinjr in #788
• [python] Add exclude_zeros to pp.mean_variance by @ebezzi in #731
• [python] Add pytorch global shuffling by @atolopko-czi in #775
• [docs] Add "what's new?" article for out-of-core methods by @pablo-gar in #765
• [docs] Add "what's new?" article for out-of-core methods by @pablo-gar in #765
• [docs] Create newsletter_september2023.md by @MaximilianLombardo in #732
• [docs] Adds normalized layer banner to landing page by @pablo-gar in #808
• [docs] editorial changes to october news by @ebezzi in #810
• [docs] Add documentation for AWS by @pablo-gar in #823
• [builder] Add additional debug logging by @ebezzi in #770
• [misc] Add authentication to py-dependency-check workflow by @bkmartinjr in #768
• [misc] LTS / package compatibility check by @bkmartinjr in #797

Fixes

• [python] Fix import in example for geneformer tokenizer by @pablo-gar in #780
• [python] remove unnecessary dependency pin by @bkmartinjr in #802
• [python] temporarily skip geneformer tests by @mlin in #818
• [python] add PEP 561 py.typed marker by @bkmartinjr in #822
• [python] Fix HVG crasher by @bkmartinjr in #821
• [docs] Fix minor typo in blog post about out-of-core functionalities by @aaronwolen in #774
• [docs] Fix minor typo in blog post about out-of-core functionalities by
• [docs] fix index docs for AWS page by @pablo-gar in #825
• [docs] fix typos AWS page by @pablo-gar in #826
• [builder] thread usage tuning by @bkmartinjr in #759
• [builder] Fix additional builder logging by @ebezzi in #771
  gar in #761
• [builder] perform all consolidation modes by @bkmartinjr in #773
• [builder] Restore h5ad replication by @ebezzi in #783
• [misc] shorten Census LTS compatibility workflow name by @bkmartinjr in #803

What's Changed

• [python] limit runtime of --expensive unit tests by @bkmartinjr in #779
• [python][builder] Explicit LTS releases in releases.json by @atolopko-czi in #737
• [r] Bump r version to 1.7.0 by @pablo-gar in #828
• [builder] use TileDB core 2.16 by @bkmartinjr in #800
• [builder] update tiledbsoma pin to 1.4.4 by @bkmartinjr in #812

New Contributors

• @aaronwolen made their first contribution in #774
• @MaximilianLombardo made their first contribution in #732

Full Changelog: v1.6.0...v1.7.0

v1.6.0

Additions

Census APIs now support access to data mirrors, geographical data mirrors are coming soon!

• [python] Mirrors support by @ebezzi in #552
• [r] support mirrors & relative URI by @mlin in #719

Other:

• [r] test framework for first-time R package installation by @mlin in #687
• [r] more vignettes by @mlin in #680
• [r][docs] minor docs updates by @mlin in #696
• [builder] Explicit log message tests by @bkmartinjr in #720
• [builder] add builder dmesg logging on error by @atolopko-czi in #745
• [docs] Add normalized layer information to user-facing schema overview by @pablo-gar in #753
• [docs] Add markdown linting to pre-commit by @bkmartinjr in #721

Fixes

• [python] numerical precision improvements in Census builder by @bkmartinjr in #716
• [python] revert Mac M1/M2 cert fix on open_soma() by @atolopko-czi in #746
• [docs] Add SingleCellExperiment mention to landing page. by @pablo-gar in #725
• [docs] pin Sphinx version by @ebezzi in #729
• [docs] add explicit filtering for None git tags by @ebezzi in #712
• [docs] fix markdown lint by @bkmartinjr in #735
• [builder] Resolve CI failure caused by X count matrix with zero-sum rows by @bkmartinjr in #742

What's Changed

• [python] get_census_version_directory returns sorted elements by @ebezzi in #728
• [python] Bump tiledbsoma pinned version to 1.4.3 by @pablo-gar in #738
• [python] Pin tiledbsoma~=1.4.3 in Python API by @pablo-gar in #756
• [r][docs] Update version to 1.5.2 and doc-site by @pablo-gar in #743
• [r][docs] Bump R API version to 1.6.0 by @pablo-gar in #757
• [docs][python] Reinforce knowledge of is_primary_data in all notebooks by @pablo-gar in
• [docs] Update API release process to support both R and Python releases by @pablo-gar in #739
• [builder][docs] update to CxG schema version 3.1.0 by @bkmartinjr in #740
  #754
• Bump gitpython from 3.1.31 to 3.1.32 in /tools/cellxgene_census_builder by @dependabot in #713
• feat: Add a Github Action that automatically closes stale PRs. by @maniarathi in #724
• fix: override default so that issues are not marked as stale by @maniarathi in #730

New Contributors

• @maniarathi made their first contribution in #724

Full Changelog: v1.5.1...v1.6.0

v1.5.1

What's Changed

• [docs] Add "What's new page" to doc-site and R V1 news article (#671) by @pablo-gar in #671
• [docs] Update publish date on new article 20230808-r_api_release.md by @pablo-gar in #701
• [r] Fixes source links in R doc site by @pablo-gar in #702
• [python] update tiledbsoma version to 1.4.0 by @pablo-gar in #709
• [docs] Add R tutorials link to landing page by @pablo-gar in #704
• [r] Bump R package version to 1.5.1 for release by @pablo-gar in #710

Full Changelog: v1.5.0...v1.5.1