Anndata format output

[rob-p]

Following up on COMBINE-lab/alevin-fry#151 raised by @ilan-gold, we are planning to first add AnnData output as an option in simpleaf, and then to potentially port it "upstream" directly into alevin-fry (possibly deprecating and dropping the old output format).

This commit 20ab78e includes the first draft implementation of this idea. It creates an anndata format version of the output quant.h5ad that lives directly alongside the existing CSR matrix. Creation of this matrix is enabled by passing --anndata-out as a flag to the simpleaf quant command. I may also add an additional flag to remove the CSR matrix as well to avoid the redundancy, but for now, they live side-by-side.

There are a few dependencies that I am waiting to stabilize (e.g. ureq 3.0 and a fix for this bug in flate2), but after that, we'll be ready to cut a simpleaf release with this feature!

P.S. @an-altosian / @DongzeHE : We should decide about the conversion in the case of visium data. Since we have extra information in the row labels, we should decide if we ignore that, put it in the anndata annotation, etc.

[DongzeHE]

I think we should definitely add the aux data into the anndata obs data. Also, we should make sure we save spliced, unspliced and ambiguous count matrices into three different layers.

[ilan-gold]

Yes! You should definitely take full advantage of the data model to store all the metadata, summary statistics, and quality info you can. Happy to hop on a call to run down with you where would be the best place for this stuff if you need.

[rob-p]

Agreed! @DongzeHE : the tip of dev currently properly splits the information into spliced/unspliced and ambiguous (if alevin-fry was run in USA mode). What other auxiliary data do you have in mind?

[an-altosian]

Mainly the optional aux information in the cell barcode file and the gene_id_to_name file. Currently there are nothing else in my mind that are used in the downstream analysis.

…

On Jan 14, 2025, at 9:27 PM, Rob Patro ***@***.***> wrote: Agreed! @DongzeHE <https://github.com/DongzeHE> : the tip of dev currently properly splits the information into spliced/unspliced and ambiguous (if alevin-fry was run in USA mode). What other auxiliary data do you have in mind? — Reply to this email directly, view it on GitHub <#177 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BI7FTBUWNY43D6VSW6TCEKD2KXWTDAVCNFSM6AAAAABU65BQQKVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTCOBTHA4TIMI>. You are receiving this because you were mentioned.