Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Qurro will break on newer (>= 2020.11) versions of QIIME 2 #315

Closed
fedarko opened this issue Dec 11, 2020 · 5 comments · Fixed by #322
Closed

Qurro will break on newer (>= 2020.11) versions of QIIME 2 #315

fedarko opened this issue Dec 11, 2020 · 5 comments · Fixed by #322
Labels
bug Something isn't working interoperability Making Qurro work well with other bioinformatics tools

Comments

@fedarko
Copy link
Collaborator

fedarko commented Dec 11, 2020

Due to #258 -- ideally we will need to cut a new release that uses pandas 1, and clearly document which version to install based on the version of QIIME 2 in use.

For now, a faster solution would be just clearly documenting somewhere that Qurro, for now, can't work with QIIME 2 2020.11 and future versions, due to the pandas thing.
@fedarko fedarko added bug Something isn't working interoperability Making Qurro work well with other bioinformatics tools labels Dec 11, 2020
@astrophys
Copy link

astrophys commented Nov 8, 2021 •
edited
Loading

A specific source of difficulties :

  1. Using conda, Qiime2/2021.8 installs pandas-1.2.5 + numpy-1.21.2.
  2. When I installed qurro (using pip), it depends on pandas < 1.0 and > 0.24, so pip happily uninstalled pandas-1.2.5 and downgraded it to pandas-0.25.3.
  3. pandas-0.25.3 is NOT compatible with numpy-1.21.2, e.g.
>>> import pandas as pd
>>> df =  pd.DataFrame(columns=['SubjectID1', 'SubjectID2', 'Group1','Group2', 'Distance'])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/export/apps/opt/QIIME2/2021.8/envs/qiime2/lib/python3.8/site-packages/pandas/core/frame.py", line 411, in __init__
    mgr = init_dict(data, index, columns, dtype=dtype)
  File "/export/apps/opt/QIIME2/2021.8/envs/qiime2/lib/python3.8/site-packages/pandas/core/internals/construction.py", line 242, in init_dict
    val = construct_1d_arraylike_from_scalar(np.nan, len(index), nan_dtype)
  File "/export/apps/opt/QIIME2/2021.8/envs/qiime2/lib/python3.8/site-packages/pandas/core/dtypes/cast.py", line 1221, in construct_1d_arraylike_from_scalar
    dtype = dtype.dtype
AttributeError: type object 'object' has no attribute 'dtype'
  1. So to 'fix' this, I used pip to install numpy-1.19.5, e.g. pip install --prefix /path/to/QIIME2/2021.8/envs/qiime2/ numpy==1.19.5

It 'fixes' that error, but it is unclear how this may break other feautures / functionality of qiime2.

@ElDeveloper
Copy link
Member

Any updates on this ticket? Just ran into this problem running the latest version of QIIME2. Also noticed the altair version is pinned at exactly 3.1.0 which is almost 3 years old, it would be nice if the version requirements were a little more forgiving.

@fedarko
Copy link
Collaborator Author

fedarko commented Apr 4, 2022 via email

@ElDeveloper
Copy link
Member

Thanks for the update! 😵

fedarko added a commit to fedarko/qurro that referenced this issue Jul 5, 2022
@fedarko
DEP: Update setup.py re: python and pandas biocore#315
31cda54
fedarko added a commit to fedarko/qurro that referenced this issue Jul 5, 2022
@fedarko
DEP: Fix biom_table_to_sparse_df for pandas >= 1
b90a834 
See biocore#258 and biocore#315. not confident this is done yet (and if nothing
else the rest of the code gleefully refers to "SparseDataFrame"
because 2019 marcus was a schmuck), but this at least fixes a fair
amount of failing tests
@fedarko fedarko mentioned this issue Jul 5, 2022
Merged
2 tasks
fedarko added a commit that referenced this issue Oct 20, 2022
@fedarko
Update Qurro to support pandas v1 and up, and thus newer versions of …
59d0d65 
…QIIME 2 (#322)

* DEP: Update setup.py re: python and pandas #315

* DEV: port CI from Travis to GH Actions: close #316

* TST: For now, omit "make notebooks" from CI

Maybe we can make another GitHub Actions for these later; but
Songbird is causing tensorflow nonsense to pop up, and this is not
the sort of thing I think we should spend time fixing (esp with
the advent of birdman)

* DEP: pin min biom vsn and add some comments

* DEP: Fix biom_table_to_sparse_df for pandas >= 1

See #258 and #315. not confident this is done yet (and if nothing
else the rest of the code gleefully refers to "SparseDataFrame"
because 2019 marcus was a schmuck), but this at least fixes a fair
amount of failing tests

* DEP: remove some warnings, docs, fix a test re: pd

* TST: Fix the python tests!!! #258

The problem was using .loc[] on these sparse dataframes. whoops

* STY: tiny style fixes

* DEP: knock out some pandas warnings

* DEP: np.matrix() -> np.array() in qarcoal tests

since apparently it's deprecated, or about to be deprecated, idk

* DEP/STY: Fix more warnings; remove unused import

most of these warnings were just pd.DataFrame.append() being
deprecated and replaced with pd.concat()

* DOC: one of the demos' JS data slightly changed

looks like it's a tiny floating-point thing -- probably an artifact
of working here on a new operating system, on a new python version,
a new pandas version, a new biom version, etc. shouldn't make a
noticeable difference

* DOC: update readme re: min Q2 vsn

* TST: matrix of qiime 2 versions

nice!

* TST: more detailed comment about Q2 vsn matrix

* DOC: remove the "Sparse" from "SparseDataFrame"

* REL: version kick

* TST: Add standalone CI

IIRC something about how our specific altair version works makes it
incompatible with python 3.10. let's test that here -- if needed,
we can update the README to disallow python versions >= 3.10. (And
then we can look into removing the altair pin when absolutely needed.)

* TST: attempt to get standalone tests working

* TST: attempt to fix pytest q2 exclusion

* DEP: ok py 3.10 is a no go

* STY: fix formatting

* DOC: Rerun 4 / 6 example notebooks

Songbird and ALDEx2 ones will cause problems

* DOC: tidy/update readme refs

* DOC: update jake fish dataset ref on website

* DOC: Fix songbird notebook!, standardize output rm

* BLD: rm (now-)unused comments from q2 ci

* DOC: fix transcriptomics ntbk :)

* REL: update changelog

* REL: update changelog

* TST: see if we can finagle q2 2020.6 / 2020.8?

since i thiiiink these versions mighta worked with the pandas >= 1
syntax

that being said, i don't think it makes sense to devote time/energy
to officially supporting them; just wanna check

* TST: remove Q2 2020.6 / 2020.8 in CI

Looks like the tests themselves pass for these versions, but the
style-checking with black fails due to incompatibility with click.

yeah this is enough for me to not bother supporting these versions
imo

* DOC: songbird compatibility deets

* DEV/DOC: update dev docs re: 2022

the apocalypse came and all i got was this pull request

* REL: update changelog about updating contributing

about about about about aboot

* REL: minor chglog tidying
@fedarko
Copy link
Collaborator Author

fedarko commented Oct 20, 2022

I apologize for the long delay in fixing this. Qurro v0.8.0 is now available on PyPI (the conda-forge version should be automatically updated soon), and supports all versions of QIIME 2 ≥ 2020.11.

@fedarko fedarko mentioned this issue Apr 22, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working interoperability Making Qurro work well with other bioinformatics tools
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update Qurro to support pandas v1 and up, and thus newer versions of QIIME 2 fedarko/qurro
3 participants
@ElDeveloper @fedarko @astrophys