Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TypeError: astype() got an unexpected keyword argument 'copy' #312

Closed
johannesbjork opened this issue Aug 29, 2020 · 3 comments
Closed

TypeError: astype() got an unexpected keyword argument 'copy' #312

johannesbjork opened this issue Aug 29, 2020 · 3 comments
Labels
bug Something isn't working external issues/bugs with other libraries, frameworks, etc.; might include reproducing an issue minimally

Comments

@johannesbjork
Copy link

Running the stand-alone version of qurro on results generated by DEICODE on the qiime2-moving-pictures-tutorial I first get a warning saying that 228 feature(s) in the BIOM table were not present in the feature rankings. These feature(s) have been removed from the visualization.--I guess these got filtered out by DEICODE (this is not, I think mentioned in the tutorial?)

But the focal error message as seen below is TypeError: astype() got an unexpected keyword argument 'copy'.

Any idea what this points to?
Thanks!

(qurro) user@user qiime2-moving-pictures-tutorial % qurro \                           
    --ranks standalone-cli/ordination.txt \
    --table table.biom \
    --sample-metadata sample-metadata.tsv \
    --feature-metadata taxonomy.tsv \
    --output-dir qurro-standalone
228 feature(s) in the BIOM table were not present in the feature rankings.
These feature(s) have been removed from the visualization.
Traceback (most recent call last):
  File "/Users/user/python/miniconda3/envs/qurro/bin/qurro", line 11, in <module>
    sys.exit(plot())
  File "/Users/user/python/miniconda3/envs/qurro/lib/python3.6/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/user/python/miniconda3/envs/qurro/lib/python3.6/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/user/python/miniconda3/envs/qurro/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/user/python/miniconda3/envs/qurro/lib/python3.6/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/user/python/miniconda3/envs/qurro/lib/python3.6/site-packages/qurro/scripts/_plot.py", line 100, in plot
    extreme_feature_count,
  File "/Users/user/python/miniconda3/envs/qurro/lib/python3.6/site-packages/qurro/generate.py", line 66, in process_and_generate
    output_dir,
  File "/Users/user/python/miniconda3/envs/qurro/lib/python3.6/site-packages/qurro/generate.py", line 470, in gen_visualization
    V, rank_type, ranking_ids, feature_metadata_cols, processed_table
  File "/Users/user/python/miniconda3/envs/qurro/lib/python3.6/site-packages/qurro/generate.py", line 260, in gen_rank_plot
    rank_data = add_sample_presence_count(rank_data, table_sdf)
  File "/Users/user/python/miniconda3/envs/qurro/lib/python3.6/site-packages/qurro/_df_utils.py", line 516, in add_sample_presence_count
    table_pa = table_sdf.mask(table_sdf > 0, 1)
  File "/Users/user/python/miniconda3/envs/qurro/lib/python3.6/site-packages/pandas/core/generic.py", line 6154, in mask
    errors=errors)
  File "/Users/user/python/miniconda3/envs/qurro/lib/python3.6/site-packages/pandas/core/generic.py", line 6130, in where
    errors=errors, try_cast=try_cast)
  File "/Users/user/python/miniconda3/envs/qurro/lib/python3.6/site-packages/pandas/core/generic.py", line 5893, in _where
    cond = cond.astype(bool, copy=False)
TypeError: astype() got an unexpected keyword argument 'copy'
@johannesbjork johannesbjork added the bug Something isn't working label Aug 29, 2020
@fedarko
Copy link
Collaborator

fedarko commented Aug 29, 2020 via email

@johannesbjork
Copy link
Author

So changing the python version to >3.6 by specifying conda install qurro "python>=3.7,<3.8" -c conda-forge seem to have worked. Although a lot of warnings were generated.

user@user qiime2-moving-pictures-tutorial % qurro \                                              
    --ranks standalone-cli/ordination.txt \
    --table table.biom \
    --sample-metadata sample-metadata.tsv \
    --feature-metadata taxonomy.tsv \
    --output-dir qurro-standalone
/Users/user/python/miniconda3/lib/python3.7/site-packages/qurro/_df_utils.py:126: FutureWarning: SparseDataFrame is deprecated and will be removed in a future version.
Use a regular DataFrame whose columns are SparseArrays instead.

See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more.

  table_sdf = pd.SparseDataFrame(table.matrix_data, default_fill_value=0.0)
/Users/user/python/miniconda3/lib/python3.7/site-packages/pandas/core/sparse/frame.py:257: FutureWarning: SparseSeries is deprecated and will be removed in a future version.
Use a Series with sparse values instead.

    >>> series = pd.Series(pd.SparseArray(...))

See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more.

  sparse_index=BlockIndex(N, blocs, blens),
/Users/user/python/miniconda3/lib/python3.7/site-packages/pandas/core/frame.py:3471: FutureWarning: SparseSeries is deprecated and will be removed in a future version.
Use a Series with sparse values instead.

    >>> series = pd.Series(pd.SparseArray(...))

See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more.

  return klass(values, index=self.index, name=items, fastpath=True)
/Users/user/python/miniconda3/lib/python3.7/site-packages/pandas/core/ops/__init__.py:1641: FutureWarning: SparseSeries is deprecated and will be removed in a future version.
Use a Series with sparse values instead.

    >>> series = pd.Series(pd.SparseArray(...))

See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more.

  return self._constructor(new_values, index=self.index, name=self.name)
/Users/user/python/miniconda3/lib/python3.7/site-packages/pandas/core/sparse/frame.py:339: FutureWarning: SparseDataFrame is deprecated and will be removed in a future version.
Use a regular DataFrame whose columns are SparseArrays instead.

See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more.

  default_fill_value=self.default_fill_value,
/Users/user/python/miniconda3/lib/python3.7/site-packages/pandas/core/generic.py:6289: FutureWarning: SparseDataFrame is deprecated and will be removed in a future version.
Use a regular DataFrame whose columns are SparseArrays instead.

See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more.

  return self._constructor(new_data).__finalize__(self)
/Users/user/python/miniconda3/lib/python3.7/site-packages/pandas/core/generic.py:5884: FutureWarning: SparseSeries is deprecated and will be removed in a future version.
Use a Series with sparse values instead.

    >>> series = pd.Series(pd.SparseArray(...))

See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more.

  return self._constructor(new_data).__finalize__(self)
/Users/user/python/miniconda3/lib/python3.7/site-packages/pandas/core/sparse/frame.py:785: FutureWarning: SparseDataFrame is deprecated and will be removed in a future version.
Use a regular DataFrame whose columns are SparseArrays instead.

See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more.

  return self._constructor(new_arrays, index=index, columns=columns).__finalize__(
228 feature(s) in the BIOM table were not present in the feature rankings.
These feature(s) have been removed from the visualization.
/Users/user/python/miniconda3/lib/python3.7/site-packages/pandas/core/generic.py:3606: FutureWarning: SparseDataFrame is deprecated and will be removed in a future version.
Use a regular DataFrame whose columns are SparseArrays instead.

See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more.

  result = self._constructor(new_data).__finalize__(self)
/Users/user/python/miniconda3/lib/python3.7/site-packages/pandas/core/generic.py:1999: FutureWarning: SparseDataFrame is deprecated and will be removed in a future version.
Use a regular DataFrame whose columns are SparseArrays instead.

See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more.

  return self._constructor(result, **d).__finalize__(self)
/Users/user/python/miniconda3/lib/python3.7/site-packages/pandas/core/sparse/frame.py:745: FutureWarning: SparseDataFrame is deprecated and will be removed in a future version.
Use a regular DataFrame whose columns are SparseArrays instead.

See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more.

  default_fill_value=self._default_fill_value,
/Users/user/python/miniconda3/lib/python3.7/site-packages/pandas/core/generic.py:9126: FutureWarning: SparseDataFrame is deprecated and will be removed in a future version.
Use a regular DataFrame whose columns are SparseArrays instead.

See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more.

  return self._constructor(new_data).__finalize__(self)
/Users/user/python/miniconda3/lib/python3.7/site-packages/pandas/core/sparse/frame.py:854: FutureWarning: SparseDataFrame is deprecated and will be removed in a future version.
Use a regular DataFrame whose columns are SparseArrays instead.

See http://pandas.pydata.org/pandas-docs/stable/user_guide/sparse.html#migrating for more.

  default_kind=self._default_kind,
Successfully generated a visualization in the folder qurro-standalone.

@fedarko fedarko added the external issues/bugs with other libraries, frameworks, etc.; might include reproducing an issue minimally label Aug 29, 2020
@fedarko
Copy link
Collaborator

fedarko commented Aug 30, 2020

Glad things worked out! Not sure why Python 3.6 was breaking things, but it's very useful to know that the problem can be avoided by using a newer Python version. Thanks for reporting the solution.

The warnings are unfortunately as expected -- they are a symptom of #258. (Qurro and a lot of related software (e.g. biocore/biom-format#838) have not yet been updated to handle the pandas v1 versions of sparse data structures; so many parts of Qurro that explicitly use the "old" sparse data structures trigger warnings, annoyingly.)

Lastly, I checked the DEICODE tutorials (standalone, QIIME 2) and it looks like they both mention the filtering parameters -- min-feature-count, min-sample-count, and min-feature-frequency. It's a pretty brief mention, though, so it seems easy to miss.

It seems like things are resolved for now, so I'm going to close this -- but please feel free to reopen (or to open a new issue) if you have any further questions :)

@fedarko fedarko closed this as completed Aug 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working external issues/bugs with other libraries, frameworks, etc.; might include reproducing an issue minimally
Projects
None yet
Development

No branches or pull requests

2 participants