all transcripts annotated as "non_coding" after isoseq/pigeon

[kmattioli]

Operating system

  Operating System: CentOS Linux 7 (Core)
       CPE OS Name: cpe:/o:centos:centos:7
            Kernel: Linux 3.10.0-1160.119.1.el7.x86_64
      Architecture: x86-64

Package name

isoseq 4.2.0 (commit v4.2.0)

Using:
  pbbam     : 2.7.0 (commit v2.7.0)
  pbcopper  : 2.6.0 (commit v2.6.0)
  pbmm2     : 1.16.0 (commit v1.16.0)
  minimap2  : 2.26
  parasail  : 2.1.3
  boost     : 1.81
  htslib    : 1.17
  zlib      : 1.2.13

pigeon 1.3.0 (commit -v1.3.0)

Using:
  pbbam     : 2.7.0 (commit v2.7.0)
  pbcopper  : 2.6.0 (commit v2.6.0)
  boost     : 1.81
  htslib    : 1.17
  zlib      : 1.2.13

Conda environment

# Name                    Version                   Build  Channel
_anaconda_depends         2020.07                  py37_0  
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
alabaster                 0.7.12                   py37_0  
anaconda                  custom                   py37_1  
anaconda-client           1.7.2                    py37_0  
anaconda-project          0.9.1              pyhd3eb1b0_1  
anyio                     2.2.0            py37h06a4308_1  
appdirs                   1.4.4                      py_0  
argh                      0.26.2                   py37_0  
argon2-cffi               20.1.0           py37h27cfd23_1  
asn1crypto                1.4.0                      py_0  
astroid                   2.5              py37h06a4308_1  
astropy                   4.2.1            py37h27cfd23_1  
async_generator           1.10             py37h28b3542_0  
atomicwrites              1.4.0                      py_0  
attrs                     20.3.0             pyhd3eb1b0_0  
autopep8                  1.5.6              pyhd3eb1b0_0  
avro-python3              1.9.0                    py37_0    bioconda
babel                     2.9.0              pyhd3eb1b0_0  
backcall                  0.2.0              pyhd3eb1b0_0  
backports                 1.0                pyhd3eb1b0_2  
backports.shutil_get_terminal_size 1.0.0              pyhd3eb1b0_3  
bamtools                  2.5.1                he513fc3_6    bioconda
beautifulsoup4            4.9.3              pyha847dfd_0  
biopython                 1.78             py37h7b6447c_0  
bitarray                  2.1.0            py37h27cfd23_1  
bkcharts                  0.2                      py37_0  
black                     19.10b0                    py_0  
blas                      1.0                         mkl  
bleach                    3.3.0              pyhd3eb1b0_0  
blosc                     1.21.0               h8c45485_0  
bokeh                     2.3.2            py37h06a4308_0  
boto                      2.49.0                   py37_0  
bottleneck                1.3.2            py37heb32a55_1  
brotlipy                  0.7.0           py37h27cfd23_1003  
bx-python                 0.8.11                   pypi_0    pypi
bzip2                     1.0.8                h7b6447c_0  
c-ares                    1.17.1               h27cfd23_0  
ca-certificates           2024.8.30            hbcca054_0    conda-forge
cairo                     1.16.0               hf32fb01_1  
certifi                   2024.8.30          pyhd8ed1ab_0    conda-forge
cffi                      1.14.5           py37h261ae71_0  
chardet                   4.0.0           py37h06a4308_1003  
click                     7.1.2              pyhd3eb1b0_0  
cloudpickle               1.6.0                      py_0  
clyent                    1.2.2                    py37_1  
colorama                  0.4.4              pyhd3eb1b0_0  
contextlib2               0.6.0.post1                py_0  
cryptography              3.4.7            py37hd23ed53_0  
cupcake                   24.3.0                   pypi_0    pypi
curl                      7.71.1               hbc83047_1  
cycler                    0.10.0                   py37_0  
cython                    0.29.23          py37h2531618_0  
cytoolz                   0.11.0           py37h7b6447c_0  
dask                      2021.4.0           pyhd3eb1b0_0  
dask-core                 2021.4.0           pyhd3eb1b0_0  
dbus                      1.13.18              hb2f20db_0  
decorator                 5.0.6              pyhd3eb1b0_0  
defusedxml                0.7.1              pyhd3eb1b0_0  
desalt                    1.5.6                h5bf99c6_1    bioconda
diff-match-patch          20200713                   py_0  
distributed               2021.4.1         py37h06a4308_0  
docutils                  0.17.1           py37h06a4308_1  
entrypoints               0.3                      py37_0  
et_xmlfile                1.0.1                   py_1001  
expat                     2.3.0                h2531618_2  
fastcache                 1.1.0            py37h7b6447c_0  
filelock                  3.0.12             pyhd3eb1b0_1  
flake8                    3.9.0              pyhd3eb1b0_0  
flask                     1.1.2              pyhd3eb1b0_0  
fontconfig                2.13.1               h6c09931_0  
freetype                  2.10.4               h5ab3b9f_0  
fribidi                   1.0.10               h7b6447c_0  
fsspec                    0.9.0              pyhd3eb1b0_0  
future                    0.18.2                   py37_1  
get_terminal_size         1.0.0                haa9412d_0  
gevent                    21.1.2           py37h27cfd23_1  
glib                      2.68.1               h36276a3_0  
glob2                     0.7                pyhd3eb1b0_0  
gmp                       6.2.1                h2531618_2  
gmpy2                     2.0.8            py37h10f8cd9_2  
graphite2                 1.3.14               h23475e2_0  
greenlet                  1.0.0            py37h2531618_2  
gst-plugins-base          1.14.0               h8213a91_2  
gstreamer                 1.14.0               h28cd5cc_2  
h5py                      2.10.0           py37h7918eee_0  
harfbuzz                  2.8.0                h6f93f22_0  
hdf5                      1.10.4               hb1b8bf9_0  
heapdict                  1.0.1                      py_0  
html5lib                  1.1                        py_0  
htslib                    1.9                  ha228f0b_7    bioconda
icu                       58.2                 he6710b0_3  
idna                      2.10               pyhd3eb1b0_0  
imageio                   2.9.0              pyhd3eb1b0_0  
imagesize                 1.2.0              pyhd3eb1b0_0  
importlib-metadata        3.10.0           py37h06a4308_0  
importlib_metadata        3.10.0               hd3eb1b0_0  
iniconfig                 1.1.1              pyhd3eb1b0_0  
intel-openmp              2021.2.0           h06a4308_610  
intervaltree              3.1.0                      py_0  
ipykernel                 5.3.4            py37h5ca1d4c_0  
ipython                   7.22.0           py37hb070fc8_0  
ipython_genutils          0.2.0              pyhd3eb1b0_1  
ipywidgets                7.6.3              pyhd3eb1b0_1  
isl                       0.12.2                        0  
iso8601                   0.1.14           py37h06a4308_1  
isort                     5.8.0              pyhd3eb1b0_0  
isoseq                    4.2.0                h9ee0642_0    bioconda
isoseq3                   3.4.0                         0    bioconda
itsdangerous              1.1.0                    py37_0  
jbig                      2.1                  hdba287a_0  
jdcal                     1.4.1                      py_0  
jedi                      0.17.2           py37h06a4308_1  
jeepney                   0.6.0              pyhd3eb1b0_0  
jinja2                    2.11.3             pyhd3eb1b0_0  
joblib                    1.0.1              pyhd3eb1b0_0  
jpeg                      9b                   h024ee3a_2  
json5                     0.9.5                      py_0  
jsonschema                3.2.0                      py_2  
jupyter                   1.0.0                    py37_7  
jupyter-packaging         0.7.12             pyhd3eb1b0_0  
jupyter_client            6.1.12             pyhd3eb1b0_0  
jupyter_console           6.4.0              pyhd3eb1b0_0  
jupyter_core              4.7.1            py37h06a4308_0  
jupyter_server            1.4.1            py37h06a4308_0  
jupyterlab                3.0.14             pyhd3eb1b0_1  
jupyterlab_pygments       0.1.2                      py_0  
jupyterlab_server         2.4.0              pyhd3eb1b0_0  
jupyterlab_widgets        1.0.0              pyhd3eb1b0_1  
k8                        0.2.5                he513fc3_0    bioconda
keyring                   22.3.0           py37h06a4308_0  
kiwisolver                1.3.1            py37h2531618_0  
krb5                      1.18.2               h173b8e3_0  
lazy-object-proxy         1.6.0            py37h27cfd23_0  
lcms2                     2.12                 h3be6417_0  
ld_impl_linux-64          2.33.1               h53a641e_7  
libarchive                3.4.2                h62408e4_0  
libcurl                   7.71.1               h20c2e04_1  
libdeflate                1.0                  h14c3975_1    bioconda
libedit                   3.1.20210216         h27cfd23_1  
libev                     4.33                 h7b6447c_0  
libffi                    3.3                  he6710b0_2  
libgcc                    14.2.0               h77fa898_1    conda-forge
libgcc-ng                 14.2.0               h69a702a_1    conda-forge
libgfortran-ng            7.3.0                hdf63c60_0  
libgomp                   14.2.0               h77fa898_1    conda-forge
liblief                   0.10.1               he6710b0_0  
libllvm10                 10.0.1               hbcb73fb_5  
libllvm9                  9.0.1                he513fc3_1    conda-forge
libpng                    1.6.37               hbc83047_0  
libsodium                 1.0.18               h7b6447c_0  
libspatialindex           1.9.3                h2531618_0  
libssh2                   1.9.0                h1ba5d50_1  
libstdcxx                 14.2.0               hc0a3c3a_1    conda-forge
libstdcxx-ng              9.1.0                hdf63c60_0  
libtiff                   4.2.0                h85742a9_0  
libtool                   2.4.6             h7b6447c_1005  
libuuid                   1.0.3                h1bed415_2  
libuv                     1.40.0               h7b6447c_0  
libwebp-base              1.2.0                h27cfd23_0  
libxcb                    1.14                 h7b6447c_0  
libxml2                   2.9.10               hb55368b_3  
libxslt                   1.1.34               hc22bd24_0  
lima                      2.12.0               h9ee0642_1    bioconda
llvmlite                  0.36.0           py37h612dafd_4  
locket                    0.2.1            py37h06a4308_1  
lxml                      4.6.3            py37h9120a33_0  
lz4-c                     1.9.3                h2531618_0  
lzo                       2.10                 h7b6447c_2  
markupsafe                1.1.1            py37h14c3975_1  
matplotlib                3.3.4            py37h06a4308_0  
matplotlib-base           3.3.4            py37h62a2d02_0  
mccabe                    0.6.1                    py37_1  
minimap2                  2.17                 hed695b0_3    bioconda
mistune                   0.8.4           py37h14c3975_1001  
mkl                       2021.2.0           h06a4308_296  
mkl-service               2.3.0            py37h27cfd23_1  
mkl_fft                   1.3.0            py37h42c9631_2  
mkl_random                1.2.1            py37ha9443f7_2  
mock                      4.0.3              pyhd3eb1b0_0  
more-itertools            8.7.0              pyhd3eb1b0_0  
mpc                       1.1.0                h10f8cd9_1  
mpfr                      4.0.2                hb69a4c5_1  
mpmath                    1.2.1            py37h06a4308_0  
msgpack-python            1.0.2            py37hff7bd54_1  
multipledispatch          0.6.0                    py37_0  
mypy_extensions           0.4.3                    py37_0  
nbclassic                 0.2.6              pyhd3eb1b0_0  
nbclient                  0.5.3              pyhd3eb1b0_0  
nbconvert                 6.0.7                    py37_0  
nbformat                  5.1.3              pyhd3eb1b0_0  
ncurses                   6.2                  he6710b0_1  
nest-asyncio              1.5.1              pyhd3eb1b0_0  
networkx                  2.5                        py_0  
nltk                      3.6.1              pyhd3eb1b0_0  
nose                      1.3.7           pyhd3eb1b0_1006  
notebook                  6.3.0            py37h06a4308_0  
numba                     0.53.1           py37ha9443f7_0  
numexpr                   2.7.3            py37h22e1b3c_1  
numpy                     1.20.1           py37h93e21f0_0  
numpy-base                1.20.1           py37h7d8b39e_0  
numpydoc                  1.1.0              pyhd3eb1b0_1  
olefile                   0.46                     py37_0  
openpyxl                  3.0.7              pyhd3eb1b0_0  
openssl                   1.1.1w               hd590300_0    conda-forge
packaging                 20.9               pyhd3eb1b0_0  
pandas                    1.2.4            py37h2531618_0  
pandoc                    2.12                 h06a4308_0  
pandocfilters             1.4.3            py37h06a4308_1  
pango                     1.45.3               hd140c19_0  
parso                     0.7.0                      py_0  
partd                     1.2.0              pyhd3eb1b0_0  
patchelf                  0.12                 h2531618_1  
path                      15.1.2           py37h06a4308_0  
path.py                   12.5.0                        0  
pathlib2                  2.3.5            py37h06a4308_2  
pathspec                  0.7.0                      py_0  
pathtools                 0.1.2                      py_1    conda-forge
patsy                     0.5.1                    py37_0  
pbbam                     1.3.0                h8e3dc82_0    bioconda
pbccs                     6.4.0                h9ee0642_0    bioconda
pbcommand                 2.1.1                      py_2    bioconda
pbcore                    2.1.2                      py_2    bioconda
pbcoretools               0.8.1                      py_1    bioconda
pbmm2                     1.16.0               h9ee0642_0    bioconda
pbpigeon                  1.3.0                h4ac6f70_0    bioconda
pcre                      8.44                 he6710b0_0  
pep8                      1.7.1                    py37_0  
pexpect                   4.8.0              pyhd3eb1b0_3  
pickleshare               0.7.5           pyhd3eb1b0_1003  
pillow                    8.2.0            py37he98fc37_0  
pip                       21.0.1           py37h06a4308_0  
pixman                    0.40.0               h7b6447c_0  
pkginfo                   1.7.0            py37h06a4308_0  
pluggy                    0.13.1           py37h06a4308_0  
ply                       3.11                     py37_0  
prometheus_client         0.10.1             pyhd3eb1b0_0  
prompt-toolkit            3.0.17             pyh06a4308_0  
prompt_toolkit            3.0.17               hd3eb1b0_0  
psutil                    5.8.0            py37h27cfd23_1  
ptyprocess                0.7.0              pyhd3eb1b0_2  
py                        1.10.0             pyhd3eb1b0_0  
py-lief                   0.10.1           py37h403a769_0  
pycodestyle               2.6.0              pyhd3eb1b0_0  
pycosat                   0.6.3            py37h27cfd23_0  
pycparser                 2.20                       py_2  
pycrypto                  2.6.1           py37h7b6447c_10  
pycurl                    7.43.0.6         py37h1ba5d50_0  
pydocstyle                6.0.0              pyhd3eb1b0_0  
pyerfa                    1.7.3            py37h27cfd23_0  
pyflakes                  2.2.0              pyhd3eb1b0_0  
pygments                  2.8.1              pyhd3eb1b0_0  
pylint                    2.7.4            py37h06a4308_1  
pyls-black                0.4.6                hd3eb1b0_0  
pyls-spyder               0.3.2              pyhd3eb1b0_0  
pyodbc                    4.0.30           py37he6710b0_0  
pyopenssl                 20.0.1             pyhd3eb1b0_1  
pyparsing                 2.4.7              pyhd3eb1b0_0  
pyqt                      5.9.2            py37h05f1152_2  
pyrsistent                0.17.3           py37h7b6447c_0  
pysam                     0.15.3           py37hda2845c_1    bioconda
pysocks                   1.7.1                    py37_1  
pytables                  3.6.1            py37h71ec239_0  
pytest                    6.2.3            py37h06a4308_2  
python                    3.7.10               hdb3f193_0  
python-dateutil           2.8.1              pyhd3eb1b0_0  
python-jsonrpc-server     0.4.0                      py_0  
python-language-server    0.36.2             pyhd3eb1b0_0  
python-libarchive-c       2.9                pyhd3eb1b0_1  
pytz                      2021.1             pyhd3eb1b0_0  
pywavelets                1.1.1            py37h7b6447c_2  
pyxdg                     0.27               pyhd3eb1b0_0  
pyyaml                    5.4.1            py37h27cfd23_1  
pyzmq                     20.0.0           py37h2531618_1  
qdarkstyle                2.8.1                      py_0  
qt                        5.9.7                h5867ecd_1  
qtawesome                 1.0.2              pyhd3eb1b0_0  
qtconsole                 5.0.3              pyhd3eb1b0_0  
qtpy                      1.9.0                      py_0  
readline                  8.1                  h27cfd23_0  
regex                     2021.4.4         py37h27cfd23_0  
requests                  2.25.1             pyhd3eb1b0_0  
ripgrep                   12.1.1                        0  
rope                      0.18.0                     py_0  
rtree                     0.9.7            py37h06a4308_1  
ruamel_yaml               0.15.100         py37h27cfd23_0  
scikit-image              0.18.1           py37ha9443f7_0  
scikit-learn              0.24.1           py37ha9443f7_0  
scipy                     1.6.2            py37had2a1c9_1  
seaborn                   0.11.1             pyhd3eb1b0_0  
secretstorage             3.3.1            py37h06a4308_0  
send2trash                1.5.0              pyhd3eb1b0_1  
setuptools                52.0.0           py37h06a4308_0  
simplegeneric             0.8.1                    py37_2  
singledispatch            3.6.1           pyhd3eb1b0_1001  
sip                       4.19.8           py37hf484d3e_0  
six                       1.15.0           py37h06a4308_0  
sklearn                   0.0                      pypi_0    pypi
snappy                    1.1.8                he1b5a44_3    conda-forge
sniffio                   1.2.0            py37h06a4308_1  
snowballstemmer           2.1.0              pyhd3eb1b0_0  
sortedcollections         2.1.0              pyhd3eb1b0_0  
sortedcontainers          2.3.0              pyhd3eb1b0_0  
soupsieve                 2.2.1              pyhd3eb1b0_0  
sphinx                    4.0.1              pyhd3eb1b0_0  
sphinxcontrib             1.0                      py37_1  
sphinxcontrib-applehelp   1.0.2              pyhd3eb1b0_0  
sphinxcontrib-devhelp     1.0.2              pyhd3eb1b0_0  
sphinxcontrib-htmlhelp    1.0.3              pyhd3eb1b0_0  
sphinxcontrib-jsmath      1.0.1              pyhd3eb1b0_0  
sphinxcontrib-qthelp      1.0.3              pyhd3eb1b0_0  
sphinxcontrib-serializinghtml 1.1.4              pyhd3eb1b0_0  
sphinxcontrib-websupport  1.2.4                      py_0  
spyder                    4.2.5            py37h06a4308_0  
spyder-kernels            1.10.2           py37h06a4308_0  
sqlalchemy                1.4.15           py37h27cfd23_0  
sqlite                    3.35.4               hdfb4753_0  
statsmodels               0.12.2           py37h27cfd23_0  
sympy                     1.8              py37h06a4308_0  
tbb                       2020.3               hfd86e86_0  
tblib                     1.7.0                      py_0  
terminado                 0.9.4            py37h06a4308_0  
testpath                  0.4.4              pyhd3eb1b0_0  
textdistance              4.2.1              pyhd3eb1b0_0  
threadpoolctl             2.1.0              pyh5ca1d4c_0  
three-merge               0.1.1              pyhd3eb1b0_0  
tifffile                  2020.10.1        py37hdd07704_2  
tk                        8.6.10               hbc83047_0  
toml                      0.10.2             pyhd3eb1b0_0  
toolz                     0.11.1             pyhd3eb1b0_0  
tornado                   6.1              py37h27cfd23_0  
tqdm                      4.59.0             pyhd3eb1b0_1  
traitlets                 5.0.5              pyhd3eb1b0_0  
typed-ast                 1.4.2            py37h27cfd23_1  
typing_extensions         3.7.4.3            pyha847dfd_0  
ujson                     4.0.2            py37h2531618_0  
unicodecsv                0.14.1                   py37_0  
unixodbc                  2.3.9                h7b6447c_0  
urllib3                   1.26.4             pyhd3eb1b0_0  
watchdog                  1.0.2            py37h06a4308_1  
wcwidth                   0.2.5                      py_0  
webencodings              0.5.1                    py37_1  
werkzeug                  1.0.1              pyhd3eb1b0_0  
wget                      1.20.1               h20c2e04_0  
wheel                     0.36.2             pyhd3eb1b0_0  
widgetsnbextension        3.5.1                    py37_0  
wrapt                     1.12.1           py37h7b6447c_1  
wurlitzer                 2.1.0            py37h06a4308_0  
xlrd                      2.0.1              pyhd3eb1b0_0  
xlsxwriter                1.3.8              pyhd3eb1b0_0  
xlwt                      1.3.0                    py37_0  
xz                        5.2.5                h7b6447c_0  
yaml                      0.2.5                h7b6447c_0  
yapf                      0.31.0             pyhd3eb1b0_0  
zeromq                    4.3.4                h2531618_0  
zict                      2.0.0              pyhd3eb1b0_0  
zipp                      3.4.1              pyhd3eb1b0_0  
zlib                      1.2.11               h7b6447c_3  
zope                      1.0                      py37_1  
zope.event                4.5.0                    py37_0  
zope.interface            5.3.0            py37h27cfd23_0  
zstd                      1.4.5                h9ceee32_0

Describe the bug
After running Isoseq/Pigeon pipeline, all of my transcripts are annotated as non_coding in the _classification.txt output (as well as the filtered classification output). This happens even for protein-coding genes...

To Reproduce
Following the CLI example per the docs, using Gencode v39. Relevant code (after demuxing) below:

ls ${LIMA_OUT}/*.fl.*.bam | xargs -n 1 basename > ${LIMA_OUT}/fl.fofn
LIMA_FILE=${LIMA_OUT}/fl.fofn

REFINE_FILE=${REFINE_OUT}/${OUT_PREFIX}.flnc.bam
isoseq refine --require-polya $LIMA_FILE $CONCAT_PRIMER_FILE $REFINE_FILE --log-level FATAL

isoseq cluster2 $REFINE_FILE ${CLUSTER_OUT}/${OUT_PREFIX}.clustered.bam --use-qvs --log-level INFO -j 8

pbmm2 align --preset ISOSEQ --sort ${CLUSTER_OUT}/${OUT_PREFIX}.clustered.bam $GENOME_FA ${ALIGN_OUT}/${OUT_PREFIX}.mapped.bam --log-level FATAL 2> ${ALIGN_OUT}/${OUT_PREFIX}.minimap2.log

isoseq collapse ${ALIGN_OUT}/${OUT_PREFIX}.mapped.bam $REFINE_FILE ${COLLAPSE_OUT}/${OUT_PREFIX}.collapsed.gff --log-level FATAL

pigeon prepare ${COLLAPSE_OUT}/${OUT_PREFIX}.collapsed.gff --log-level FATAL

pigeon classify ${COLLAPSE_OUT}/${OUT_PREFIX}.collapsed.sorted.gff $GENCODE_SORTED $GENOME_FA --fl ${COLLAPSE_OUT}/${OUT_PREFIX}.collapsed.flnc_count.txt -d ${PIGEON_OUT} -o ${OUT_PREFIX}  --cage-peak $CAGE_PEAK_SORTED --poly-a $POLYA_LIST --c
overage $COVERAGE_SORTED --log-level FATAL

pigeon filter ${PIGEON_OUT}/${OUT_PREFIX}_classification.txt --isoforms ${COLLAPSE_OUT}/${OUT_PREFIX}.collapsed.sorted.gff --log-level FATAL

pigeon report ${PIGEON_OUT}/${OUT_PREFIX}_classification.filtered_lite_classification.txt ${PIGEON_OUT}/${OUT_PREFIX}.saturation.txt --log-level FATAL

Expected behavior
I should get many coding transcripts and annotated ORFs! I am not sure what step is causing the issue. Let me know and I'd be happy to provide some files to reproduce, but not sure where to start...

[kmattioli]

Hi, just pinging this issue and adding a bit more context -- it looks like in the previous iterations where I've run this pipeline, SQANTI3 produced the output gtf file with CDS information (corrected.gtf.cds.gff), but the same isn't happening with pigeon classify. Any ideas? Do I have to use SQANTI3 directly to get CDS information?

thanks

[Magdoll]

Hi @kmattioli thx for using pigeon! SQANTI3 (the academic version pigeon is based off) including GENEMARK for ORF prediction which is not included in pigeon, therefore, when running pigeon, all transcripts will be marked as noncoding since this information is not given.