Problem with running install_conda_env.sh

[pangwopi]

Hi,

I am new to this package. I would like to install and use chip-seq-pipeline2 on my mac but had problem configuring the environment. I followed the installation instruction but failed to run install_conda_env.sh. I got the following message at the end:

=== Configuring for pipeline's Conda environments ===
usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
[-e pattern] [-f file] [--binary-files=value] [--color=when]
[--context[=num]] [--directories=action] [--label] [--line-buffered]
[--null] [pattern] [file ...]
Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'>
BrokenPipeError: [Errno 32] Broken pipe
usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
[-e pattern] [-f file] [--binary-files=value] [--color=when]
[--context[=num]] [--directories=action] [--label] [--line-buffered]
[--null] [pattern] [file ...]
Exception ignored in: <_io.TextIOWrapper name='' mode='w' encoding='UTF-8'>
BrokenPipeError: [Errno 32] Broken pipe
Error: Pipeline's Conda environments not found.

I am using a Mac system, version 10.14.4. I had python version 2.7 and version 3.5 installed on my Mac. I am not an expert on python too so I could not figure out what the "Bronken pipe" error is all about. Any help will be appreciated. Thanks a lot!

[leepc12]

Did you install Conda?

[pangwopi]

Yes, I have conda version 4.7.12

[leepc12]

It looks like grep -P doesn't work on MacOS.

Edit that shell script

Replace

CONDA_PREFIX_PY3=$(conda env list | grep -P "\b${CONDA_ENV_PY3}\s" | awk '{if (NF==3) print $3; else print $2}')
CONDA_PREFIX_PY2=$(conda env list | grep -P "\b${CONDA_ENV_PY2}\s" | awk '{if (NF==3) print $3; else print $2}')

with

CONDA_PREFIX_PY3=$(conda env list | grep "${CONDA_ENV_PY3}" | awk '{if (NF==3) print $3; else print $2}')
CONDA_PREFIX_PY2=$(conda env list | grep "{CONDA_ENV_PY2}" | awk '{if (NF==3) print $3; else print $2}')

[pangwopi]

Hi Jin

Thanks for the prompt reply and solution. Really appreciate it. It does address the grep problem, but the last error message is still there. Below is the updated last part of the message:

=== Configuring for pipeline's Conda environments ===
Error: Pipeline's Conda environments not found.
Try to reinstall pipeline's Conda environments.

1. $ bash uninstall_conda_env.sh
2. $ bash install_conda_env.sh

What does "Error: Pipeline's Conda environments not found." mean? Am I supposed to be configuring the conda environment for the pipeline by running this script? Please help.

[leepc12]

Can you do this for debugging?

$ CONDA_ENV_PY3=encode-atac-seq-pipeline
$ conda env list | grep "${CONDA_ENV_PY3}" | awk '{if (NF==3) print $3; else print $2}'

Also, this one.

$ conda env list

[pangwopi]

I assume you meant CONDA_ENV_PY3=encode-chip-seq-pipeline.

conda env list | grep "${CONDA_ENV_PY3}" | awk '{if (NF==3) print $3; else print $2}'

/Users/compcore/miniconda3/envs/encode-chip-seq-pipeline
/Users/compcore/miniconda3/envs/encode-chip-seq-pipeline-python2

conda env list

(hashtag) conda environments:
(hashtag)
/Users/compcore/anaconda3
/Users/compcore/anaconda3/envs/encode-chip-seq-pipeline
/Users/compcore/anaconda3/envs/encode-chip-seq-pipeline-python2
base * /Users/compcore/miniconda3
encode-chip-seq-pipeline /Users/compcore/miniconda3/envs/encode-chip-seq-pipeline
encode-chip-seq-pipeline-python2 /Users/compcore/miniconda3/envs/encode-chip-seq-pipeline-python2

[leepc12]

Ok that patched grep command didn't work... It grabbed multiple lines. Let's do this way.

Can you uninstall your macOS' old grep and reinstall GNU grep (which has a -P parameter)?
https://apple.stackexchange.com/questions/193288/how-to-install-and-use-gnu-grep-in-osx

$ brew uninstall grep
$ brew install grep

Add export PATH="/usr/local/opt/grep/libexec/gnubin:$PATH" to your ~/.bashrc.

$ grep --help  # check if -P exists

If this doesn't work, can you somehow figure out how to install GNU grep on your system?

[pangwopi]

Hi Jin,

Thanks a lot. By installing GNU grep on Mac, the problem is solved.

[pangwopi]

Hi Jin,

It seems like the installation was successful. However, when I tried to test the workflow with the test json file. It failed. My command is the following:

caper run chip.wdl -i ENCSR000DYI_subsampled_chr19_only_caper.json

Got the following failure message at the end:

"submission": "2019-11-14T18:54:58.856Z",
"status": "Failed",
"failures": [
{
"message": "Workflow failed",
"causedBy": [
{
"causedBy": [],
"message": "Job chip.align_ctl:1:2 exited with return code 3 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details."
},
{
"causedBy": [],
"message": "Job chip.align_ctl:0:2 exited with return code 3 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details."
},
{
"causedBy": [],
"message": "Job chip.align_R1:0:2 exited with return code 3 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details."
},
{
"causedBy": [],
"message": "Job chip.align:0:2 exited with return code 3 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details."
},
{
"causedBy": [],
"message": "Job chip.align_R1:1:2 exited with return code 3 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details."
},
{
"message": "Job chip.align:1:2 exited with return code 3 which has not been declared as a valid return code. See 'continueOnReturnCode' runtime attribute for more details.",
"causedBy": []
}
]
}
],

I have activated conda environment before I run it. Could you please point out to me where could be wrong?

[leepc12]

Sorry about late response. align is the first task in the pipeline. I think something is misconfigured in ~/.caper/default.conf.