nextflow-tutorial.md

This workflow re-implements in Nextflow the logic of the Snakemake workflow presented in this webinar using the same data sets.

• Nextflow installation
• Software environment
• Nextflow version
• Running the example pipeline
  • Local/server profiles
  • HPC (SLURM) profiles
  • Cloud profile
• Contributing

Nextflow installation

You can install Nextflow using conda

conda install nextflow
conda update nextflow

Alternatively, check if you have Java 8 or newer (java -version, should be 1.8 or newer), and if you do, run

curl -s https://get.nextflow.io | bash

This will place the executable in your working directory and you should be able to run it

./nextflow

It probably makes sense to move the executable to a directory accessible via $PATH, just to be able to run nextflow rather than having to remember to type the full path to nextflow each time you want to run it.

For consistency of how nextflow output is presented in the terminal, you should use the specific version of nextflow, e.g. NXF_VER=19.01.0 nextflow run .... Alternatively, use -ansi-log false with later versions of nextflow.

Software environment

• Conda or either one of Docker or Singularity to run the appropriate container with all the remaining required software.

  

  

Nextflow version

Running the example pipeline

If you happen to already have all the required tools installed (available on $PATH), you should be able to simply

nextflow run csiro-crop-informatics/nextflow-embl-abr-webinar

By default, only one of the 16 wheat accessions will be processed. To process a subset of say, 5 use --take 5 or to process all use --take all.

The intended use is with one of the execution profiles.

Local/server profiles

nextflow run csiro-crop-informatics/nextflow-embl-abr-webinar -profile conda
nextflow run csiro-crop-informatics/nextflow-embl-abr-webinar -profile docker
nextflow run csiro-crop-informatics/nextflow-embl-abr-webinar -profile singularity

HPC (SLURM) profiles

nextflow run csiro-crop-informatics/nextflow-embl-abr-webinar -profile slurm,conda
nextflow run csiro-crop-informatics/nextflow-embl-abr-webinar -profile slurm,singularity,singularitymodule

Cloud profile

If you are new to AWS batch and/or nextflow, follow this blog post, once you are done, or you already use AWS batch, you may be able to simply run

nextflow run csiro-crop-informatics/nextflow-embl-abr-webinar -profile awsbatch \
  -work-dir s3://your_s3_bucket/work --outdir s3://your_s3_bucket/results

after replacing your_s3_bucket with a bucket you have created on S3.

Contributing

• Please submit GitHub issues to indicate what needs to be clarified or explained.
• Pull requests are very welcome.