Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding ONT functionality to RAW and CLEAN #133

Open
wants to merge 18 commits into
base: dev
Choose a base branch
from

Conversation

simonleandergrimm
Copy link
Collaborator

@simonleandergrimm simonleandergrimm commented Dec 18, 2024

Changes

  • Added Oxford Nanopore (ONT) data processing support:
    • Added FILTLONG module for ONT read filtering and quality control
    • Modified CLEAN workflow to use FILTLONG for ONT data and FASTP_SINGLE/PAIRED for Illumina data
    • Added ONT detection in read_type.config based on samplesheet content
    • Added ONT test data and configuration
  • Enhanced sequence length reporting:
    • Added length statistics to MultiQC summarization
    • Added length distribution plots to QC output
    • Renamed and generalized MultiQC processing modules (removed "pair" suffix)
    • Added new length statistics TSV output to results
  • Added staphb/filtlong:0.2.1 container to support ONT processing

Possible Todos (I don't think these should block merging)

  • Drop creation and publication of adapters -> not applicable for ONT atm.

@simonleandergrimm simonleandergrimm changed the base branch from master to single-read-raw-clean December 18, 2024 20:42
@simonleandergrimm simonleandergrimm marked this pull request as ready for review December 18, 2024 21:35
@simonleandergrimm
Copy link
Collaborator Author

@harmonbhasin Can you take an initial look and let me know if you see obvious flaws?

Here is what the output of the MultiQC output for ONT data looks like: https://data.securebio.org/simons-notebook/posts/2024-12-08-mgs-ont-raw-eval/

@simonleandergrimm simonleandergrimm changed the title Simon ont raw clean Adding ONT functionality to RAW and CLEAN Dec 18, 2024
Copy link
Collaborator

@harmonbhasin harmonbhasin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks correct to me. The only thing I ask is that you confirm that the output for the length stats file looks good for single-end/paired-end short reads.

modules/local/summarizeMultiqc/main.nf Show resolved Hide resolved
Copy link
Collaborator

@harmonbhasin harmonbhasin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noticed that you added the ONT test to the nf-test file, but not to the yaml file that makes it run on Github Actions, please do this, thank you!

Base automatically changed from single-read-raw-clean to dev December 20, 2024 13:55
@simonleandergrimm
Copy link
Collaborator Author

Just noticed that you added the ONT test to the nf-test file, but not to the yaml file that makes it run on Github Actions, please do this, thank you!

Fixed

@simonleandergrimm
Copy link
Collaborator Author

simonleandergrimm commented Jan 6, 2025

@harmonbhasin This is the error I get when nf-test is run on github. nf-test works locally. Can you let me know if you see what might be wrong here?

Otherwise I adopted your suggestions. Once the nf-test thing is resolved this is good for review by Will.

Run nf-test test --tag run --verbose

🚀 nf-test 0.9.2
https://www.nf-test.com
(c) 2021 - 2024 Lukas Forer and Sebastian Schoenherr


End-to-end test of MGS workflow

  Test [4ae1c173] 'Test run workflow' 
    > N E X T F L O W  ~  version 24.10.3
    > Launching `/home/runner/work/mgs-workflow/mgs-workflow/main.nf` [admiring_stallman] DSL2 - revision: 73db19e0f9
    > Downloading plugin [email protected]
    > WARN: WORKFLOW OUTPUT DSL IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
    > WARN: WORKFLOW OUTPUT DSL IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
    > WARN: WORKFLOW OUTPUT DSL IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
    > Downloading plugin [email protected]
    > [94/67ea50] Submitted process > RUN:RAW:QC:FASTQC_LABELED (1)
    > [d0/3647f3] Submitted process > RUN:CLEAN:FILTER_READS (1)
    > [9f/eb6025] Submitted process > RUN:RAW:QC:MULTIQC_LABELED (1)
    > [24/d22e6[8](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:9)] Submitted process > RUN:CLEAN:QC:FASTQC_LABELED (1)
    > [5d/1c48d5] Submitted process > RUN:PROFILE:SUBSET_READS_TARGET (1)
    > [54/1f6453] Submitted process > RUN:CLEAN:QC:MULTIQC_LABELED (1)
    > Staging foreign file: s3://nao-testing/index-test/output/results/ribo-ref-concat.fasta.gz
    > [87/1fc755] Submitted process > RUN:EXTRACT_VIRAL_READS:BBDUK_HITS (1)
    > [6e/d760af] Submitted process > RUN:PROFILE:BBDUK (1)
    > [64/fdca31] Submitted process > RUN:CLEAN:QC:SUMMARIZE_MULTIQC (1)
    > [cf/8dea48] Submitted process > RUN:RAW:QC:SUMMARIZE_MULTIQC (1)
    > [76/f[9](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:10)1a7d] Submitted process > RUN:EXTRACT_RAW_READS_FROM_PROCESSED (1)
    > [40/942af5] Submitted process > RUN:PROFILE:TAXONOMY_NORIBO:BBMERGE (1)
    > [19/fa812b] Submitted process > RUN:PROFILE:TAXONOMY_RIBO:BBMERGE (1)
    > [6f/0589c1] Submitted process > RUN:PROFILE:TAXONOMY_NORIBO:SUMMARIZE_BBMERGE (1)
    > [5c/b45ec3] Submitted process > RUN:PROFILE:TAXONOMY_NORIBO:JOIN_FASTQ (1)
    > [ae/b4ea7a] Submitted process > RUN:PROFILE:TAXONOMY_RIBO:SUMMARIZE_BBMERGE (1)
    > [ad/f49455] Submitted process > RUN:PROFILE:TAXONOMY_RIBO:JOIN_FASTQ (1)
    > [61/38d663] Submitted process > RUN:EXTRACT_VIRAL_READS:CUTADAPT (1)
    > Staging foreign file: s3://nao-testing/index-test/output/results/kraken_db
    > [9c/0e0866] Submitted process > RUN:RAW:QC:MERGE_MULTIQC_BASIC
    > [b3/6a0269] Submitted process > RUN:CLEAN:QC:MERGE_MULTIQC_QBASE
    > ERROR ~ Error executing process > 'RUN:RAW:QC:MERGE_MULTIQC_BASIC'
    > 
    > Caused by:
    >   Process `RUN:RAW:QC:MERGE_MULTIQC_BASIC` terminated with an error exit status (125)
    > 
    > 
    > Command executed:
    > 
    >   #!/usr/bin/env Rscript
    >   library(tidyverse)
    >   in_paths <- str_split("raw_concat_gold_standard_qc_basic_stats.tsv.gz", " ")[[1]]
    >   print(in_paths)
    >   tabs <- lapply(in_paths, function(t) read_tsv(t, col_names = TRUE, cols(.default="c")))
    >   for (t in tabs) print(dim(t))
    >   tab_out <- bind_rows(tabs)
    >   print(dim(tab_out))
    >   sapply(tabs, nrow) %>% sum %>% print
    >   write_tsv(tab_out, "raw_concat_qc_basic_stats.tsv.gz")
    > 
    > Command exit status:
    >   125
    > 
    > Command output:
    >   (empty)
    > 
    > Command error:
    >   c0e4127c6ef4: Retrying in 1 second
    >   802254caa4ee: Retrying in 8 seconds
    >   802254caa4ee: Retrying in 7 seconds
    >   c0e4127c6ef4: Downloading
    >   802254caa4ee: Retrying in 6 seconds
    >   802254caa4ee: Retrying in 5 seconds
    >   802254caa4ee: Retrying in 4 seconds
    >   802254caa4ee: Retrying in 3 seconds
    >   802254caa4ee: Retrying in 2 seconds
    >   802254caa4ee: Retrying in 1 second
    >   802254caa4ee: Downloading
    >   802254caa4ee: Retrying in 15 seconds
    >   802254caa4ee: Retrying in 14 seconds
    >   802254caa4ee: Retrying in 13 seconds
    >   802254caa4ee: Retrying in 12 seconds
    >   802254caa4ee: Retrying in 11 seconds
    >   802254caa4ee: Retrying in [10](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:11) seconds
    >   802254caa4ee: Retrying in 9 seconds
    >   802254caa4ee: Retrying in 8 seconds
    >   802254caa4ee: Retrying in 7 seconds
    >   802254caa4ee: Retrying in 6 seconds
    >   802254caa4ee: Retrying in 5 seconds
    >   802254caa4ee: Retrying in 4 seconds
    >   802254caa4ee: Retrying in 3 seconds
    >   802254caa4ee: Retrying in 2 seconds
    >   802254caa4ee: Retrying in 1 second
    >   802254caa4ee: Retrying in 20 seconds
    >   802254caa4ee: Retrying in 19 seconds
    >   802254caa4ee: Retrying in 18 seconds
    >   802254caa4ee: Retrying in 17 seconds
    >   802254caa4ee: Retrying in 16 seconds
    >   802254caa4ee: Retrying in 15 seconds
    >   802254caa4ee: Retrying in 14 seconds
    >   802254caa4ee: Retrying in 13 seconds
    >   802254caa4ee: Retrying in 12 seconds
    >   802254caa4ee: Retrying in [11](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:12) seconds
    >   802254caa4ee: Retrying in 10 seconds
    >   802254caa4ee: Retrying in 9 seconds
    >   802254caa4ee: Retrying in 8 seconds
    >   802254caa4ee: Retrying in 7 seconds
    >   802254caa4ee: Retrying in 6 seconds
    >   802254caa4ee: Retrying in 5 seconds
    >   802254caa4ee: Retrying in 4 seconds
    >   802254caa4ee: Retrying in 3 seconds
    >   802254caa4ee: Retrying in 2 seconds
    >   802254caa4ee: Retrying in 1 second
    >   802254caa4ee: Downloading
    >   docker: unknown: <?xml version="1.0" encoding="UTF-8"?>
    >   <Error><Code>ExpiredToken</Code><Message>The provided token has expired.</Message><Token-0>IQoJb3JpZ2luX2VjEGIaCXVzLWVhc3QtMSJHMEUCIQCeIV4tfQcJ4orTMrZ5u4B8veMnlidVp99W1l6joTQRugIgDanE0IlIJAZ/aX61tAmHihmGywEMl7B15456kFL7MtIq7gQISxAAGgw3MTAwMTUwNDA4OTIiDGBDvMDxo9r+sdUzOCrLBIlSzKWaOEko/eQmED0w2OtilOeboB1PHAMnTlzh+Ru6CJ/sLjjwyQ5XsSNbDbWPezZZ/oKk2D8XEERqY+Xgma0H+PbLFkqROMKnNFtv7zvXwI7uYZdsov7YvSHnRZ+8KIztxvam+iBia3QAPgpqoP1Cv9mNw0kkB8aSMvjBNmqMyDkp7VJI+EJtFLnxTKQPDMeDR9eT4mFynBXuSMvoHNqoybqG7K2IRQA5wnhr/P6eZZ5xF3OySxo3P5Co9VeOkNOLpx2WaeZ+TqksZdFTjQML+j7MRi5NLDI7HUWP/P47APIbXrVaj4WFIs2BKvVXUeFwIJ+GxSsRu66YoXie53jnDWZiRAvfUOA2P3AKGYoVXlmwxN3UBDrbgFtUI6twAzH2UTBfviNoYYXHDa2EFjIQ8bLM1LdN554CCncJRzIheCBVDARw8+LIeT4zTJCRPJ4cdSL2QOFvYQiN8m3nEn09rMAUXOMRfGnvMXVkHC4EUtDtZUrZxKdHvA9MzqtVKMfYV3NPJtqxF9nxBqu4GxUIVhKTBozZtTMJtfDV2QsuwgvT3ZpY0Vr3qtBOnxGZ/cqwqGnCEjiYcIg43wSxmrUDR0/9StBCR4Fb1QTUqGcBVScT2R5oE5CKLoFxXkcCx8FNYz5B8Gqj0JCfFuKc/gUZQdCBMDPTFO8cp5brNfqimGtwfzaleQ65Co/4/HdlUGLWT4xdcEPPzxFSc/XCy0C7XefOhCPyc9Tu+FeoJZ6GTaI5YxcdtQHn+282ScMZA61LyiQ6NIMX2zR3MJyz8LsGOpoBWqE7+eLUXpfwp/Im0R2QzUeZdR85rxggo2vfOIG+jpJtye5KUxhxKvKN5pIiFOnnVADVhQNskCdF1SyntDNBK4IqE41480C/LdFyPuI362qp3EIVZ/aamajzroZUfw8EDyx7c+PEn6iz2CbJNKKpfFcvxMKM4yxVhFUWZdr/Xig10rH8E/KCLFsvAEO4jljVwNgxJZpLq51fqw==</Token-0><RequestId>XXTBWVERT9Z1XXV5</RequestId><HostId>asY4icJxCgIPWPkzZkKvjdKhjE0Ukw4Hmr1mihJ7J7ah+7z8a8mg3h0sabPaUrepodnjw471hbM=</HostId></Error>.
    >   See 'docker run --help'.
    > 
    > Work dir:
    >   /home/runner/work/mgs-workflow/mgs-workflow/.nf-test/tests/4ae1c1738a3e7a1cd8a9ce8f488ab570/work/9c/0e0866d01d98ff46588f1ba7004635
    > 
    > Container:
    >   wave.seqera.io/wt/5ac75ac6191e/rocker/tidyverse:4.4.1
    > 
    > Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
    > 
    >  -- Check '/home/runner/work/mgs-workflow/mgs-workflow/.nf-test/tests/4ae1c1738a3e7a1cd8a9ce8f488ab570/meta/nextflow.log' file for details
    > Execution cancelled -- Finishing pending tasks before exit
    FAILED (390.383s)

  Assertion failed: 
  
  assert workflow.success
         |        |
         workflow false
  
  Nextflow stdout:
  
  N E X T F L O W  ~  version 24.10.3
  Launching `/home/runner/work/mgs-workflow/mgs-workflow/main.nf` [admiring_stallman] DSL2 - revision: 73db19e0f9
  Downloading plugin [email protected]
  WARN: WORKFLOW OUTPUT DSL IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
  WARN: WORKFLOW OUTPUT DSL IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
  WARN: WORKFLOW OUTPUT DSL IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
  Downloading plugin [email protected]
  [94/67ea50] Submitted process > RUN:RAW:QC:FASTQC_LABELED (1)
  [d0/3647f3] Submitted process > RUN:CLEAN:FILTER_READS (1)
  [9f/eb6025] Submitted process > RUN:RAW:QC:MULTIQC_LABELED (1)
  [24/d22e68] Submitted process > RUN:CLEAN:QC:FASTQC_LABELED (1)
  [5d/1c48d5] Submitted process > RUN:PROFILE:SUBSET_READS_TARGET (1)
  [54/1f6453] Submitted process > RUN:CLEAN:QC:MULTIQC_LABELED (1)
  Staging foreign file: s3://nao-testing/index-test/output/results/ribo-ref-concat.fasta.gz
  [87/1fc755] Submitted process > RUN:EXTRACT_VIRAL_READS:BBDUK_HITS (1)
  [6e/d760af] Submitted process > RUN:PROFILE:BBDUK (1)
  [64/fdca31] Submitted process > RUN:CLEAN:QC:SUMMARIZE_MULTIQC (1)
  [cf/8dea48] Submitted process > RUN:RAW:QC:SUMMARIZE_MULTIQC (1)
  [76/f91a7d] Submitted process > RUN:EXTRACT_RAW_READS_FROM_PROCESSED (1)
  [40/942af5] Submitted process > RUN:PROFILE:TAXONOMY_NORIBO:BBMERGE (1)
  [19/fa8[12](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:13)b] Submitted process > RUN:PROFILE:TAXONOMY_RIBO:BBMERGE (1)
  [6f/0589c1] Submitted process > RUN:PROFILE:TAXONOMY_NORIBO:SUMMARIZE_BBMERGE (1)
  [5c/b45ec3] Submitted process > RUN:PROFILE:TAXONOMY_NORIBO:JOIN_FASTQ (1)
  [ae/b4ea7a] Submitted process > RUN:PROFILE:TAXONOMY_RIBO:SUMMARIZE_BBMERGE (1)
  [ad/f49455] Submitted process > RUN:PROFILE:TAXONOMY_RIBO:JOIN_FASTQ (1)
  [61/38d663] Submitted process > RUN:EXTRACT_VIRAL_READS:CUTADAPT (1)
  Staging foreign file: s3://nao-testing/index-test/output/results/kraken_db
  [9c/0e0866] Submitted process > RUN:RAW:QC:MERGE_MULTIQC_BASIC
  [b3/6a0269] Submitted process > RUN:CLEAN:QC:MERGE_MULTIQC_QBASE
  ERROR ~ Error executing process > 'RUN:RAW:QC:MERGE_MULTIQC_BASIC'
  
  Caused by:
    Process `RUN:RAW:QC:MERGE_MULTIQC_BASIC` terminated with an error exit status (125)
  
  
  Command executed:
  
    #!/usr/bin/env Rscript
    library(tidyverse)
    in_paths <- str_split("raw_concat_gold_standard_qc_basic_stats.tsv.gz", " ")[[1]]
    print(in_paths)
    tabs <- lapply(in_paths, function(t) read_tsv(t, col_names = TRUE, cols(.default="c")))
    for (t in tabs) print(dim(t))
    tab_out <- bind_rows(tabs)
    print(dim(tab_out))
    sapply(tabs, nrow) %>% sum %>% print
    write_tsv(tab_out, "raw_concat_qc_basic_stats.tsv.gz")
  
  Command exit status:
    125
  
  Command output:
    (empty)
  
  Command error:
    c0e4127c6ef4: Retrying in 1 second
    802254caa4ee: Retrying in 8 seconds
    802254caa4ee: Retrying in 7 seconds
    c0e4127c6ef4: Downloading
    802254caa4ee: Retrying in 6 seconds
    802254caa4ee: Retrying in 5 seconds
    802254caa4ee: Retrying in 4 seconds
    802254caa4ee: Retrying in 3 seconds
    802254caa4ee: Retrying in 2 seconds
    802254caa4ee: Retrying in 1 second
    802254caa4ee: Downloading
    802254caa4ee: Retrying in 15 seconds
    802254caa4ee: Retrying in 14 seconds
    802254caa4ee: Retrying in [13](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:14) seconds
    802254caa4ee: Retrying in 12 seconds
    802254caa4ee: Retrying in 11 seconds
    802254caa4ee: Retrying in 10 seconds
    802254caa4ee: Retrying in 9 seconds
    802254caa4ee: Retrying in 8 seconds
    802254caa4ee: Retrying in 7 seconds
    802254caa4ee: Retrying in 6 seconds
    802254caa4ee: Retrying in 5 seconds
    802254caa4ee: Retrying in 4 seconds
    802254caa4ee: Retrying in 3 seconds
    802254caa4ee: Retrying in 2 seconds
    802254caa4ee: Retrying in 1 second
    802254caa4ee: Retrying in 20 seconds
    802254caa4ee: Retrying in 19 seconds
    802254caa4ee: Retrying in 18 seconds
    802254caa4ee: Retrying in 17 seconds
    802254caa4ee: Retrying in 16 seconds
    802254caa4ee: Retrying in 15 seconds
    802254caa4ee: Retrying in [14](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:15) seconds
    802254caa4ee: Retrying in 13 seconds
    802254caa4ee: Retrying in 12 seconds
    802254caa4ee: Retrying in 11 seconds
    802254caa4ee: Retrying in 10 seconds
    802254caa4ee: Retrying in 9 seconds
    802254caa4ee: Retrying in 8 seconds
    802254caa4ee: Retrying in 7 seconds
    802254caa4ee: Retrying in 6 seconds
    802254caa4ee: Retrying in 5 seconds
    802254caa4ee: Retrying in 4 seconds
    802254caa4ee: Retrying in 3 seconds
    802254caa4ee: Retrying in 2 seconds
    802254caa4ee: Retrying in 1 second
    802254caa4ee: Downloading
    docker: unknown: <?xml version="1.0" encoding="UTF-8"?>
    <Error><Code>ExpiredToken</Code><Message>The provided token has expired.</Message><Token-0>IQoJb3JpZ2luX2VjEGIaCXVzLWVhc3QtMSJHMEUCIQCeIV4tfQcJ4orTMrZ5u4B8veMnlidVp99W1l6joTQRugIgDanE0IlIJAZ/aX61tAmHihmGywEMl7B[15](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:16)456kFL7MtIq7gQISxAAGgw3MTAwMTUwNDA4OTIiDGBDvMDxo9r+sdUzOCrLBIlSzKWaOEko/eQmED0w2OtilOeboB1PHAMnTlzh+Ru6CJ/sLjjwyQ5XsSNbDbWPezZZ/oKk2D8XEERqY+Xgma0H+PbLFkqROMKnNFtv7zvXwI7uYZdsov7YvSHnRZ+8KIztxvam+iBia3QAPgpqoP1Cv9mNw0kkB8aSMvjBNmqMyDkp7VJI+EJtFLnxTKQPDMeDR9eT4mFynBXuSMvoHNqoybqG7K2IRQA5wnhr/P6eZZ5xF3OySxo3P5Co9VeOkNOLpx2WaeZ+TqksZdFTjQML+j7MRi5NLDI7HUWP/P47APIbXrVaj4WFIs2BKvVXUeFwIJ+GxSsRu66YoXie53jnDWZiRAvfUOA2P3AKGYoVXlmwxN3UBDrbgFtUI6twAzH2UTBfviNoYYXHDa2EFjIQ8bLM1LdN554CCncJRzIheCBVDARw8+LIeT4zTJCRPJ4cdSL2QOFvYQiN8m3nEn09rMAUXOMRfGnvMXVkHC4EUtDtZUrZxKdHvA9MzqtVKMfYV3NPJtqxF9nxBqu4GxUIVhKTBozZtTMJtfDV2QsuwgvT3ZpY0Vr3qtBOnxGZ/cqwqGnCEjiYcIg43wSxmrUDR0/9StBCR4Fb1QTUqGcBVScT2R5oE5CKLoFxXkcCx8FNYz5B8Gqj0JCfFuKc/gUZQdCBMDPTFO8cp5brNfqimGtwfzaleQ65Co/4/HdlUGLWT4xdcEPPzxFSc/XCy0C7XefOhCPyc9Tu+FeoJZ6GTaI5YxcdtQHn+282ScMZA61LyiQ6NIMX2zR3MJyz8LsGOpoBWqE7+eLUXpfwp/Im0R2QzUeZdR85rxggo2vfOIG+jpJtye5KUxhxKvKN5pIiFOnnVADVhQNskCdF1SyntDNBK4IqE41480C/LdFyPuI362qp3EIVZ/aamajzroZUfw8EDyx7c+PEn6iz2CbJNKKpfFcvxMKM4yxVhFUWZdr/Xig10rH8E/KCLFsvAEO4jljVwNgxJZpLq51fqw==</Token-0><RequestId>XXTBWVERT9Z1XXV5</RequestId><HostId>asY4icJxCgIPWPkzZkKvjdKhjE0Ukw4Hmr1mihJ7J7ah+7z8a8mg3h0sabPaUrepodnjw471hbM=</HostId></Error>.
    See 'docker run --help'.
  
  Work dir:
    /home/runner/work/mgs-workflow/mgs-workflow/.nf-test/tests/4ae1c[17](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:18)38a3e7a1cd8a9ce8f488ab570/work/9c/0e0866d01d98ff46588f1ba7004635
  
  Container:
    wave.seqera.io/wt/5ac75ac6191e/rocker/tidyverse:4.4.1
  
  Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
  
   -- Check '/home/runner/work/mgs-workflow/mgs-workflow/.nf-test/tests/4ae1c1738a3e7a1cd8a9ce8f488ab570/meta/nextflow.log' file for details
  Execution cancelled -- Finishing pending tasks before exit
  Nextflow stderr:
  
  



FAILURE: Executed 1 tests in 390.406s (1 failed)

@harmonbhasin
Copy link
Collaborator

@harmonbhasin This is the error I get when nf-test is run on github. nf-test works locally. Can you let me know if you see what might be wrong here?

Otherwise I adopted your suggestions. Once the nf-test thing is resolved this is good for review by Will.

Run nf-test test --tag run --verbose

🚀 nf-test 0.9.2
https://www.nf-test.com
(c) 2021 - 2024 Lukas Forer and Sebastian Schoenherr


End-to-end test of MGS workflow

  Test [4ae1c173] 'Test run workflow' 
    > N E X T F L O W  ~  version 24.10.3
    > Launching `/home/runner/work/mgs-workflow/mgs-workflow/main.nf` [admiring_stallman] DSL2 - revision: 73db19e0f9
    > Downloading plugin [email protected]
    > WARN: WORKFLOW OUTPUT DSL IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
    > WARN: WORKFLOW OUTPUT DSL IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
    > WARN: WORKFLOW OUTPUT DSL IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
    > Downloading plugin [email protected]
    > [94/67ea50] Submitted process > RUN:RAW:QC:FASTQC_LABELED (1)
    > [d0/3647f3] Submitted process > RUN:CLEAN:FILTER_READS (1)
    > [9f/eb6025] Submitted process > RUN:RAW:QC:MULTIQC_LABELED (1)
    > [24/d22e6[8](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:9)] Submitted process > RUN:CLEAN:QC:FASTQC_LABELED (1)
    > [5d/1c48d5] Submitted process > RUN:PROFILE:SUBSET_READS_TARGET (1)
    > [54/1f6453] Submitted process > RUN:CLEAN:QC:MULTIQC_LABELED (1)
    > Staging foreign file: s3://nao-testing/index-test/output/results/ribo-ref-concat.fasta.gz
    > [87/1fc755] Submitted process > RUN:EXTRACT_VIRAL_READS:BBDUK_HITS (1)
    > [6e/d760af] Submitted process > RUN:PROFILE:BBDUK (1)
    > [64/fdca31] Submitted process > RUN:CLEAN:QC:SUMMARIZE_MULTIQC (1)
    > [cf/8dea48] Submitted process > RUN:RAW:QC:SUMMARIZE_MULTIQC (1)
    > [76/f[9](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:10)1a7d] Submitted process > RUN:EXTRACT_RAW_READS_FROM_PROCESSED (1)
    > [40/942af5] Submitted process > RUN:PROFILE:TAXONOMY_NORIBO:BBMERGE (1)
    > [19/fa812b] Submitted process > RUN:PROFILE:TAXONOMY_RIBO:BBMERGE (1)
    > [6f/0589c1] Submitted process > RUN:PROFILE:TAXONOMY_NORIBO:SUMMARIZE_BBMERGE (1)
    > [5c/b45ec3] Submitted process > RUN:PROFILE:TAXONOMY_NORIBO:JOIN_FASTQ (1)
    > [ae/b4ea7a] Submitted process > RUN:PROFILE:TAXONOMY_RIBO:SUMMARIZE_BBMERGE (1)
    > [ad/f49455] Submitted process > RUN:PROFILE:TAXONOMY_RIBO:JOIN_FASTQ (1)
    > [61/38d663] Submitted process > RUN:EXTRACT_VIRAL_READS:CUTADAPT (1)
    > Staging foreign file: s3://nao-testing/index-test/output/results/kraken_db
    > [9c/0e0866] Submitted process > RUN:RAW:QC:MERGE_MULTIQC_BASIC
    > [b3/6a0269] Submitted process > RUN:CLEAN:QC:MERGE_MULTIQC_QBASE
    > ERROR ~ Error executing process > 'RUN:RAW:QC:MERGE_MULTIQC_BASIC'
    > 
    > Caused by:
    >   Process `RUN:RAW:QC:MERGE_MULTIQC_BASIC` terminated with an error exit status (125)
    > 
    > 
    > Command executed:
    > 
    >   #!/usr/bin/env Rscript
    >   library(tidyverse)
    >   in_paths <- str_split("raw_concat_gold_standard_qc_basic_stats.tsv.gz", " ")[[1]]
    >   print(in_paths)
    >   tabs <- lapply(in_paths, function(t) read_tsv(t, col_names = TRUE, cols(.default="c")))
    >   for (t in tabs) print(dim(t))
    >   tab_out <- bind_rows(tabs)
    >   print(dim(tab_out))
    >   sapply(tabs, nrow) %>% sum %>% print
    >   write_tsv(tab_out, "raw_concat_qc_basic_stats.tsv.gz")
    > 
    > Command exit status:
    >   125
    > 
    > Command output:
    >   (empty)
    > 
    > Command error:
    >   c0e4127c6ef4: Retrying in 1 second
    >   802254caa4ee: Retrying in 8 seconds
    >   802254caa4ee: Retrying in 7 seconds
    >   c0e4127c6ef4: Downloading
    >   802254caa4ee: Retrying in 6 seconds
    >   802254caa4ee: Retrying in 5 seconds
    >   802254caa4ee: Retrying in 4 seconds
    >   802254caa4ee: Retrying in 3 seconds
    >   802254caa4ee: Retrying in 2 seconds
    >   802254caa4ee: Retrying in 1 second
    >   802254caa4ee: Downloading
    >   802254caa4ee: Retrying in 15 seconds
    >   802254caa4ee: Retrying in 14 seconds
    >   802254caa4ee: Retrying in 13 seconds
    >   802254caa4ee: Retrying in 12 seconds
    >   802254caa4ee: Retrying in 11 seconds
    >   802254caa4ee: Retrying in [10](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:11) seconds
    >   802254caa4ee: Retrying in 9 seconds
    >   802254caa4ee: Retrying in 8 seconds
    >   802254caa4ee: Retrying in 7 seconds
    >   802254caa4ee: Retrying in 6 seconds
    >   802254caa4ee: Retrying in 5 seconds
    >   802254caa4ee: Retrying in 4 seconds
    >   802254caa4ee: Retrying in 3 seconds
    >   802254caa4ee: Retrying in 2 seconds
    >   802254caa4ee: Retrying in 1 second
    >   802254caa4ee: Retrying in 20 seconds
    >   802254caa4ee: Retrying in 19 seconds
    >   802254caa4ee: Retrying in 18 seconds
    >   802254caa4ee: Retrying in 17 seconds
    >   802254caa4ee: Retrying in 16 seconds
    >   802254caa4ee: Retrying in 15 seconds
    >   802254caa4ee: Retrying in 14 seconds
    >   802254caa4ee: Retrying in 13 seconds
    >   802254caa4ee: Retrying in 12 seconds
    >   802254caa4ee: Retrying in [11](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:12) seconds
    >   802254caa4ee: Retrying in 10 seconds
    >   802254caa4ee: Retrying in 9 seconds
    >   802254caa4ee: Retrying in 8 seconds
    >   802254caa4ee: Retrying in 7 seconds
    >   802254caa4ee: Retrying in 6 seconds
    >   802254caa4ee: Retrying in 5 seconds
    >   802254caa4ee: Retrying in 4 seconds
    >   802254caa4ee: Retrying in 3 seconds
    >   802254caa4ee: Retrying in 2 seconds
    >   802254caa4ee: Retrying in 1 second
    >   802254caa4ee: Downloading
    >   docker: unknown: <?xml version="1.0" encoding="UTF-8"?>
    >   <Error><Code>ExpiredToken</Code><Message>The provided token has expired.</Message><Token-0>IQoJb3JpZ2luX2VjEGIaCXVzLWVhc3QtMSJHMEUCIQCeIV4tfQcJ4orTMrZ5u4B8veMnlidVp99W1l6joTQRugIgDanE0IlIJAZ/aX61tAmHihmGywEMl7B15456kFL7MtIq7gQISxAAGgw3MTAwMTUwNDA4OTIiDGBDvMDxo9r+sdUzOCrLBIlSzKWaOEko/eQmED0w2OtilOeboB1PHAMnTlzh+Ru6CJ/sLjjwyQ5XsSNbDbWPezZZ/oKk2D8XEERqY+Xgma0H+PbLFkqROMKnNFtv7zvXwI7uYZdsov7YvSHnRZ+8KIztxvam+iBia3QAPgpqoP1Cv9mNw0kkB8aSMvjBNmqMyDkp7VJI+EJtFLnxTKQPDMeDR9eT4mFynBXuSMvoHNqoybqG7K2IRQA5wnhr/P6eZZ5xF3OySxo3P5Co9VeOkNOLpx2WaeZ+TqksZdFTjQML+j7MRi5NLDI7HUWP/P47APIbXrVaj4WFIs2BKvVXUeFwIJ+GxSsRu66YoXie53jnDWZiRAvfUOA2P3AKGYoVXlmwxN3UBDrbgFtUI6twAzH2UTBfviNoYYXHDa2EFjIQ8bLM1LdN554CCncJRzIheCBVDARw8+LIeT4zTJCRPJ4cdSL2QOFvYQiN8m3nEn09rMAUXOMRfGnvMXVkHC4EUtDtZUrZxKdHvA9MzqtVKMfYV3NPJtqxF9nxBqu4GxUIVhKTBozZtTMJtfDV2QsuwgvT3ZpY0Vr3qtBOnxGZ/cqwqGnCEjiYcIg43wSxmrUDR0/9StBCR4Fb1QTUqGcBVScT2R5oE5CKLoFxXkcCx8FNYz5B8Gqj0JCfFuKc/gUZQdCBMDPTFO8cp5brNfqimGtwfzaleQ65Co/4/HdlUGLWT4xdcEPPzxFSc/XCy0C7XefOhCPyc9Tu+FeoJZ6GTaI5YxcdtQHn+282ScMZA61LyiQ6NIMX2zR3MJyz8LsGOpoBWqE7+eLUXpfwp/Im0R2QzUeZdR85rxggo2vfOIG+jpJtye5KUxhxKvKN5pIiFOnnVADVhQNskCdF1SyntDNBK4IqE41480C/LdFyPuI362qp3EIVZ/aamajzroZUfw8EDyx7c+PEn6iz2CbJNKKpfFcvxMKM4yxVhFUWZdr/Xig10rH8E/KCLFsvAEO4jljVwNgxJZpLq51fqw==</Token-0><RequestId>XXTBWVERT9Z1XXV5</RequestId><HostId>asY4icJxCgIPWPkzZkKvjdKhjE0Ukw4Hmr1mihJ7J7ah+7z8a8mg3h0sabPaUrepodnjw471hbM=</HostId></Error>.
    >   See 'docker run --help'.
    > 
    > Work dir:
    >   /home/runner/work/mgs-workflow/mgs-workflow/.nf-test/tests/4ae1c1738a3e7a1cd8a9ce8f488ab570/work/9c/0e0866d01d98ff46588f1ba7004635
    > 
    > Container:
    >   wave.seqera.io/wt/5ac75ac6191e/rocker/tidyverse:4.4.1
    > 
    > Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
    > 
    >  -- Check '/home/runner/work/mgs-workflow/mgs-workflow/.nf-test/tests/4ae1c1738a3e7a1cd8a9ce8f488ab570/meta/nextflow.log' file for details
    > Execution cancelled -- Finishing pending tasks before exit
    FAILED (390.383s)

  Assertion failed: 
  
  assert workflow.success
         |        |
         workflow false
  
  Nextflow stdout:
  
  N E X T F L O W  ~  version 24.10.3
  Launching `/home/runner/work/mgs-workflow/mgs-workflow/main.nf` [admiring_stallman] DSL2 - revision: 73db19e0f9
  Downloading plugin [email protected]
  WARN: WORKFLOW OUTPUT DSL IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
  WARN: WORKFLOW OUTPUT DSL IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
  WARN: WORKFLOW OUTPUT DSL IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
  Downloading plugin [email protected]
  [94/67ea50] Submitted process > RUN:RAW:QC:FASTQC_LABELED (1)
  [d0/3647f3] Submitted process > RUN:CLEAN:FILTER_READS (1)
  [9f/eb6025] Submitted process > RUN:RAW:QC:MULTIQC_LABELED (1)
  [24/d22e68] Submitted process > RUN:CLEAN:QC:FASTQC_LABELED (1)
  [5d/1c48d5] Submitted process > RUN:PROFILE:SUBSET_READS_TARGET (1)
  [54/1f6453] Submitted process > RUN:CLEAN:QC:MULTIQC_LABELED (1)
  Staging foreign file: s3://nao-testing/index-test/output/results/ribo-ref-concat.fasta.gz
  [87/1fc755] Submitted process > RUN:EXTRACT_VIRAL_READS:BBDUK_HITS (1)
  [6e/d760af] Submitted process > RUN:PROFILE:BBDUK (1)
  [64/fdca31] Submitted process > RUN:CLEAN:QC:SUMMARIZE_MULTIQC (1)
  [cf/8dea48] Submitted process > RUN:RAW:QC:SUMMARIZE_MULTIQC (1)
  [76/f91a7d] Submitted process > RUN:EXTRACT_RAW_READS_FROM_PROCESSED (1)
  [40/942af5] Submitted process > RUN:PROFILE:TAXONOMY_NORIBO:BBMERGE (1)
  [19/fa8[12](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:13)b] Submitted process > RUN:PROFILE:TAXONOMY_RIBO:BBMERGE (1)
  [6f/0589c1] Submitted process > RUN:PROFILE:TAXONOMY_NORIBO:SUMMARIZE_BBMERGE (1)
  [5c/b45ec3] Submitted process > RUN:PROFILE:TAXONOMY_NORIBO:JOIN_FASTQ (1)
  [ae/b4ea7a] Submitted process > RUN:PROFILE:TAXONOMY_RIBO:SUMMARIZE_BBMERGE (1)
  [ad/f49455] Submitted process > RUN:PROFILE:TAXONOMY_RIBO:JOIN_FASTQ (1)
  [61/38d663] Submitted process > RUN:EXTRACT_VIRAL_READS:CUTADAPT (1)
  Staging foreign file: s3://nao-testing/index-test/output/results/kraken_db
  [9c/0e0866] Submitted process > RUN:RAW:QC:MERGE_MULTIQC_BASIC
  [b3/6a0269] Submitted process > RUN:CLEAN:QC:MERGE_MULTIQC_QBASE
  ERROR ~ Error executing process > 'RUN:RAW:QC:MERGE_MULTIQC_BASIC'
  
  Caused by:
    Process `RUN:RAW:QC:MERGE_MULTIQC_BASIC` terminated with an error exit status (125)
  
  
  Command executed:
  
    #!/usr/bin/env Rscript
    library(tidyverse)
    in_paths <- str_split("raw_concat_gold_standard_qc_basic_stats.tsv.gz", " ")[[1]]
    print(in_paths)
    tabs <- lapply(in_paths, function(t) read_tsv(t, col_names = TRUE, cols(.default="c")))
    for (t in tabs) print(dim(t))
    tab_out <- bind_rows(tabs)
    print(dim(tab_out))
    sapply(tabs, nrow) %>% sum %>% print
    write_tsv(tab_out, "raw_concat_qc_basic_stats.tsv.gz")
  
  Command exit status:
    125
  
  Command output:
    (empty)
  
  Command error:
    c0e4127c6ef4: Retrying in 1 second
    802254caa4ee: Retrying in 8 seconds
    802254caa4ee: Retrying in 7 seconds
    c0e4127c6ef4: Downloading
    802254caa4ee: Retrying in 6 seconds
    802254caa4ee: Retrying in 5 seconds
    802254caa4ee: Retrying in 4 seconds
    802254caa4ee: Retrying in 3 seconds
    802254caa4ee: Retrying in 2 seconds
    802254caa4ee: Retrying in 1 second
    802254caa4ee: Downloading
    802254caa4ee: Retrying in 15 seconds
    802254caa4ee: Retrying in 14 seconds
    802254caa4ee: Retrying in [13](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:14) seconds
    802254caa4ee: Retrying in 12 seconds
    802254caa4ee: Retrying in 11 seconds
    802254caa4ee: Retrying in 10 seconds
    802254caa4ee: Retrying in 9 seconds
    802254caa4ee: Retrying in 8 seconds
    802254caa4ee: Retrying in 7 seconds
    802254caa4ee: Retrying in 6 seconds
    802254caa4ee: Retrying in 5 seconds
    802254caa4ee: Retrying in 4 seconds
    802254caa4ee: Retrying in 3 seconds
    802254caa4ee: Retrying in 2 seconds
    802254caa4ee: Retrying in 1 second
    802254caa4ee: Retrying in 20 seconds
    802254caa4ee: Retrying in 19 seconds
    802254caa4ee: Retrying in 18 seconds
    802254caa4ee: Retrying in 17 seconds
    802254caa4ee: Retrying in 16 seconds
    802254caa4ee: Retrying in 15 seconds
    802254caa4ee: Retrying in [14](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:15) seconds
    802254caa4ee: Retrying in 13 seconds
    802254caa4ee: Retrying in 12 seconds
    802254caa4ee: Retrying in 11 seconds
    802254caa4ee: Retrying in 10 seconds
    802254caa4ee: Retrying in 9 seconds
    802254caa4ee: Retrying in 8 seconds
    802254caa4ee: Retrying in 7 seconds
    802254caa4ee: Retrying in 6 seconds
    802254caa4ee: Retrying in 5 seconds
    802254caa4ee: Retrying in 4 seconds
    802254caa4ee: Retrying in 3 seconds
    802254caa4ee: Retrying in 2 seconds
    802254caa4ee: Retrying in 1 second
    802254caa4ee: Downloading
    docker: unknown: <?xml version="1.0" encoding="UTF-8"?>
    <Error><Code>ExpiredToken</Code><Message>The provided token has expired.</Message><Token-0>IQoJb3JpZ2luX2VjEGIaCXVzLWVhc3QtMSJHMEUCIQCeIV4tfQcJ4orTMrZ5u4B8veMnlidVp99W1l6joTQRugIgDanE0IlIJAZ/aX61tAmHihmGywEMl7B[15](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:16)456kFL7MtIq7gQISxAAGgw3MTAwMTUwNDA4OTIiDGBDvMDxo9r+sdUzOCrLBIlSzKWaOEko/eQmED0w2OtilOeboB1PHAMnTlzh+Ru6CJ/sLjjwyQ5XsSNbDbWPezZZ/oKk2D8XEERqY+Xgma0H+PbLFkqROMKnNFtv7zvXwI7uYZdsov7YvSHnRZ+8KIztxvam+iBia3QAPgpqoP1Cv9mNw0kkB8aSMvjBNmqMyDkp7VJI+EJtFLnxTKQPDMeDR9eT4mFynBXuSMvoHNqoybqG7K2IRQA5wnhr/P6eZZ5xF3OySxo3P5Co9VeOkNOLpx2WaeZ+TqksZdFTjQML+j7MRi5NLDI7HUWP/P47APIbXrVaj4WFIs2BKvVXUeFwIJ+GxSsRu66YoXie53jnDWZiRAvfUOA2P3AKGYoVXlmwxN3UBDrbgFtUI6twAzH2UTBfviNoYYXHDa2EFjIQ8bLM1LdN554CCncJRzIheCBVDARw8+LIeT4zTJCRPJ4cdSL2QOFvYQiN8m3nEn09rMAUXOMRfGnvMXVkHC4EUtDtZUrZxKdHvA9MzqtVKMfYV3NPJtqxF9nxBqu4GxUIVhKTBozZtTMJtfDV2QsuwgvT3ZpY0Vr3qtBOnxGZ/cqwqGnCEjiYcIg43wSxmrUDR0/9StBCR4Fb1QTUqGcBVScT2R5oE5CKLoFxXkcCx8FNYz5B8Gqj0JCfFuKc/gUZQdCBMDPTFO8cp5brNfqimGtwfzaleQ65Co/4/HdlUGLWT4xdcEPPzxFSc/XCy0C7XefOhCPyc9Tu+FeoJZ6GTaI5YxcdtQHn+282ScMZA61LyiQ6NIMX2zR3MJyz8LsGOpoBWqE7+eLUXpfwp/Im0R2QzUeZdR85rxggo2vfOIG+jpJtye5KUxhxKvKN5pIiFOnnVADVhQNskCdF1SyntDNBK4IqE41480C/LdFyPuI362qp3EIVZ/aamajzroZUfw8EDyx7c+PEn6iz2CbJNKKpfFcvxMKM4yxVhFUWZdr/Xig10rH8E/KCLFsvAEO4jljVwNgxJZpLq51fqw==</Token-0><RequestId>XXTBWVERT9Z1XXV5</RequestId><HostId>asY4icJxCgIPWPkzZkKvjdKhjE0Ukw4Hmr1mihJ7J7ah+7z8a8mg3h0sabPaUrepodnjw471hbM=</HostId></Error>.
    See 'docker run --help'.
  
  Work dir:
    /home/runner/work/mgs-workflow/mgs-workflow/.nf-test/tests/4ae1c[17](https://github.com/naobservatory/mgs-workflow/actions/runs/12638560900/job/35215087099?pr=133#step:6:18)38a3e7a1cd8a9ce8f488ab570/work/9c/0e0866d01d98ff46588f1ba7004635
  
  Container:
    wave.seqera.io/wt/5ac75ac6191e/rocker/tidyverse:4.4.1
  
  Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
  
   -- Check '/home/runner/work/mgs-workflow/mgs-workflow/.nf-test/tests/4ae1c1738a3e7a1cd8a9ce8f488ab570/meta/nextflow.log' file for details
  Execution cancelled -- Finishing pending tasks before exit
  Nextflow stderr:
  
  



FAILURE: Executed 1 tests in 390.406s (1 failed)

@simonleandergrimm specific docker images from sequera (and even dockerhub for that matter) have given me trouble on Github Actions in the past, and my guess is that you're having a similar issue, although I have not run into this specific issue before. As of right now, the best way I know of fixing issues with docker images issues related to Github Actions is trial-and-error. The first thing I would do is try a different distribution of tidyverse, this might be a good place to start, https://hub.docker.com/r/rocker/tidyverse.

Copy link
Collaborator

@harmonbhasin harmonbhasin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks good to me! Once the docker image issue is fixed, it'll be ready for Will.

@simonleandergrimm
Copy link
Collaborator Author

@harmonbhasin The run works now (updated tidyverse container from 4.4.1 to 4.4.2). Index run now fails, but probably due to timing out as has happened previously? If this seems fine, please assign to Will and ping for re-review.

@simonleandergrimm
Copy link
Collaborator Author

Also @harmonbhasin can you "assign" the PR to me when I'm blocking it? This makes it easier to know in which court the PR is!

@simonleandergrimm
Copy link
Collaborator Author

@willbradshaw Can you review? Once done, I will create a Changelog.

Copy link
Collaborator

@harmonbhasin harmonbhasin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@simonleandergrimm I'll try to add you as PR in the future when the ball is in your court, thanks for fixing this!

@@ -55,7 +55,7 @@ process {
container = "securebio/nao-pypkg"
}
withLabel: tidyverse {
container = "rocker/tidyverse:4.4.1"
container = "rocker/tidyverse:4.4.2"
Copy link
Collaborator

@harmonbhasin harmonbhasin Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turns out my test ran fine with the old version of the docker container, #138, so this was probably just a temporary error. However, all we're doing is updating the docker container, which I have no issue with so this looks good to me.

@harmonbhasin harmonbhasin self-requested a review January 10, 2025 20:11
@harmonbhasin
Copy link
Collaborator

@simonleandergrimm I was trying to run the BLAST workflow (configs/run_validation.config) and noticed that I was getting an error. This error goes away when I remove includeConfig "${projectDir}/configs/read_type.config from the config file. I'm not sure if this is supposed to work for the BLAST workflow or not, so could you fix it if it's supposed to, and if it's not supposed to, could you fix it by removing this line from configs/run_validation.config. I would also check if the index workflow if you've added this statement there.

Also, make sure to update the tests with any new import statements you add so that we don't run into these problems in the future. I'll be sure to look out for these things in the future as well!

@simonleandergrimm
Copy link
Collaborator Author

This issue was in dev, I fixed it there, see #140

Copy link
Collaborator

@harmonbhasin harmonbhasin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for fixing the issue, this is good for you to look at @willbradshaw.

@willbradshaw
Copy link
Contributor

@simonleandergrimm based on our discussion yesterday, are you planning on adding any tests before we merge this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants