Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chewbbaca batch input #2

Conversation

ryanjameskennedy
Copy link

The following changes were made:

  • bwa output filepath was changed to avoid error listed in this issue.
  • chewbbaca batch input list generator was created to avoid multiple processes competing for the same databases whilst running in parallel (issue).
  • Modules involved in chewbbaca were updated to separate the resulting chewbbaca batch output.

path input

output:
path("${output}")
tuple val(sampleName), path("${output}")

script:
id = "${input.simpleName}"
Copy link

@LordRust LordRust Feb 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need line 86? Which seems to have the wrong name anyways (simpleName)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simpleName is a nextflow function that strips extension and filepath base. Regardless, it should be removed as it is old code.

Copy link

@LordRust LordRust left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file locking is not global on a multi-node cluster and would only work on a single machine, unless localTempDir is supposed to be a global temp dir.

Comment on lines 34 to 37
trainingFile = trainingFile ? "--ptf ${trainingFile}" : ""
flockfile = file(params.localTempDir + '/chewbbaca.lock')
flocking = flockfile.exists() ? "flock -e $flockfile \\" : ""

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only a local lock on a single node? Would fail with multiple nodes I think

Copy link

@LordRust LordRust left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not think the file lock works as supposed. Have a look into that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants