Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding single-read functionality to RAW and CLEAN #80

Merged
merged 108 commits into from
Dec 20, 2024
Merged
Changes from 1 commit
Commits
Show all changes
108 commits
Select commit Hold shift + click to select a range
15354f6
Adding single read option to raw/main.nf
simonleandergrimm Oct 21, 2024
ad2115d
Adding WIP version of run.nf to enable testing raw and clean versions…
simonleandergrimm Oct 21, 2024
03ee37a
Created separate versions of summarize-multiqc-single.R and summarize…
simonleandergrimm Oct 22, 2024
b517340
Split processes in fastp to a single read and paired-end read version.
simonleandergrimm Oct 22, 2024
01ea0c5
Split processes in MultiQC to a single read and paired-end read versi…
simonleandergrimm Oct 22, 2024
ad8faf9
Deleted summarizeMultiqcSingle, which was superseded by summarizeMultiqc
simonleandergrimm Oct 22, 2024
ef0e9c8
Split processes in truncateConcat to a single read and paired-end rea…
simonleandergrimm Oct 22, 2024
2535ccd
Created a single_end if clause in Clean to either use the single read…
simonleandergrimm Oct 22, 2024
cbcb109
Created a single_end if clause in hv_screen to either use the single …
simonleandergrimm Oct 22, 2024
c7f8c83
Created a single_end if clause in qc to either use the single read or…
simonleandergrimm Oct 22, 2024
ff0a8be
Renamed test dir to test-paired-end. Added clause in nextflow.config …
simonleandergrimm Oct 22, 2024
6048dd3
Edited gitignore to leave out test-paired-end and test-single-read ru…
simonleandergrimm Oct 22, 2024
92270e5
Fixed name of test-single-end dir to test-single-read
simonleandergrimm Oct 22, 2024
b13ac94
Created a version of test dir that allows the run of single-read data.
simonleandergrimm Oct 22, 2024
dff2302
Added script to quickly download the s3 output of test single read an…
simonleandergrimm Oct 23, 2024
64bb7f4
Added nextflow config for test paired and test single read.
simonleandergrimm Oct 23, 2024
5bd1aec
Fixed if clause in main.nf
simonleandergrimm Oct 23, 2024
c8fd3ac
Updated gen samplesheet scripts to pull in data from s3://nao-mgs-sim…
simonleandergrimm Oct 23, 2024
578fde0
Updated gitignore
simonleandergrimm Oct 23, 2024
59218b9
Activated CLEAN subworkflow in run.nf
simonleandergrimm Oct 23, 2024
fd9dc1e
Starting to adapt Will's https://data.securebio.org/wills-public-note…
simonleandergrimm Oct 23, 2024
81ff0ba
Adding ignoring mgs-results to gitignore
simonleandergrimm Oct 23, 2024
590b2c3
Adding Will's auxiliary scripts to run his quarto notebooks.
simonleandergrimm Oct 23, 2024
6a650b4
Merge branch 'master' into single-read-raw
simonleandergrimm Oct 23, 2024
9f1eb03
Amended qmd somewhat so data imports work.
simonleandergrimm Oct 24, 2024
9622004
Added a flag to summarize-multiqc-single.R that provides info on the…
simonleandergrimm Oct 25, 2024
c61ed0c
Amended logic of split_sample, so it does not split and pull out read…
simonleandergrimm Oct 25, 2024
f8d9c28
Deleting seperate version of summarize-multiqc I created for paired r…
simonleandergrimm Oct 25, 2024
8e1c7b5
Revert "Split processes in MultiQC to a single read and paired-end re…
simonleandergrimm Oct 25, 2024
0ba0552
Revert "Deleted summarizeMultiqcSingle, which was superseded by summa…
simonleandergrimm Oct 25, 2024
8bafee8
Revert "Created a single_end if clause in qc to either use the single…
simonleandergrimm Oct 25, 2024
68c7c50
Amended main.nf of summarizeMultiqcSingle, clean, qc, and raw, to pro…
simonleandergrimm Oct 25, 2024
1656b33
Amended summarize-multiqc-single.R's basic_info_fastqc so it also sub…
simonleandergrimm Oct 25, 2024
4ec6788
Switched the --paired flag to instead be --read_type, and have it be …
simonleandergrimm Oct 25, 2024
f2bb836
Merge branch 'dev' into single-read-raw
simonleandergrimm Oct 25, 2024
e13acc6
Deleted a directory with testing scripts that was superseded by https…
simonleandergrimm Oct 25, 2024
9c62aa4
this script is now in https://github.com/naobservatory/simon-analysis…
simonleandergrimm Oct 25, 2024
be46ee9
Adding normal test dataset back in.
simonleandergrimm Oct 26, 2024
17d61ff
removing new versions of generate_samplesheet.sh (will add two differ…
simonleandergrimm Oct 26, 2024
0ba23fb
Reinstating dev version of run.nf, and creating new version of run.nf…
simonleandergrimm Oct 26, 2024
118378c
Adding run_dev_se to main.nf, a run specifically used for checking if…
simonleandergrimm Oct 26, 2024
8cd5239
Fixing default value for --read_type in summarize-multiqc-single.R. A…
simonleandergrimm Oct 26, 2024
7d3e725
Dropping commented out sections in split_sample
simonleandergrimm Oct 26, 2024
c107e91
Pulling in newest version of generate_samplesheet.sh
simonleandergrimm Oct 26, 2024
2d07ae6
Fixing single vs paired end read logic in hv_screen
simonleandergrimm Oct 26, 2024
74cb53a
Turned generate_samplesheet.sh back into dev version. Will and single…
simonleandergrimm Oct 28, 2024
3b0a11c
Adding read_type information to run.nf so the correct processes are p…
simonleandergrimm Oct 28, 2024
8f6beda
Extended generate_samplesheet.sh so it also takes in single-read data.
Nov 12, 2024
69f404c
Merge branch 'master' into single-read-raw-clean
simonleandergrimm Nov 19, 2024
654dd1c
Amended subworkflows to take in single end data.
simonleandergrimm Nov 19, 2024
2a01243
Merge branch 'master' into single-read-raw-clean
simonleandergrimm Nov 19, 2024
e9f7384
Reworked summarize_multiqc_pair.R to take in single_end data.
simonleandergrimm Nov 19, 2024
793a061
Made run_dev_se.nf follow updates to run.nf, and fixed single_end det…
simonleandergrimm Nov 19, 2024
fdf81af
Dropped two versions of FASTP, created conditional statement instead.
simonleandergrimm Nov 19, 2024
95dcf91
Dropped two different versions of the truncate_concat and added condi…
simonleandergrimm Nov 19, 2024
ada8c5e
dropped conditional selsection of processes.
simonleandergrimm Nov 19, 2024
a448dc9
Fixed single_end variable passing
simonleandergrimm Nov 19, 2024
e9b89be
Added new single read flagging in run.nf
simonleandergrimm Nov 19, 2024
eb82a32
removed old summarize-multiqc file
simonleandergrimm Nov 19, 2024
00ddcfc
fixed index in nextflow.config for paired end data.
simonleandergrimm Nov 19, 2024
e5b5ec5
added grouping and ndew index info to test-single-read config
simonleandergrimm Nov 19, 2024
8e201e7
Adding improved configs
simonleandergrimm Nov 23, 2024
591138d
dropped single end definition in run file.
simonleandergrimm Nov 23, 2024
27244bd
Adding params to single end variable invocation
simonleandergrimm Nov 23, 2024
517961f
removed whitespace
simonleandergrimm Nov 23, 2024
c28749f
updating nextflow.config of test
simonleandergrimm Nov 23, 2024
e132ec4
fixed single_end config in normal run workflow
simonleandergrimm Nov 23, 2024
51b9cf3
make single-end variable logical.
simonleandergrimm Nov 23, 2024
12c3fdd
Reverted to old gitignore structure.
simonleandergrimm Nov 23, 2024
4fd3ce6
Changed test dirs to only have one dir for run_dev_se.
simonleandergrimm Nov 24, 2024
d460813
Adding WIP progress
simonleandergrimm Nov 24, 2024
f412b07
Merge branch 'dev' into single-read-raw-clean
simonleandergrimm Nov 24, 2024
3d10bb0
Fixing single_end being unbound.
simonleandergrimm Nov 24, 2024
7899979
Merge branch 'dev' into single-read-raw-clean
simonleandergrimm Nov 29, 2024
dd942fa
Took into account new testing setup
simonleandergrimm Nov 29, 2024
50c2edc
adding single end info to config
simonleandergrimm Nov 29, 2024
61ea369
Moved single end eval from config to run files
simonleandergrimm Nov 29, 2024
ad640c6
Update nextflow.config
simonleandergrimm Dec 3, 2024
e85dd45
Merge remote-tracking branch 'origin/harmon_fix_gh_actions_test' into…
simonleandergrimm Dec 3, 2024
3a6f6b5
Merge remote-tracking branch 'origin/harmon_fix_gh_actions_test' into…
simonleandergrimm Dec 4, 2024
a0f5f32
Put single_end into profiles.config
simonleandergrimm Dec 4, 2024
d14da14
fixed run-dev-se config in tests
simonleandergrimm Dec 4, 2024
3fe2bd2
Creating a new config for read_type flag.
simonleandergrimm Dec 4, 2024
d0375ab
added run dev se to end-to-end yml
simonleandergrimm Dec 4, 2024
f5cf80a
Made rundevse index and outputs look the same as run.nf
simonleandergrimm Dec 4, 2024
3dc323e
Fixing setup of run_dev_se test config.
simonleandergrimm Dec 5, 2024
1904931
Update .gitignore (dropped new line)
simonleandergrimm Dec 5, 2024
e24d79e
Setting profiles.config back to original
simonleandergrimm Dec 5, 2024
b38b93d
Updated comments in main.nf to represent the posiblity of not not ala…
simonleandergrimm Dec 9, 2024
21b15b8
Fixed duplicate par statement in fastp.
simonleandergrimm Dec 9, 2024
9d717b7
Responding to Harmon's comments.
simonleandergrimm Dec 9, 2024
ee7baf4
dropped unncessary single-end variable.
simonleandergrimm Dec 9, 2024
034914b
fixed faulty paired-end fastp
simonleandergrimm Dec 10, 2024
c5454b9
added end-to-end-se.yml
simonleandergrimm Dec 10, 2024
4b966d8
adedd subworkflow to create samplesheet
simonleandergrimm Dec 11, 2024
7a3a59b
split truncate concat into two processes/
simonleandergrimm Dec 11, 2024
6ad3ce2
removed run dev se from end to end yml.
simonleandergrimm Dec 11, 2024
c096c48
fixed samplesheet typo.
simonleandergrimm Dec 16, 2024
10dbc48
Put additional things into loadsamplesheet.
simonleandergrimm Dec 16, 2024
4384fc4
added params. info
simonleandergrimm Dec 16, 2024
92c9312
Added new logic for handling start_time_str variable.
simonleandergrimm Dec 18, 2024
2c356f8
Update .gitignore
simonleandergrimm Dec 18, 2024
69386c7
Update end-to-end.yml
simonleandergrimm Dec 18, 2024
a96cc36
Update .gitignore
simonleandergrimm Dec 18, 2024
73d1e70
Updated index
simonleandergrimm Dec 18, 2024
e2af24d
Merge branch 'single-read-raw-clean' of https://github.com/naobservat…
simonleandergrimm Dec 18, 2024
65a7b76
Edited CHANGELOG.md to take into account changes made.
simonleandergrimm Dec 18, 2024
be30318
Amended CHANGELOG.md with changes suggested by Will
simonleandergrimm Dec 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Updated gen samplesheet scripts to pull in data from s3://nao-mgs-sim…
…on/*/raw
simonleandergrimm committed Oct 23, 2024
commit c8fd3ace0022d0090da37cce4b9f0d7785c26ae5
10 changes: 5 additions & 5 deletions test-paired-end/generate_samplesheet.sh
Original file line number Diff line number Diff line change
@@ -4,10 +4,10 @@
##### Input parameters #####

# Initialize variables
dir_path=""
forward_suffix=""
reverse_suffix=""
s3=""
dir_path="s3://nao-mgs-simon/test_paired_end/raw/"
forward_suffix="_1"
reverse_suffix="_2"
s3="1"

# Parse command-line arguments
while [[ $# -gt 0 ]]; do
@@ -51,7 +51,7 @@ echo "s3: $s3"

#### EXAMPLES ####

# dir_path="" # Cannot share this as it's restricted, but imagine the read looks like this
# dir_path="" # Cannot share this as it's restricted, but imagine the read looks like this
# forward_suffix="_S[0-9]_L[0-9][0-9][0-9]_R1_001"
# reverse_suffix="_S[0-9]_L[0-9][0-9][0-9]_R2_001"
# s3=1
4 changes: 2 additions & 2 deletions test-single-read/generate_samplesheet.sh
Original file line number Diff line number Diff line change
@@ -4,10 +4,10 @@
##### Input parameters #####

# Initialize variables
dir_path="raw/"
dir_path="s3://nao-mgs-simon/test_single_read/raw/"
forward_suffix=""
reverse_suffix=""
s3=0
s3=1
single_end=1

# Parse command-line arguments