Merge pull request #83 from uab-cgds-worthey/joss_manuscript

Bring master branch up to date on doc fixes

docs/Changelog.md

@@ -12,6 +12,11 @@ YYYY-MM-DD  John Doe
 ```
 ---
 
+2023-07-16  Manavalan Gajapathy
+
+* Updates doc based on users feedback. 
+
+
 2023-06-30  Manavalan Gajapathy
 
 * Merges `joss_manuscript` to the `master` branch to bring it up to date. 

docs/faq.md

@@ -0,0 +1,12 @@
+# FAQ
+
+## QuaC workflow failed due to `Error: Directory cannot be locked`
+
+See [snakemake docs
+here](https://snakemake.readthedocs.io/en/stable/project_info/faq.html#how-does-snakemake-lock-the-working-directory) on
+why snakemake locks the working directory. `Error: Directory cannot be locked` might happen when the parent snakemake
+process gets killed unexpectedly before completion. It is recommended to investigate why it got killed before proceeding
+to the next step. If you want to remove the lock (ie. unlock it), add `-e='--unlock'` to your original run
+`src/run_quac.py` command. Once that has completed you can run the original command again and the pipeline will pick up
+from it's last state.
+

docs/input_output.md

@@ -101,8 +101,8 @@ as input to QuaC.
 ## Output
 
 QuaC results are stored at the path specified via option `--outdir` (default:
-`data/quac/results/test_project/analysis`).  Refer to the [system testing's output](./system_testing.md) to learn more
-about the output directory structure. 
+`data/quac/results/test_project/analysis`).  Refer to the [system testing's
+output](./system_testing.md#expected-output-files) to learn more about the output directory structure. 
 
 !!! tip 
 

docs/quac_cli.md

@@ -64,38 +64,6 @@ QuaC wrapper options:
                         wrapper's) will be stored (default: data/quac/logs)
 ```
 
-### Useful features
-
-Besides the basic features, wrapper script [`src/run_quac.py`](../src/run_quac.py) offers the following:
-
-- Pass custom snakemake args using option `--extra_args`.
-- Dry-run snakemake using flag `--dryrun`. Note that this is same as `--extra_args='-n'`.
-- Submit snakemake process to Slurm, instead of running it locally, using `--cli_cluster_config`. 
-- Submit jobs triggered by snakemake workflow to Slurm using `--snakemake_cluster_config`.
-
-## Minimal example
-
-Minimal example to run the wrapper script, which in turn will execute the QuaC pipeline on-machine: (instead of using a
-SLURM job scheduler on an HPC system for running on a distributed system)
-
-```sh
-# First set up dependencies in the environment. 
-### Cheaha users can set them up as follows. 
-module reset
-module load Anaconda3/2020.02
-module load Singularity/3.5.2-GCC-5.4.0-2.26
-
-# activate conda env
-conda activate quac
-
-# run CLI/wrapper script
-python src/run_quac.py \
-      --project_name "PROJECT_DUCK" \
-      --projects_path "/path/to/the/projects" \
-      --pedigree "path/to/lake/with/ducks_pedigree_file.ped" \
-      --quac_watch_config "path/to/quac_watch_config.yaml"
-```
-
 ## Example usage
 
 Refer to commands used in [system testing](./system_testing.md) for example usage.

docs/quac_watch.md

@@ -11,10 +11,8 @@ and readily highlight samples that need further review.
 We provide pre-defined thresholds for QC metrics as part of the QuaC repo and they need to be supplied via
 `--quac_watch_config`:
 
-* For Genome sequencing -
-  [configs/quac_watch/wgs_quac_watch_config.yaml](../configs/quac_watch/wgs_quac_watch_config.yaml)
-* For Exome sequencing -
-  [configs/quac_watch/exome_quac_watch_config.yaml](../configs/quac_watch/exome_quac_watch_config.yaml)
+* For Genome sequencing - `configs/quac_watch/wgs_quac_watch_config.yaml`
+* For Exome sequencing - `configs/quac_watch/exome_quac_watch_config.yaml`
 
 These thresholds were curated based on
 

docs/system_testing.md

@@ -1,10 +1,9 @@
 # System testing
 
 The system testing implemented for this pipeline tests whether the pipeline runs from start to finish without any error.
-This testing uses test datasets present in [`.test/ngs-data/test_project`](../.test/ngs-data/test_project), which
-reflects a test project containing four samples -- Two samples without priorQC data (`no_priorQC`) and two with priorQC
-data (`include_priorQC`). [See .test/README.md](../.test/README.md) for more info on how these test datasets were
-created.
+This testing uses test datasets present in `.test/ngs-data/test_project`, which reflects a test project containing four
+samples -- Two samples without priorQC data (`no_priorQC`) and two with priorQC data (`include_priorQC`). See
+`.test/README.md` for more info on how these test datasets were created.
 
 !!! warning
 
@@ -115,7 +114,7 @@ data/quac/results/test_project_2samples_wgs-include_priorQC/
     │       │   └── ...
     │       ├── multiqc_final_pass
     │       │   ├── ...
-    │       │   └── A_multiqc.html
+    │       │   └── A_multiqc.html        <--- Sample-level multiqc output file
     │       ├── multiqc_initial_pass
     │       │   ├── ...
     │       │   └── A_multiqc.html
@@ -144,7 +143,7 @@ data/quac/results/test_project_2samples_wgs-include_priorQC/
         │   │   └── aggregated_rename_configs.tsv
         │   ├── multiqc_report_data
         │   │   └── ...
-        │   └── multiqc_report.html
+        │   └── multiqc_report.html        <--- Project-level multiqc output file
         └── somalier
             ├── ancestry
             │   └── ...

docs/visualize_pipeline.md

@@ -1,7 +1,7 @@
 # Visualization of pipeline
 
 [Visualization of the pipeline](https://snakemake.readthedocs.io/en/stable/executing/cli.html#visualization) based on
-the test datasets are available in [directory `./pipeline_visualized/`](../pipeline_visualized/). Commands used to
+the test datasets are available in directory `./pipeline_visualized/`. Commands used to
 create this visualization:
 
 ```sh

mkdocs.yaml

@@ -2,11 +2,12 @@ site_name: QuaC pipeline
 nav:
   - Home: index.md
   - Installation and Configuration: installation_configuration.md
-  - How to run QuaC: quac_cli.md
+  - Command line interface: quac_cli.md
   - Input/Output: input_output.md
   - QuaC-Watch: quac_watch.md
   - System testing: system_testing.md
   - Pipeline visualization: visualize_pipeline.md
+  - Frequently Asked Questions: faq.md
   - Changelog: Changelog.md
   - QC review:
       - Sample QC review system: sample_qc_review_system.md

src/run_quac.py

@@ -311,33 +311,33 @@ def create_dirpath(arg):
     )
 
     ############ Args for QuaC workflow  ############
-    WORKFLOW = PARSER.add_argument_group("QuaC workflow options")
+    WORKFLOW = PARSER.add_argument_group("QuaC snakemake workflow options")
 
     WORKFLOW.add_argument(
         "--project_name",
-        help="Project name",
-        metavar="",
+        help="Project name. Required.",
+        required=True,
     )
     WORKFLOW.add_argument(
         "--projects_path",
-        help="Path where all projects are hosted. Do not include project name here.",
+        help="Path where all projects are hosted. Do not include project name here. Required.",
         type=lambda x: is_valid_dir(PARSER, x),
-        metavar="",
+        required=True,
     )
     WORKFLOW.add_argument(
         "--pedigree",
-        help="Pedigree filepath. Must correspond to the project supplied via --project_name",
+        help="Pedigree filepath. Must correspond to the project supplied via --project_name. Required.",
         type=lambda x: is_valid_file(PARSER, x),
-        metavar="",
+        required=True,
     )
     WORKFLOW.add_argument(
         "--quac_watch_config",
         help=(
             "YAML config path specifying QC thresholds for QuaC-Watch."
-            " See directory 'configs/quac_watch/' in quac repo for the included config files."
+            " See directory 'configs/quac_watch/' in quac repo for the included config files. Required."
         ),
         type=lambda x: is_valid_file(PARSER, x),
-        metavar="",
+        required=True,
     )
     WORKFLOW.add_argument(
         "--workflow_config",