Add HTML report figures back into workflow (#33)

* Work on HTML reports.

* Keep working.

* Keep going.

* Remove unused import.

* Whoops!

* Fix.

* Things need to be nodes.

* Update aroma.py

* It runs but doesn't denoise.

* Remove melodic report (duplicate of AROMA report).

* Update base.py

* Start adding in carpet plot.

* Update reportlets.py

* Update base.py

* Update base.py

* Bit more work.

* Disable output spaces.

* Update parser.py

.gitignore

@@ -1,3 +1,4 @@
+.vscode
 .DS_Store
 .*.swp
 .mypy_cache

docs/faq.rst

@@ -12,3 +12,10 @@ What is fMRIPost?
 -----------------
 
 fMRIPost workflows are BIDS Apps that ingest fMRI preprocessing derivative datasets.
+They fall under the broader umbrella of NiPost workflows,
+which are pipelines that perform post-processing on BIDS-Derivative datasets.
+
+These workflows are primarily tested against the outputs of the `fMRIPrep`_ pipeline,
+but we plan to ensure that they can work with derivatives from any pipeline that produces BIDS-Derivative compliant datasets.
+
+.. _fMRIPrep: https://fmriprep.org

docs/outputs.rst

@@ -5,28 +5,32 @@
 ---------------------------
 Outputs of *fMRIPost-AROMA*
 ---------------------------
+
 *fMRIPost-AROMA* outputs conform to the :abbr:`BIDS (brain imaging data structure)`
 Derivatives specification (see `BIDS Derivatives`_, along with the
 upcoming `BEP 011`_ and `BEP 012`_).
 *fMRIPost-AROMA* generates three broad classes of outcomes:
 
-1. **Visual QA (quality assessment) reports**:
-   one :abbr:`HTML (hypertext markup language)` per subject,
-   that allows the user a thorough visual assessment of the quality
-   of processing and ensures the transparency of *fMRIPost-AROMA* operation.
+1.  **Visual QA (quality assessment) reports**:
+    One :abbr:`HTML (hypertext markup language)` per subject,
+    that allows the user a thorough visual assessment of the quality
+    of processing and ensures the transparency of *fMRIPost-AROMA* operation.
 
-2. **ICA outputs**:
-   Outputs from the independent component analysis (ICA).
-   For example, the mixing matrix and component weight maps.
+2.  **ICA outputs**:
+    Outputs from the independent component analysis (ICA).
+    For example, the mixing matrix and component weight maps.
 
-3. **Derivatives (denoised data)** denoised fMRI data in the requested output spaces and resolutions.
+3.  **Derivatives (denoised data)**:
+    Denoised fMRI data in the requested output spaces and resolutions.
 
-4. **Confounds**:
-   Time series of ICA components, as well as labels indicating if the components are accepted or rejected.
+4.  **Confounds**:
+    Time series of ICA components,
+    as well as labels indicating if the components are accepted or rejected.
 
 
 Layout
 ------
+
 Assuming fMRIPost-AROMA is invoked with::
 
     fmripost_aroma <input_dir>/ <output_dir>/ participant [OPTIONS]
@@ -47,60 +51,69 @@ The log directory contains `citation boilerplate`_ text.
 ``dataset_description.json`` is a metadata file in which fMRIPost-AROMA
 records metadata recommended by the BIDS standard.
 
+
 Visual Reports
 --------------
-*fMRIPost-AROMA* outputs summary reports, written to ``<output dir>/fmripost_aroma/sub-<subject_label>.html``.
+
+*fMRIPost-AROMA* outputs summary reports,
+written to ``<output dir>/fmripost_aroma/sub-<label>.html``.
 These reports provide a quick way to make visual inspection of the results easy.
 
+
 Derivatives of *fMRIPost-AROMA* (denoised data)
 -----------------------------------------------
+
 Derivative data are written to
-``<output dir>/sub-<subject_label>/``.
+``<output dir>/sub-<label>/``.
 The `BIDS Derivatives`_ specification describes the naming and metadata conventions we follow.
 
+
 ICA derivatives
 ~~~~~~~~~~~~~~~
 
 ICA outputs are stored in the ``func/`` subfolder::
 
-  sub-<subject_label>/
+  sub-<label>/
     func/
-      sub-<subject_label>_space-<space_label>_desc-melodic_mixing.tsv
-      sub-<subject_label>_space-<space_label>_desc-melodic_mixing.json
-      sub-<subject_label>_space-<space_label>_desc-melodic_components.nii.gz
-      sub-<subject_label>_space-<space_label>_desc-melodic_components.json
+      sub-<label>_space-<label>_desc-melodic_mixing.tsv
+      sub-<label>_space-<label>_desc-melodic_mixing.json
+      sub-<label>_space-<label>_desc-melodic_components.nii.gz
+      sub-<label>_space-<label>_desc-melodic_components.json
+
 
 Functional derivatives
 ~~~~~~~~~~~~~~~~~~~~~~
+
 Functional derivatives are stored in the ``func/`` subfolder.
 All derivatives contain ``task-<task_label>`` (mandatory) and ``run-<run_index>`` (optional), and
 these will be indicated with ``[specifiers]``::
 
-  sub-<subject_label>/
+  sub-<label>/
     func/
-      sub-<subject_label>_[specifiers]_space-<space_label>_desc-brain_mask.nii.gz
-      sub-<subject_label>_[specifiers]_space-<space_label>_desc-aggrDenoised_bold.nii.gz
-      sub-<subject_label>_[specifiers]_space-<space_label>_desc-nonaggrDenoised_bold.nii.gz
-      sub-<subject_label>_[specifiers]_space-<space_label>_desc-orthaggrDenoised_bold.nii.gz
+      sub-<label>_[specifiers]_space-<label>_desc-brain_mask.nii.gz
+      sub-<label>_[specifiers]_space-<label>_desc-aggrDenoised_bold.nii.gz
+      sub-<label>_[specifiers]_space-<label>_desc-nonaggrDenoised_bold.nii.gz
+      sub-<label>_[specifiers]_space-<label>_desc-orthaggrDenoised_bold.nii.gz
 
 **Regularly gridded outputs (images)**.
-Volumetric output spaces labels (``<space_label>`` above, and in the following) include
-``T1w`` and ``MNI152NLin6Asym`` (default).
+Volumetric output spaces labels (``<label>`` above, and in the following) include
+``MNI152NLin6Asym`` (default).
 
 **Extracted confounding time series**.
 For each :abbr:`BOLD (blood-oxygen level dependent)` run processed with *fMRIPost-AROMA*,
 an accompanying *confounds* file will be generated.
 Confounds_ are saved as a :abbr:`TSV (tab-separated value)` file::
 
-  sub-<subject_label>/
+  sub-<label>/
     func/
-      sub-<subject_label>_[specifiers]_desc-aroma_metrics.tsv
-      sub-<subject_label>_[specifiers]_desc-aroma_metrics.json
-      sub-<subject_label>_[specifiers]_desc-confounds_timeseries.tsv
-      sub-<subject_label>_[specifiers]_desc-confounds_timeseries.json
+      sub-<label>_[specifiers]_desc-aroma_metrics.tsv
+      sub-<label>_[specifiers]_desc-aroma_metrics.json
+      sub-<label>_[specifiers]_desc-confounds_timeseries.tsv
+      sub-<label>_[specifiers]_desc-confounds_timeseries.json
 
 Confounds
 ---------
+
 *fMRIPost-AROMA* outputs a set of confounds that can be used to denoise the data.
 These are stored in a TSV file (``desc-confounds_timeseries.tsv``) and a JSON file
 (``desc-confounds_timeseries.json``) that contains metadata about the confounds.
@@ -110,6 +123,7 @@ which may be labeled as "accepted" or "rejected" based on the ICA-AROMA classifi
 
 Confounds and "carpet"-plot on the visual reports
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
 The visual reports provide several sections per task and run to aid designing
 a denoising strategy for subsequent analysis.
 Some of the estimated confounds are plotted with a "carpet" visualization of the

docs/usage.rst

@@ -20,7 +20,7 @@ Structure)` format, and it must include at least one T1w structural image and
 We highly recommend that you validate your dataset with the free, online
 `BIDS Validator <https://bids-standard.github.io/bids-validator/>`_.
 
-The exact command to run *fMRIPRep* depends on the Installation_ method.
+The exact command to run *fMRIPost-AROMA* depends on the Installation_ method.
 The common parts of the command follow the `BIDS-Apps
 <https://github.com/BIDS-Apps>`_ definition.
 Example: ::
@@ -39,92 +39,18 @@ Command-Line Arguments
    :nodefault:
    :nodefaultconst:
 
-
-.. _fs_license:
-
-The FreeSurfer license
-----------------------
-*fMRIPRep* uses FreeSurfer tools, which require a license to run.
-
-To obtain a FreeSurfer license, simply register for free at
-https://surfer.nmr.mgh.harvard.edu/registration.html.
-
-When using manually-prepared environments or singularity, FreeSurfer will search
-for a license key file first using the ``$FS_LICENSE`` environment variable and then
-in the default path to the license key file (``$FREESURFER_HOME/license.txt``).
-If using the ``--cleanenv`` flag and ``$FS_LICENSE`` is set, use ``--fs-license-file $FS_LICENSE``
-to pass the license file location to *fMRIPRep*.
-
-It is possible to run the docker container pointing the image to a local path
-where a valid license file is stored.
-For example, if the license is stored in the ``$HOME/.licenses/freesurfer/license.txt``
-file on the host system: ::
-
-    $ docker run -ti --rm \
-        -v $HOME/fullds005:/data:ro \
-        -v $HOME/dockerout:/out \
-        -v $HOME/.licenses/freesurfer/license.txt:/opt/freesurfer/license.txt \
-        nipreps/fmripost_aroma:latest \
-        /data /out/out \
-        participant \
-        --ignore fieldmaps
-
 .. _prev_derivs:
 
 Reusing precomputed derivatives
 -------------------------------
 
 Reusing a previous, partial execution of *fMRIPost-AROMA*
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 *fMRIPost-AROMA* will pick up where it left off a previous execution, so long as the work directory
 points to the same location, and this directory has not been changed/manipulated.
 Some workflow nodes will rerun unconditionally, so there will always be some amount of
 reprocessing.
 
-Using a previous run of *FreeSurfer*
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-*fMRIPost-AROMA* will automatically reuse previous runs of *FreeSurfer* if a subject directory
-named ``freesurfer/`` is found in the output directory (``<output_dir>/freesurfer``).
-Reconstructions for each participant will be checked for completeness, and any missing
-components will be recomputed.
-You can use the ``--fs-subjects-dir`` flag to specify a different location to save
-FreeSurfer outputs.
-If precomputed results are found, they will be reused.
-
-BIDS Derivatives reuse
-~~~~~~~~~~~~~~~~~~~~~~
-As of version 23.2.0, *fMRIPost-AROMA* can reuse precomputed derivatives that follow BIDS Derivatives
-conventions. To provide derivatives to *fMRIPost-AROMA*, use the ``--derivatives`` (``-d``) flag one
-or more times.
-
-This mechanism replaces the earlier, more limited ``--anat-derivatives`` flag.
-
-.. note::
-   Derivatives reuse is considered *experimental*.
-
-This feature has several intended use-cases:
-
-  * To enable fMRIPost-AROMA to be run in a "minimal" mode, where only the most essential
-    derivatives are generated. This can be useful for large datasets where disk space
-    is a concern, or for users who only need a subset of the derivatives. Further
-    derivatives may be generated later, or by a different tool.
-  * To enable fMRIPost-AROMA to be integrated into a larger processing pipeline, where
-    other tools may generate derivatives that fMRIPost-AROMA can use in place of its own
-    steps.
-  * To enable users to substitute their own custom derivatives for those generated
-    by fMRIPost-AROMA. For example, a user may wish to use a different brain extraction
-    tool, or a different registration tool, and then use fMRIPost-AROMA to generate the
-    remaining derivatives.
-  * To enable complicated meta-workflows, where fMRIPost-AROMA is run multiple times
-    with different options, and the results are combined. For example, the
-    `My Connectome <https://openneuro.org/datasets/ds000031/>`__ dataset contains
-    107 sessions for a single-subject. Processing of all sessions simultaneously
-    would be impractical, but the anatomical processing can be done once, and
-    then the functional processing can be done separately for each session.
-
-See also the ``--level`` flag, which can be used to control which derivatives are
-generated.
-
 Troubleshooting
 ---------------
 Logs and crashfiles are output into the
@@ -139,14 +65,14 @@ The documentation of this project is found here: https://fmripost_aroma.org/en/l
 All bugs, concerns and enhancement requests for this software can be submitted here:
 https://github.com/nipreps/fmripost_aroma/issues.
 
-If you have a problem or would like to ask a question about how to use *fMRIPRep*,
+If you have a problem or would like to ask a question about how to use *fMRIPost-AROMA*,
 please submit a question to `NeuroStars.org <https://neurostars.org/tag/fmripost_aroma>`_ with an ``fmripost_aroma`` tag.
 NeuroStars.org is a platform similar to StackOverflow but dedicated to neuroinformatics.
 
-Previous questions about *fMRIPRep* are available here:
+Previous questions about *fMRIPost-AROMA* are available here:
 https://neurostars.org/tag/fmripost_aroma/
 
-To participate in the *fMRIPRep* development-related discussions please use the
+To participate in the *fMRIPost-AROMA* development-related discussions please use the
 following mailing list: https://mail.python.org/mailman/listinfo/neuroimaging
 Please add *[fmripost_aroma]* to the subject line when posting on the mailing list.
 

docs/workflows.rst

@@ -3,21 +3,21 @@
 ===========================
 Processing pipeline details
 ===========================
+
 *fMRIPost-AROMA* adapts its pipeline depending on what data and metadata are
 available and are used as the input.
 For example, slice timing correction will be
 performed only if the ``SliceTiming`` metadata field is found for the input
 dataset.
 
-A (very) high-level view of the simplest pipeline (for a single-band dataset with only
-one task, single-run, with no slice-timing information nor fieldmap acquisitions)
-is presented below:
+A (very) high-level view of the simplest pipeline is presented below:
 
 .. workflow::
     :graph2use: orig
     :simple_form: yes
 
     from fmripost_aroma.workflows.tests import mock_config
     from fmripost_aroma.workflows.base import init_single_subject_wf
+
     with mock_config():
         wf = init_single_subject_wf('01')

src/fmripost_aroma/cli/parser.py

@@ -37,11 +37,12 @@ def _build_parser(**kwargs):
     from functools import partial
     from pathlib import Path
 
-    from niworkflows.utils.spaces import OutputReferencesAction
     from packaging.version import Version
 
     from fmripost_aroma.cli.version import check_latest, is_flagged
 
+    # from niworkflows.utils.spaces import OutputReferencesAction
+
     class ToDict(Action):
         def __call__(self, parser, namespace, values, option_string=None):
             d = {}
@@ -286,27 +287,28 @@ def _bids_filter(value, parser):
         action='store',
         nargs='+',
         default=[],
-        choices=['fieldmaps', 'slicetiming'],
+        choices=['fieldmaps', 'slicetiming', 'jacobian'],
         help=(
             'Ignore selected aspects of the input dataset to disable corresponding '
             'parts of the resampling workflow (a space delimited list)'
         ),
     )
-    g_conf.add_argument(
-        '--output-spaces',
-        nargs='*',
-        action=OutputReferencesAction,
-        help="""\
-Standard and non-standard spaces to resample denoised functional images to. \
-Standard spaces may be specified by the form \
-``<SPACE>[:cohort-<label>][:res-<resolution>][...]``, where ``<SPACE>`` is \
-a keyword designating a spatial reference, and may be followed by optional, \
-colon-separated parameters. \
-Non-standard spaces imply specific orientations and sampling grids. \
-For further details, please check out \
-https://fmriprep.readthedocs.io/en/%s/spaces.html"""
-        % (currentv.base_version if is_release else 'latest'),
-    )
+    # Disable output spaces until warping works
+    # g_conf.add_argument(
+    #     '--output-spaces',
+    #     nargs='*',
+    #     action=OutputReferencesAction,
+    #     help="""\
+    # Standard and non-standard spaces to resample denoised functional images to. \
+    # Standard spaces may be specified by the form \
+    # ``<SPACE>[:cohort-<label>][:res-<resolution>][...]``, where ``<SPACE>`` is \
+    # a keyword designating a spatial reference, and may be followed by optional, \
+    # colon-separated parameters. \
+    # Non-standard spaces imply specific orientations and sampling grids. \
+    # For further details, please check out \
+    # https://fmriprep.readthedocs.io/en/%s/spaces.html"""
+    #    % (currentv.base_version if is_release else 'latest'),
+    # )
     g_conf.add_argument(
         '--dummy-scans',
         required=False,