Pipeline Scripts
Table of Contents
This section is designed to outline the key procedures of each of the pipeline scripts.
This script features the main preprocessing and cleaning of the data before the ICA process begins. Data is flagged for high standard deviation by channel to remove comically bad channels. The data is then re-referenced to the average site, and high/low pass filtered. Channel neighbour correlations are then calculated to find odd channels and bridged channels. The most redundant channel is also flagged to reduce the rank of the data. The data is average re-referenced without any flagged channels and channel correlations are calculated by time and flagged. The file is then saved containing the marks structure, then saved again with all of the mentioned flags purged from the data in order to prime it for ICA.
| Variable | Default in Config File | Description |
|---|---|---|
| [in_path] | [] | Relative path to input data files assuming cd = work_path |
| [montage_info] | Montage information to fit channel locations on a standard surface. This can be a transformations matrix (e.g. [-.14,-22,-48,-.07,0,-1.57,1080,1260,1240]), or a file containing a transformation matrix (e.g. [batch_dfn]_transmat.mat), or a BIDS channel location tsv file (assuming that during BIDS init the channels were coregistered to the standard surface). | |
| [staging_script] | derivatives/lossless/code/scripts/staging_script.m | Script file path/name that creates marks based on events in the raw data file. |
| [recur_sec] | 1 | Recurrence (sec) for artefact detection epoching |
| [limit_sec] | [0 1] | Limits (sec) for artifact detection epoching |
| [sd_t_meth] | q | Method used for identifying outlying epochs (e.g. 'q' (quantiles), 'z' (z-score), or 'na') |
| [sd_t_vals] | [.3 .7] | Percentage trim for confidence intervals during epoch std/quantile criteria for identifying outlying epochs (e.g. [.3 .7]) |
| [sd_t_o] | 6 | threshold for number of standard deviations or quantiles from the mean for identifying outlying epochs |
| [sd_t_f_meth] | fixed | Method used for flagging epochs. Can be 'fixed', or a distribution of values ('z-score' or 'q') |
| [sd_t_f_vals] | Percentage trim for confidence intervals during std/quantile criteria for flagging outlying epochs (e.g. [.3 .7], leave empty for 'fixed') | |
| [sd_t_f_o] | 0.2 | Threshold for flagging epochs (percentage for how many flagged channels are required for rejection of an epoch) |
| [sd_t_pad] | 1 | Number of windows to pad onto each side of the ch_sd time flag |
| [sd_ch_meth] | q | Method used for identifying outlying channels (e.g. 'q' (quantiles), 'z' (z-score), or 'na') |
| [sd_ch_vals] | [.3 .7] | Percentage trim for confidence intervals during channel std/quantile criteria for identifying outlying epochs (e.g. [.3 .7]) |
| [sd_ch_o] | 6 | threshold for number of standard deviations or quantiles from the mean for identifying outlying channels |
| [sd_ch_f_meth] | fixed | Method used for flagging channels. Can be 'fixed', or a distribution of values ('z-score' or 'q') |
| [sd_ch_f_vals] | Percentage trim for confidence intervals during std/quantile criteria for flagging outlying channels (e.g. [.3 .7], leave empty for 'fixed') | |
| [sd_ch_f_o] | 0.2 | Threshold for flagging channels (percentage for how many flagged epochs are required for rejection of a channel) |
| [ref_loc_file] | derivatives/lossless/code/misc/standard_1020_bucanl19.elc | Name of file containing the reference locations (including the relative path) |
| [low_bound_hz] | 1 | Lower bound of the filter bass-band |
| [high_bound_hz] | Upper bound of the filter bass-band | |
| [save_f_res] | 1 | 1 if you want to save the filter residuals |
| [n_nbr_ch] | 3 | Number of channels to use in nearest neighbour r calculation (for channel criteria) |
| [r_ch_meth] | q | Method used for flagging low r channels (e.g. 'q' (quantiles), or 'na') |
| [r_ch_vals] | [.3 .7] | Percentage trim for confidence intervals during low r channel standard deviation criteria (e.g. [.3 .7]) |
| [r_ch_o] | 6 | z threshold for flagging channel during neighbour r criteria (e.g. 3) |
| [r_ch_f_meth] | fixed | Fixed method used for flagging low r channels (should be 'fixed') |
| [r_ch_f_vals] | Percentage trim for confidence intervals during low r channel standard deviation criteria (e.g. [.3 .7], leave empty for 'fixed') | |
| [r_ch_f_o] | 0.2 | z threshold for flagging channel during fixed neighbour r criteria (e.g. 3) |
| [bridge_trim] | 40 | Percentage trim for z calculation of bridged channels (e.g. 40 = 20% top and bottom) |
| [bridge_z] | 6 | z threshold for flagging channel during bridging criteria |
| [n_nbr_t] | 3 | Number of channels to use in nearest neighbour r calculation (for epoch criteria) |
| [r_t_meth] | q | Method used for flagging low r epochs (e.g. 'q' (quantiles), or 'na') |
| [r_t_vals] | [.3 .7] | Percentage trim for confidence intervals during low r epoch standard deviation criteria (e.g. [.3 .7]) |
| [r_t_o] | 6 | z threshold for flagging epochs during neighbour r criteria (e.g. 3) |
| [r_t_f_meth] | fixed | Fixed method used for flagging low r epochs (should be 'fixed') |
| [r_t_f_vals] | Percentage trim for confidence intervals during low r epoch standard deviation criteria (e.g. [.3 .7], leave empty for 'fixed') | |
| [r_t_f_o] | 0.2 | z threshold for flagging epochs during fixed neighbour r criteria |
| [min_gap_ms] | 2000 | Minimum time (ms) to allow between periods marked for rejection |
| [out_path] | [] | Relative path to output data files assuming cd = work_path |
| [amica_param_file] | derivatives/lossless/code/misc/amica15_default.param | template amicadefs.param file to modify |
| [amica_threads_s02] | 8 | number of threads to use for running s02 Amica script |
The Amica script is run once initially, and then three more times in parallel after the component artefact rejection to ensure that the Amica procedure is replicable if run multiple times on the same dataset.
Versions
- amica_init (s02)
- amica_A,B,C (s04a, s04b, s04c)
| Variable | Default in Config File | Description |
|---|---|---|
| [dep_path] | derivatives/lossless/code/dependencies/eeglab_asr_amica/plugins/amica | Relative path to Amica executable program assuming cd = work_path |
| [in_path] | [] | Relative path to input data files assuming cd = work_path |
| [amica_ver] | amica15 | Name of the Amica executable program version |
Versions
- compart_data (s03)
| Variable | Default in Config | Description |
|---|---|---|
| [in_path] | derivatives/lossless | Path to input and output data files assuming cd = work_path |
| [recur_sec] | 1 | Recurrence (sec) for artifact detection epoching |
| [limit_sec] | [0 1] | Limits (sec) for artifact detection epoching |
| [sd_t_meth] | q | Method used for flagging epochs (e.g. 'q' (quantiles), or 'na') |
| [sd_t_vals] | [.3 .7] | Percentage trim for confidence intervals during epoch standard deviation criteria (e.g. [.3 .7]) |
| [sd_t_o] | 6 | z threshold for flagging epochs during standard deviation criteria (e.g. 6) |
| [sd_t_f_meth] | fixed | Fixed method used for flagging epochs (should be 'fixed') |
| [sd_t_f_vals] | Percentage trim for confidence intervals during epoch standard deviation criteria (e.g. [.3 .7], leave empty for 'fixed') | |
| [sd_t_f_o] | 0.2 | z threshold for flagging epochs during fixed standard deviation criteria (e.g. .2) |
| [out_path] | derivatives/lossless | Relative path to output data files assuming cd = work_path |
| [min_gap_ms] | 2000 | Minimum time (ms) to allow between periods marked for rejection |
| [amica_param_file] | derivatives/lossless/code/misc/amica_default.param | Template amicadefs.param file to modify |
| [amica_threads_s04] | 8 | number of threads to use for running s04a, s04b, s04c amica scripts |
Versions
- concat_data (s05)
| Variables | Default in Config | Description |
|---|---|---|
| [in_path] | derivatives/lossless | Relative path to input data files assuming cd = work_path |
| [recur_sec] | 1 | Recurrence (sec) for the current artifact detection epoching |
| [limit_sec] | [0 1] | Limits (sec) for the current artifact detection epoching |
| [sd_t_meth] | q | Method used for flagging epochs (e.g. 'q' (quantiles), or 'na') |
| [sd_t_vals] | [.3 .7] | Percentage trim for confidence intervals during epoch standard deviation criteria (e.g. [.3 .7]) |
| [sd_t_o] | 6 | z threshold for flagging epochs during standard deviation criteria (e.g. 6) |
| [sd_t_f_meth] | fixed | Fixed method used for flagging epochs (should be 'fixed') |
| [sd_t_f_vals] | Percentage trim for confidence intervals during epoch standard deviation criteria (e.g. [.3 .7], leave empty for 'fixed') | |
| [sd_t_f_o] | 0.2 | z threshold for flagging epochs during fixed standard deviation criteria (e.g. .2) |
| [min_gap_ms] | 2000 | Minimum time (ms) to allow between periods marked for rejection |
| [icr_crit] | 0.05 | False discovery rate of component replicablilty, measure of sensitivity % |
| [dip_vol] | derivatives/lossless/code/misc/standard_vol.mat | file path/name of the standard volume file used in dipole fitting |
| [dip_mri] | derivatives/lossless/code/misc/standard_mri.mat | file path/name of the standard MRI file used in dipole fitting |
| [dip_elc] | derivatives/lossless/code/misc/standard_1020.elc | file path/name of the standard electrode file used in dipole fitting |
| [dip_transmat] | [0 0 0 0 0 0 1 1 1] | Transformation matrix to apply during dipole fit |
| [cor_transmat] | [0,0,0,0,0,1.57,1,1,1] | The transformation matrix to apply between dipole fitting to the standard head and performing ICLabel to the EEGLAB default orientation |
| [cor_nosedir] | +X | Nose direction of montage to apply between dipole fitting to the standard head and performing ICLabel to the EEGLAB default orientation (e.g. +Y) |
| [out_path] | derivatives/lossless | Relative path to output data files assuming cd = work_path |