Resolving merge conflicts with bugfix/jacobian-perturbation-fix

config.yml

@@ -108,6 +108,7 @@ Met: "GEOSFP"
 
 ## Setup modules
 ##   Turn on/off different steps in setting up the inversion 
+RunSetup: true
 SetupTemplateRundir: true
 SetupSpinupRun: false
 SetupJacobianRuns: false
@@ -116,7 +117,8 @@ SetupPosteriorRun: false
 
 ## Run modules
 ##   Turn on/off different steps in performing the inversion
-RunSetup: true
+##   NOTE: DoPriorEmis must be true to run DoPriorEmis
+DoPriorEmis: true
 DoSpinup: false
 DoJacobian: false
 ReDoJacobian: false
@@ -129,16 +131,20 @@ DoPreview: true
 DOFSThreshold: 0
 
 ## Resource allocation settings for slurm jobs
-SimulationCPUs: 32
-SimulationMemory: "32gb"
-JacobianCPUs: 1
-JacobianMemory: 2000
+RequestedCPUs: 32
+RequestedMemory: "32gb"
 RequestedTime: "24:00:00"
 SchedulerPartition: "debug"
 
 ## Max number of simultaneous Jacobian runs from the job array (-1: no limit)
 MaxSimultaneousRuns: -1
 
+## Number of Jacobians tracers to use for each jacobian simulation
+##   Specifying a value = 1 will submit a separate jacobian simulation for each
+##   state vector element. Specifying a value > 1 will combine state vector 
+##   elements into a single jacobian simulation.
+NumJacobianTracers: 5
+
 ##====================================================================
 ##
 ## Advanced Settings (optional)
@@ -155,10 +161,11 @@ MaxSimultaneousRuns: -1
 ##--------------------------------------------------------------------
 
 ## Jacobian settings
-## Note PerturbValue and PerturbValueOH are relative scale factors and 
-## PerturbValueBCs is in ppb
-PerturbValue: 1.5
-PerturbValueOH: 1.5
+## Note PerturbValue is in 1e-8 kg/m2/s. eg specifying 10 means the 
+## perturbation will be 10e-8 kg/m2/s 
+## PerturbValueOH is a relative scale factor and PerturbValueBCs is in ppb
+PerturbValue: 10.0
+PerturbValueOH: 1.1
 PerturbValueBCs: 10.0
 
 ## Apply scale factors from a previous inversion?

docs/source/getting-started/imi-config-file.rst

@@ -23,8 +23,6 @@ General
      - S3 path to upload files to (eg. ``s3://imi-output-dir/example-output/``). Only used if ``S3Upload`` is ``true``.
    * - ``S3UploadFiles``
      - Files to upload from the IMI Output directory (eg. ``[*]`` will upload everything). Only used if ``S3Upload`` is ``true``.
-   * - ``PointSourceDataset``
-     - Files to upload from the IMI Output directory (eg. ``[*]`` will upload everything). Only used if ``S3Upload`` is ``true``.
 
 Period of interest
 ~~~~~~~~~~~~~~~~~~
@@ -100,7 +98,7 @@ State vector
      - Boolean to optimize boundary conditions during the inversion. Must also include ``PerturbValueBCs`` and ``PriorErrorBCs``. Default value is ``false``.
    * - ``OptimizeOH``
      - Boolean to optimize OH during the inversion. Must also include ``PerturbValueOH`` and ``PriorErrorOH``. Default value is ``false``.
-
+       
 Point source datasets
 ~~~~~~~~~~~~~~~~~~~~~
 .. list-table::
@@ -193,6 +191,8 @@ These settings turn on/off (``true`` / ``false``) different steps for setting up
    :widths: 30, 70
    :class: tight-table
 
+   * - ``RunSetup``
+     - Boolean to run the setup script (``setup_imi.sh``), including selected setup modules above.
    * - ``SetupTemplateRundir``
      - Boolean to create a GEOS-Chem run directory and modify it with settings from ``config.yml``.
    * - ``SetupSpinupRun``
@@ -212,8 +212,9 @@ These settings turn on/off (``true`` / ``false``) different steps for running th
    :widths: 30, 70
    :class: tight-table
 
-   * - ``RunSetup``
-     - Boolean to run the setup script (``setup_imi.sh``), including selected setup modules above.
+   * - ``DoPriorEmis``
+     - Boolean to run a HEMCO standalone simulation to generate the
+       prior emissions.
    * - ``DoSpinup``
      - Boolean to run the spin-up simulation.
    * - ``DoJacobian``
@@ -235,30 +236,31 @@ IMI preview
      - Threshold for estimated DOFS below which the IMI should automatically exit with a warning after performing the preview.
        Default value ``0`` prevents exit.
 
-SLURM Resource Allocation
+Job Resource Allocation
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 These settings are used to allocate resources (CPUs and Memory) to the different simulations needed to run the inversion.
-Note: some python scripts are also deployed using slurm and default to using the ``SimulationCPUs`` and ``SimulationMemory`` settings.
+Note: some python scripts are also deployed using slurm and default to using the ``RequestedCPUs`` and ``RequestedMemory`` settings.
 
 .. list-table::
    :widths: 30, 70
    :class: tight-table
 
+   * - ``RequestedCPUs``
+     - Number of cores to allocate to slurm jobs.
+   * - ``RequestedMemory``
+     - Amount of memory to allocate to each in series simulation (in MB).
    * - ``RequestedTime``
      - Max amount of time to allocate to each sbatch job (eg. "0-6:00")
-   * - ``SimulationCPUs``
-     - Number of cores to allocate to each in series simulation.
-   * - ``SimulationMemory``
-     - Amount of memory to allocate to each in series simulation (in MB).
-   * - ``JacobianCPUs``
-     - Number of cores to allocate to each jacobian simulation (run in parallel).
-   * - ``JacobianMemory``
-     - Amount of memory to allocate to each jacobian simulation (in MB).
    * - ``SchedulerPartition``
      - Name of the partition(s) you would like all slurm jobs to run on (eg. "debug,huce_cascade,seas_compute,etc").
    * - ``MaxSimultaneousRuns``
      - The maximum number of jacobian simulations to run simultaneously. The default is -1 (no limit) which will submit all jacobian simulations at once. If the value is greater than zero, the sbatch array statement will be modified to include the "%" separator and will limit the number of simultaneously running tasks from the job array to the specifed value.
-
+   * - ``NumJacobianTracers``
+     - The number of tracers to use for each jacobian simulation. A value of 1
+       will create and submit a jacobian run for each state vector element. 
+       Specifying a value greater than 1 will combine state vector elements 
+       into fewer runs. The default values is 5 tracers per simulation.
+
 Advanced settings: GEOS-Chem options
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 These settings are intended for advanced users who wish to modify additional GEOS-Chem options.
@@ -268,7 +270,7 @@ These settings are intended for advanced users who wish to modify additional GEO
    :class: tight-table
 
    * - ``PerturbValue``
-     - Value to perturb emissions by in each sensitivity simulation. Default value is ``1.5``.
+     - Target perturbation amount on the emissions in each sensitivity simulation. Default value is ``10.0``. Corresponding to a 10e-8 kg/m2/s perturbation.
    * - ``PerturbValueOH``
      - Value to perturb OH by if using ``OptimizeOH``. Default value is ``1.5``.
    * - ``PerturbValueBCs``

envs/Harvard-Cannon/config.harvard-cannon.global_inv.yml

@@ -62,7 +62,7 @@ BufferDeg: 0
 LandThreshold: 0.25
 OffshoreEmisThreshold: 0
 OptimizeBCs: false
-OptimizeOH: false
+OptimizeOH: true
 
 ## Point source datasets
 ## Used for visualizations and state vector clustering
@@ -72,7 +72,7 @@ PointSourceDatasets: ["SRON"]
 ReducedDimensionStateVector: false
 DynamicKFClustering: false
 ClusteringMethod: "kmeans"
-NumberOfElements: 45
+NumberOfElements: 1000
 ForcedNativeResolutionElements: 
   - [31.5, -104]
 EmissionRateFilter: 2500
@@ -84,10 +84,13 @@ StateVectorFile: "/path/to/StateVector.nc"
 ShapeFile: "None"
 
 ## Inversion
-## Note PriorError and PriorErrorOH are relative fractions (e.g. 0.5 = 50%)
-## if lognormal PriorError is the geometric standard deviation (e.g. 2 = factor of 2 uncertainty)
-## PriorErrorOH is a relative fraction (e.g. 0.1 = 10%)
-##  and PriorErrorBCs is in ppb (not used for global)
+## Notes:
+## - PriorError is a relative fraction (e.g. 0.5 = 50%)
+## - If lognormal PriorError is the geometric standard deviation (e.g. 2 = factor of 2 uncertainty)
+## - PriorErrorOH is a relative fraction (e.g. 0.1 = 10%)
+## - PriorErrorBCs is in ppb (not used for global)
+## - PriorErrorBufferElements is only used if LognormalErrors is true
+LognormalErrors: false
 PriorError: 0.5
 PriorErrorBCs: 10.0
 PriorErrorBufferElements: 0.5
@@ -106,6 +109,7 @@ Met: "GEOSFP"
 
 ## Setup modules
 ##   Turn on/off different steps in setting up the inversion 
+RunSetup: true
 SetupTemplateRundir: true
 SetupSpinupRun: false
 SetupJacobianRuns: false
@@ -114,10 +118,11 @@ SetupPosteriorRun: false
 
 ## Run modules
 ##   Turn on/off different steps in performing the inversion
-RunSetup: true
+##   NOTE: DoPriorEmis must be true to run DoPriorEmis
+DoPriorEmis: true
 DoSpinup: false
 DoJacobian: false
-ReDoJacobian: true
+ReDoJacobian: false
 DoInversion: false
 DoPosterior: false
 
@@ -127,15 +132,19 @@ DoPreview: true
 DOFSThreshold: 0
 
 ## Resource allocation settings for slurm jobs
-SimulationCPUs: 32
-SimulationMemory: "32gb"
-JacobianCPUs: 1
-JacobianMemory: 2000
+RequestedCPUs: 32
+RequestedMemory: "32gb"
 RequestedTime: "24:00:00"
-SchedulerPartition: "sapphire,huce_cascade,huce_intel,seas_compute,shared"
+SchedulerPartition: "sapphire,huce_cascade,seas_compute,shared"
 
 ## Max number of simultaneous Jacobian runs from the job array (-1: no limit)
-MaxSimultaneousRuns: 50
+MaxSimultaneousRuns: -1
+
+## Number of Jacobians tracers to use for each jacobian simulation
+##   Specifying a value = 1 will submit a separate jacobian simulation for each
+##   state vector element. Specifying a value > 1 will combine state vector 
+##   elements into a single jacobian simulation.
+NumJacobianTracers: 5
 
 ##====================================================================
 ##
@@ -153,10 +162,11 @@ MaxSimultaneousRuns: 50
 ##--------------------------------------------------------------------
 
 ## Jacobian settings
-## Note PerturbValue and PerturbValueOH are relative scale factors and 
-## PerturbValueBCs is in ppb
-PerturbValue: 1.5
-PerturbValueOH: 1.5
+## Note PerturbValue is in 1e-8 kg/m2/s. eg specifying 10 means the 
+## perturbation will be 10e-8 kg/m2/s 
+## PerturbValueOH is a relative scale factor and PerturbValueBCs is in ppb
+PerturbValue: 10.0
+PerturbValueOH: 1.1
 PerturbValueBCs: 10.0
 
 ## Apply scale factors from a previous inversion?

envs/Harvard-Cannon/config.harvard-cannon.yml

@@ -84,11 +84,12 @@ StateVectorFile: "/path/to/StateVector.nc"
 ShapeFile: "resources/shapefiles/PermianBasin_Extent_201712.shp"
 
 ## Inversion
-## Note PriorError is a relative fraction (e.g. 0.5 = 50%)
-## if lognormal PriorError is the geometric standard deviation (e.g. 2 = factor of 2 uncertainty)
-## PriorErrorOH is a relative fraction (e.g. 0.1 = 10%)
-## and PriorErrorBCs is in ppb
-## PriorErrorBufferElements is only used if LognormalErrors is true
+## Notes:
+## - PriorError is a relative fraction (e.g. 0.5 = 50%)
+## - If lognormal PriorError is the geometric standard deviation (e.g. 2 = factor of 2 uncertainty)
+## - PriorErrorOH is a relative fraction (e.g. 0.1 = 10%)
+## - PriorErrorBCs is in ppb
+## - PriorErrorBufferElements is only used if LognormalErrors is true
 LognormalErrors: false
 PriorError: 0.5
 PriorErrorBCs: 10.0
@@ -108,6 +109,7 @@ Met: "GEOSFP"
 
 ## Setup modules
 ##   Turn on/off different steps in setting up the inversion 
+RunSetup: true
 SetupTemplateRundir: true
 SetupSpinupRun: false
 SetupJacobianRuns: false
@@ -116,7 +118,8 @@ SetupPosteriorRun: false
 
 ## Run modules
 ##   Turn on/off different steps in performing the inversion
-RunSetup: true
+##   NOTE: DoPriorEmis must be true to run DoPriorEmis
+DoPriorEmis: true
 DoSpinup: false
 DoJacobian: false
 ReDoJacobian: false
@@ -129,16 +132,20 @@ DoPreview: true
 DOFSThreshold: 0
 
 ## Resource allocation settings for slurm jobs
-SimulationCPUs: 32
-SimulationMemory: "32gb"
-JacobianCPUs: 1
-JacobianMemory: 2000
+RequestedCPUs: 32
+RequestedMemory: "32gb"
 RequestedTime: "24:00:00"
-SchedulerPartition: "sapphire,huce_cascade,huce_intel,seas_compute,shared"
+SchedulerPartition: "sapphire,huce_cascade,seas_compute,shared"
 
 ## Max number of simultaneous Jacobian runs from the job array (-1: no limit)
 MaxSimultaneousRuns: -1
 
+## Number of Jacobians tracers to use for each jacobian simulation
+##   Specifying a value = 1 will submit a separate jacobian simulation for each
+##   state vector element. Specifying a value > 1 will combine state vector 
+##   elements into a single jacobian simulation.
+NumJacobianTracers: 5
+
 ##====================================================================
 ##
 ## Advanced Settings (optional)
@@ -155,10 +162,11 @@ MaxSimultaneousRuns: -1
 ##--------------------------------------------------------------------
 
 ## Jacobian settings
-## Note PerturbValue and PerturbValueOH are relative scale factors and 
-## PerturbValueBCs is in ppb
-PerturbValue: 1.5
-PerturbValueOH: 1.5
+## Note PerturbValue is in 1e-8 kg/m2/s. eg specifying 10 means the 
+## perturbation will be 10e-8 kg/m2/s 
+## PerturbValueOH is a relative scale factor and PerturbValueBCs is in ppb
+PerturbValue: 10.0
+PerturbValueOH: 1.1
 PerturbValueBCs: 10.0
 
 ## Apply scale factors from a previous inversion?

resources/containers/al2/container_config.yml

@@ -108,6 +108,7 @@ Met: "GEOSFP"
 
 ## Setup modules
 ##   Turn on/off different steps in setting up the inversion 
+RunSetup: true
 SetupTemplateRundir: true
 SetupSpinupRun: false
 SetupJacobianRuns: false
@@ -116,7 +117,8 @@ SetupPosteriorRun: false
 
 ## Run modules
 ##   Turn on/off different steps in performing the inversion
-RunSetup: true
+##   NOTE: DoPriorEmis must be true to run DoPriorEmis
+DoPriorEmis: true
 DoSpinup: false
 DoJacobian: false
 ReDoJacobian: true
@@ -129,16 +131,20 @@ DoPreview: true
 DOFSThreshold: 0
 
 ## Resource allocation settings for slurm jobs
-SimulationCPUs: 16
-SimulationMemory: "16gb"
-JacobianCPUs: 1
-JacobianMemory: 2000
+RequestedCPUs: 16
+RequestedMemory: "16gb"
 RequestedTime: "24:00:00"
 SchedulerPartition: "debug"
 
 ## Max number of simultaneous Jacobian runs from the job array (-1: no limit)
 MaxSimultaneousRuns: -1
 
+## Number of Jacobians tracers to use for each jacobian simulation
+##   Specifying a value = 1 will submit a separate jacobian simulation for each
+##   state vector element. Specifying a value > 1 will combine state vector 
+##   elements into a single jacobian simulation.
+NumJacobianTracers: 5
+
 ##====================================================================
 ##
 ## Advanced Settings (optional)
@@ -155,10 +161,11 @@ MaxSimultaneousRuns: -1
 ##--------------------------------------------------------------------
 
 ## Jacobian settings
-## Note PerturbValue and PerturbValueOH are relative scale factors and 
-## PerturbValueBCs is in ppb
-PerturbValue: 1.5
-PerturbValueOH: 1.5
+## Note PerturbValue is in 1e-8 kg/m2/s. eg specifying 10.0 means the 
+## perturbation will be 10e-8 kg/m2/s 
+## PerturbValueOH is a relative scale factor and PerturbValueBCs is in ppb
+PerturbValue: 10.0
+PerturbValueOH: 1.1
 PerturbValueBCs: 10.0
 
 ## Apply scale factors from a previous inversion?