Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial ICI changes #238

Draft
wants to merge 114 commits into
base: dev
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
114 commits
Select commit Hold shift + click to select a range
882ce86
Creating default config, python environment, and GCC environment sett…
Mar 19, 2024
466fefc
Changing DataPathTROPOMI to DataPathObs (and adding Species to config…
Mar 20, 2024
4afa68e
Changes to generalize supporting other clusters, including adding Spe…
Mar 20, 2024
8b7e200
Default species for Harvard should be CH4
Mar 20, 2024
beb0651
Generalizing scheduler options
Mar 20, 2024
277bcf3
Moving conda environment activation into environment files
Mar 21, 2024
a217e2e
Removing conda environment specification from config file
Mar 21, 2024
e8312be
Removing conda environment specification from config file
Mar 21, 2024
5d3abb3
Moving conda environment activation into environment files
Mar 21, 2024
08023af
Removing UseSlurm option and replacing with UseScheduler/SchedulerType
Mar 21, 2024
a8576c4
Switching from UseSlurm to UseScheduler/SchedulerType
Mar 21, 2024
e8b13e7
Adding PythonEnv option
Mar 22, 2024
639a1f4
Adding PythonEnv to variables required for local simulation
Mar 22, 2024
55bd28c
Removing python loading from GEOS-Chem environment
Mar 22, 2024
5c05ed4
Creating separate environment file for python loading
Mar 22, 2024
0960227
Adding separate statement for loading Python env
Mar 22, 2024
fb1e776
Switching conda activate to source
Mar 22, 2024
fdf9e33
Handling python environment loading for AWS (now hardcoded?) and loca…
Mar 22, 2024
cdad798
Added reference to PythonEnv since htat's now specified along with GE…
Mar 22, 2024
035b444
Changes related to generalizing the IMI for multiple clusters
hannahnesser Apr 2, 2024
6e6980e
Amending config files to include PythonEnv and simplify SchedulerType…
hannahnesser Apr 2, 2024
59227ab
These files contained non-default uses of sbatch. As a result, we rep…
hannahnesser Apr 2, 2024
9eea594
Removed reference to UseScheduler
hannahnesser Apr 2, 2024
b0ef56b
Documentation changes consistent with the changes described in a prev…
hannahnesser Apr 2, 2024
d9d7c62
Adding tmux option to SchedulerType
hannahnesser Apr 2, 2024
e6bfae8
Adding species argument
hannahnesser Apr 2, 2024
656b013
Removing only aws requirementS
hannahnesser Apr 2, 2024
a2b24b4
Modified to handle tmux
hannahnesser Apr 3, 2024
6436535
Switched to standard SBATCH headers
Apr 11, 2024
01c522a
Adding in corrections for PBS
Apr 12, 2024
4279240
Changes to SBATCH to PBS conversion
Apr 12, 2024
17d4dda
Moved SBATCH to PBS conversion to common.sh
Apr 12, 2024
262ad9e
Added units to mem to avoid confusion in conversion from SBATCH to PBS
Apr 12, 2024
211932d
Added units to mem to avoid confusion in conversion from SBATCH to PBS
Apr 12, 2024
b29b878
Switched condaenv options to PythonEnv
Apr 12, 2024
5318cb8
Minor syntax fix
Jun 7, 2024
d0d6a41
Mostly changes to get replacement of SBATCH to PBS working
Jun 7, 2024
bc91e5e
Adding print statement to check
Jun 7, 2024
f133892
Continued bug fixes to awk
Jun 7, 2024
288d049
Continued debugging
Jun 8, 2024
6c829e6
Continued bug fixes
Jun 24, 2024
5d50a2a
Adding node request to sbatch
Jun 24, 2024
adbe07e
Removing nodes options, which will now be passed with sbatch -N directly
Jun 24, 2024
b9839a2
Changing default format of time request to HH:MM:SS for consistency b…
Jun 25, 2024
338d10a
Removed BlendedTROPOMI variable and replaced it with satellite_str (s…
Jun 26, 2024
dded98c
Removing references to TROPOMI and correcting function names to match…
Jun 26, 2024
ecdbe13
Removing references to TROPOMI and correcting function names to match…
Jun 26, 2024
a6ed832
Switched HourlyCH4 to HourlySpecies, changed ch4_run.template to run.…
Jun 27, 2024
ad5303e
Changed HourlyCH4 to HourlySpecies and ch4_run.template to run.template
Jun 27, 2024
4b55f6b
Changed tropomi_cache --> satellite_cache
Jun 27, 2024
694dfd5
Moving away from specific TROPOMI/CH4 references (mostly in variable …
Jun 27, 2024
9156a77
Changes mostly to variable names to move away from TROPOMI-specific a…
Jun 27, 2024
2f83ab5
Removing TROPOMI specific references
Jun 27, 2024
ac3385a
- Added species argument to average_satellite_observations
Jun 27, 2024
dfce7e6
Adding functions: mixing_ratio_conv_factor, species_molar_mass, read_…
Jun 27, 2024
9a33d72
Change BlendedTROPOMI to SatelliteProduct, change HourlyCH4 to Hourly…
Jun 27, 2024
1ab74f2
While this script is implicitly very TROPOMI-specific, we still remai…
Jun 27, 2024
2873f6c
- Change ch4_run.template to use the species variable from the config…
Jun 27, 2024
49fc3e1
Changed data directory name from data_TROPOMI to data_satellite; ther…
Jun 27, 2024
771b962
Changed ch4_run.template to <species>_run.template
Jun 27, 2024
e128bda
Changed ch4_run.template to <species>_run.template
Jun 27, 2024
46b6dd3
Move UseBCsForRestart from setup.sh to config file
Jun 27, 2024
0156426
Switched from HourlyCH4 to HourlySpecies and from ch4_run.template to…
Jun 27, 2024
af530d4
Changed name of TROPOMI data directory from data_TROPOMI to data_sate…
Jun 27, 2024
dafd54c
Changed HourlyCH4 to HourlySpecies and ch4_run.template to <species>_…
Jun 27, 2024
79127be
Added species argument to main options
Jun 28, 2024
74d79df
Changed tropomiCache to satelliteCache and addded Species and Satelli…
Jun 28, 2024
262a478
Removed bug check print statements and fixed a few PBS specific bugs
Jun 28, 2024
275295b
Corrected calls to write_boundary_conditions to include new arguments
Jun 28, 2024
0013b7f
Changed tropomiCache to satelliteCache, data_TROPOMI default storage …
Jun 28, 2024
3bc4444
Changed HourlyCH4 to HourlySpecies and BlendedTROPOMI to SatellitePro…
Jun 28, 2024
2150008
Changed HourlyCH4 to HourlySpecies
Jun 28, 2024
02090d7
Changed <species>_run.template to run.template
Jun 28, 2024
6a6f6de
Generalizing name of run template
Jun 28, 2024
fd0c727
Changed ch4_run.template to just run.template
Jun 28, 2024
732e551
Local changes
Jun 28, 2024
9c4bc72
Changing back to default config files
Jun 28, 2024
d96bf30
Merging dev and ICI branches before moving the ICI branch to a sub br…
Jun 28, 2024
a8baba5
Moving sbatch requests to use the submit_job function
Jun 28, 2024
f8a67e6
Adding False argument for the new viz Boolean
Jun 28, 2024
ae02566
Adding species arguments
Jun 28, 2024
56eea96
Removing specific CH4 references and replacing with {species} and doi…
Jun 28, 2024
2086e7c
Fixing comment string
Jul 1, 2024
d24cb3e
Updating to include new flags from dev branch
Jul 1, 2024
5243ef9
Removing condaEnv/condaFile refs and replacing with PythonEnv per the…
Jul 1, 2024
2864204
Adjusting for SBATCH options that were previously not caught by conve…
Jul 1, 2024
9665bb4
Adding a check to see if the PBS -l site=needed option has been previ…
Jul 1, 2024
8a8b167
Printing out any SBATCH options not caught by the conversion script
Jul 1, 2024
15c139a
Removed print statement that didn't really work
Jul 1, 2024
02e3747
Removing activation of python, which we get instead from a designated…
Jul 2, 2024
2190404
Switched tabs to spaces for readability
Jul 2, 2024
2e1d6bc
Changed tabs to spaces and changed hard coded GEOSCHEM_VERSION to the…
Jul 2, 2024
32efc58
Changes to allow submit_job to take a SaveOutput true/false boolean
Jul 2, 2024
ac774b0
Added SaveOut option to submit_job
Jul 2, 2024
208ffe8
Added SaveOut option to submit_job
Jul 2, 2024
62cf5c4
Looks like I accidentally deletd a fi?
Jul 2, 2024
7d5698f
Added a missing fi
Jul 2, 2024
192eadf
Updated config for Pleiades
Jul 2, 2024
b936c5b
Resolving merge conflicts with bugfix/jacobian-perturbation-fix
Jul 3, 2024
94d65ee
Switching from Simulation CPUs/Memory to Requested CPUs/Memory
Jul 3, 2024
35c4a09
Removing commented out vestige
Jul 3, 2024
958b1ea
Removing lingering references to TROPOMI
Jul 3, 2024
69ec97a
Updating options to better match bugfix/jacobian-perturbation-fix con…
Jul 3, 2024
acf033f
UseEmisSF and UseOHSF seem to have been removed as options
Jul 3, 2024
a156273
- Bug fix for BCs being used in all simulations, not just regional si…
Jul 3, 2024
4d30edb
Beginning to generalize HEMCO tracer changes for not just methane
Jul 3, 2024
da7fe89
Generalized function to be a function of species
Jul 3, 2024
e951e42
Changed from tabs to spaces for readability and removed portions of c…
Jul 3, 2024
c9ca0c3
Updating
Jul 3, 2024
4c87fbc
Merge resolution with dev branch
Jul 3, 2024
03fd4ea
Merge remote-tracking branch 'origin/dev'
laestrada Oct 22, 2024
6767732
Merge remote-tracking branch 'origin/dev'
laestrada Oct 23, 2024
e64b99b
Merge remote-tracking branch 'origin/dev'
laestrada Oct 31, 2024
c754c5a
Resolving merge conflicts
Nov 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Changing back to default config files
  • Loading branch information
Hannah Nesser committed Jun 28, 2024
commit 9c4bc72436f994e16c68e31a68524a3ac9d874a2
95 changes: 44 additions & 51 deletions config.yml
Original file line number Diff line number Diff line change
@@ -2,16 +2,16 @@
## Documentation @ https://imi.readthedocs.io/en/latest/getting-started/imi-config-file.html

## General
RunName: "Test_ICI_Global"
Species: "CO2"
isAWS: false
SchedulerType: "PBS"
SafeMode: false
RunName: "Test_Permian_1week"
Species: "CH4"
isAWS: true
SchedulerType: "slurm"
SafeMode: true
S3Upload: false

## Period of interest
StartDate: 20221001
EndDate: 20221003
StartDate: 20180501
EndDate: 20180508
SpinupMonths: 1

## What satellite data product should be used? Current options are:
@@ -26,7 +26,7 @@ SatelliteProduct: "Other"
# BlendedTROPOMI: false

## Is this a regional inversion? Set to false for global inversion
isRegional: false
isRegional: true

## Select two character region ID (for using pre-cropped meteorological fields)
## Current options are listed below with ([lat],[lon]) bounds:
@@ -40,15 +40,15 @@ isRegional: false
## "SA" : South America ([-59,16], [-88,-31])
## "" : Use for global global simulation or custom regions
## For example, if the region of interest is in Europe ([33,61],[-30,70]), select "EU".
RegionID: ""
RegionID: "NA"

## Region of interest
## These lat/lon bounds are only used if CreateAutomaticRectilinearStateVectorFile: true
## Otherwise lat/lon bounds are determined from StateVectorFile
LonMin: -102.5
LonMax: -87.5
LatMin: 16
LatMax: 24
LonMin: -105
LonMax: -103
LatMin: 31
LatMax: 33

## Kalman filter options
KalmanMode: false
@@ -57,11 +57,11 @@ NudgeFactor: 0.1

## State vector
CreateAutomaticRectilinearStateVectorFile: true
nBufferClusters:
BufferDeg: 0
OptimizeBCs: false
nBufferClusters: 8
BufferDeg: 5
LandThreshold: 0.25
OffshoreEmisThreshold: 0
OptimizeBCs: false
OptimizeOH: false

## Point source datasets
@@ -77,8 +77,8 @@ ForcedNativeResolutionElements:
- [31.5, -104]

## Custom state vector
StateVectorFile: "/path/to/StateVector.nc"
ShapeFile: "None"
StateVectorFile: "/home/ubuntu/integrated_methane_inversion/resources/statevectors/StateVector.nc"
ShapeFile: "/home/ubuntu/integrated_methane_inversion/resources/shapefiles/PermianBasin_Extent_201712.shp"

## Inversion
## Note PriorError and PriorErrorOH are relative fractions (e.g. 0.5 = 50%)
@@ -92,25 +92,25 @@ PrecomputedJacobian: false

## Grid
## Options are 0.25x0.3125 (GEOSFP only), 0.5x0.625, 2.0x2.5, or 4.0x5.0
Res: "2.0x2.5"
Res: "0.25x0.3125"

## Meteorology
## Options are GEOSFP or MERRA2
Met: "MERRA2"
Met: "GEOSFP"

## Setup modules
## Turn on/off different steps in setting up the inversion
SetupTemplateRundir: true
SetupSpinupRun: true
SetupJacobianRuns: true
SetupSpinupRun: false
SetupJacobianRuns: false
SetupInversion: false
SetupPosteriorRun: false

## Run modules
## Turn on/off different steps in performing the inversion
RunSetup: false
DoSpinup: true
DoJacobian: true
RunSetup: true
DoSpinup: false
DoJacobian: false
DoInversion: false
DoPosterior: false

@@ -124,11 +124,11 @@ SimulationCPUs: 32
SimulationMemory: "32gb"
JacobianCPUs: 1
JacobianMemory: 2000
RequestedTime: "01:00:00"
RequestedTime: "24:00:00"
SchedulerPartition: "debug"

## Max number of simultaneous Jacobian runs from the job array (-1: no limit)
MaxSimultaneousRuns: 50
MaxSimultaneousRuns: -1

##====================================================================
##
@@ -181,40 +181,33 @@ UseBCsForRestart: False
##------------------------------------------------------------------

## Path for IMI runs and output
OutputPath: "/nobackupp27/$USER/IMI_demo"
OutputPath: "/home/ubuntu/imi_output_dir"

## Path to GEOS-Chem input data
DataPath: "/nobackupp27/$USER/ExtData"

## Path to satellite data
# DataPathObs: "/nobackup/$USER/CO2_inversion/observations/OCO-2"
DataPathObs: "/nobackupp27/$USER/IMI_demo/data_TROPOMI"

## GEOS-Chem environment file (with fortran compiler, netcdf libraries, etc.)
## NOTE: Copy your own file in the envs/ directory within the IMI
GEOSChemEnv: "envs/NASA-Pleiades/gcclassic.pleiades.env"
DataPath: "/home/ubuntu/ExtData"

## Python environment file (this is normally one or two lines)
PythonEnv: "envs/NASA-Pleiades/python.env"
## Conda environment file
PythonEnv: "/home/ubuntu/integrated_methane_inversion/envs/aws/python.env"

## Download initial restart file from AWS S3?
## NOTE: Must have AWS CLI enabled
RestartDownload: false
## NOTE: Must have AWS CLI enabled
RestartDownload: true

## Path to initial GEOS-Chem restart file + prefix
## ("YYYYMMDD_0000z.nc4" will be appended)
RestartFilePrefix: "/nobackup/$USER/CO2_inversion/restart_"
RestartFilePreviewPrefix: "/nobackup/$USER/CO2_inversion/restart_"
RestartFilePrefix: "/home/ubuntu/ExtData/BoundaryConditions/GEOSChem.BoundaryConditions."
RestartFilePreviewPrefix: "/home/ubuntu/ExtData/BoundaryConditions/GEOSChem.BoundaryConditions."

## Path to GEOS-Chem boundary condition files (for regional simulations)
## BCversion will be appended to the end of this path. ${BCpath}/${BCversion}
BCpath: "/nobackup/$USER"
BCversion: "v2023-10"
BCpath: "/home/ubuntu/ExtData/BoundaryConditions"
BCversion: "v2023-06"

## Options to download missing GEOS-Chem input data from AWS S3
## NOTE: Must have AWS CLI enabled
PreviewDryRun: false
SpinupDryrun: false
ProductionDryRun: false
PosteriorDryRun: false
BCdryrun: false
## NOTE: You will be charged if your ec2 instance is not in the
## us-east-1 region.
PreviewDryRun: true
SpinupDryrun: true
ProductionDryRun: true
PosteriorDryRun: true
BCdryrun: true
38 changes: 20 additions & 18 deletions envs/NASA-Pleiades/config.nasa-pleiades.global_inv.yml
Original file line number Diff line number Diff line change
@@ -6,12 +6,12 @@ RunName: "Test_ICI_Global"
Species: "CO2"
isAWS: false
SchedulerType: "PBS"
SafeMode: true
SafeMode: false
S3Upload: false

## Period of interest
StartDate: 20180501
EndDate: 20180502
StartDate: 20221001
EndDate: 20221003
SpinupMonths: 1

## What satellite data product should be used? Current options are:
@@ -45,10 +45,10 @@ RegionID: ""
## Region of interest
## These lat/lon bounds are only used if CreateAutomaticRectilinearStateVectorFile: true
## Otherwise lat/lon bounds are determined from StateVectorFile
LonMin: -180
LonMax: 180
LatMin: -90
LatMax: 90
LonMin: -102.5
LonMax: -87.5
LatMin: 16
LatMax: 24

## Kalman filter options
KalmanMode: false
@@ -57,7 +57,7 @@ NudgeFactor: 0.1

## State vector
CreateAutomaticRectilinearStateVectorFile: true
nBufferClusters: 0
nBufferClusters:
BufferDeg: 0
OptimizeBCs: false
LandThreshold: 0.25
@@ -96,21 +96,21 @@ Res: "2.0x2.5"

## Meteorology
## Options are GEOSFP or MERRA2
Met: "GEOSFP"
Met: "MERRA2"

## Setup modules
## Turn on/off different steps in setting up the inversion
SetupTemplateRundir: true
SetupSpinupRun: false
SetupJacobianRuns: false
SetupSpinupRun: true
SetupJacobianRuns: true
SetupInversion: false
SetupPosteriorRun: false

## Run modules
## Turn on/off different steps in performing the inversion
RunSetup: true
DoSpinup: false
DoJacobian: false
RunSetup: false
DoSpinup: true
DoJacobian: true
DoInversion: false
DoPosterior: false

@@ -124,7 +124,8 @@ SimulationCPUs: 32
SimulationMemory: "32gb"
JacobianCPUs: 1
JacobianMemory: 2000
RequestedTime: "24:00:00"
RequestedTime: "01:00:00"
SchedulerPartition: "debug"

## Max number of simultaneous Jacobian runs from the job array (-1: no limit)
MaxSimultaneousRuns: 50
@@ -180,13 +181,14 @@ UseBCsForRestart: False
##------------------------------------------------------------------

## Path for IMI runs and output
OutputPath: "/nobackup/$USER"
OutputPath: "/nobackupp27/$USER/IMI_demo"

## Path to GEOS-Chem input data
DataPath: "/nobackup/$USER/ExtData"
DataPath: "/nobackupp27/$USER/ExtData"

## Path to satellite data
DataPathObs: "/nobackup/$USER/CO2_inversion/observations/OCO-2"
# DataPathObs: "/nobackup/$USER/CO2_inversion/observations/OCO-2"
DataPathObs: "/nobackupp27/$USER/IMI_demo/data_TROPOMI"

## GEOS-Chem environment file (with fortran compiler, netcdf libraries, etc.)
## NOTE: Copy your own file in the envs/ directory within the IMI
Loading