Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Phydro merge into main #222

Draft
wants to merge 129 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
129 commits
Select commit Hold shift + click to select a range
73df4f2
enable run with extra columns in driver data
jaideep777 Sep 11, 2023
193079b
code compiles with phydro module PROVIDED mod file is first created
jaideep777 Sep 11, 2023
b1d9b27
basic working gpp calc; compiles correctly; use_phydro option
jaideep777 Sep 11, 2023
032a02c
working gpp from instantaneous model
jaideep777 Sep 12, 2023
0c1d7f2
correct unit for obs LE, identified numerical issue in gs with ET_DIF…
jaideep777 Sep 12, 2023
bfa6c57
phydro now takes pa rather than elv as input
jaideep777 Sep 12, 2023
97936d1
add netrad to averager, fix output units
jaideep777 Sep 12, 2023
8adda14
jaideep's comments on equations in waterbal
jaideep777 Sep 12, 2023
f052df8
working latent energy calcs in waterbal
jaideep777 Sep 12, 2023
cd0cc9c
add use_pml as program option; add pml comparison in vignette
jaideep777 Sep 12, 2023
7088d63
feed back wcont from waterbal to phydro
jaideep777 Sep 14, 2023
0f0d093
all params are now supplied from R
jaideep777 Oct 20, 2023
0880620
working version with all vars in the right classes
jaideep777 Nov 10, 2023
a39b3ef
basic working kphio calib with pmodel; with whc in params
jaideep777 Nov 11, 2023
52033ad
working gensa/bayesian calibration with gpp and le
jaideep777 Nov 11, 2023
d04e469
add full calibration in vignette
jaideep777 Nov 11, 2023
b7f5781
fix NaN in e; <--- All sites calibrated here
jaideep777 Nov 12, 2023
ac03b51
fix pmodel calibration (separate vignette)
jaideep777 Nov 14, 2023
230b9a8
param env dependencies plot
jaideep777 Nov 15, 2023
237824d
try calib with alpha/gamma fixed
jaideep777 Nov 15, 2023
613a5f2
Pull changes from upstream master into phydro_new
jaideep777 Nov 17, 2023
5bab098
modify vignette to ensure that code works with pulled changes
jaideep777 Nov 17, 2023
ffc9097
get forcing_acclim dataframe into fortran
jaideep777 Nov 17, 2023
4a9aeff
add acclimation forcing
jaideep777 Nov 17, 2023
a9a275a
add 2-param SWP function; limit SWP for numerical stability; bash script
jaideep777 Jan 7, 2024
7d91975
completed pipeline for batch calibration
jaideep777 Jan 17, 2024
1c86008
read data and output paths from cmdline args
jaideep777 Jan 17, 2024
497fe22
update script with data/output dir inputs
jaideep777 Jan 17, 2024
17efddd
first run for all 250 sites attempted here.
jaideep777 Jan 17, 2024
b45ad15
try P/PET to regulate le_soil; output lwp, le_soil; script to generat…
jaideep777 Jan 31, 2024
2c5b1f6
improved calib script; trial with const kphio
jaideep777 Feb 21, 2024
ef0ed2e
add r2 and nrmse outputs
jaideep777 Feb 21, 2024
7ff00b8
include obs quantities in phydro_output for convenience
jaideep777 Feb 22, 2024
da09478
using p50 priors
jaideep777 Feb 27, 2024
17f5896
narrower alpha prior; r2 map
jaideep777 Feb 28, 2024
0514f72
nothing really
stineb Feb 29, 2024
f18238a
Merge branch 'master' of github.com:computationales/rsofun
stineb Feb 29, 2024
57a9fc6
use new cwdx80_forcing nc file
jaideep777 May 26, 2024
188b2e6
Merge pull request #2 from geco-bern/master
jaideep777 May 26, 2024
ae3f04a
update phydro_new with changes from upstream/master
jaideep777 May 26, 2024
301c3cd
revert some changes to ensure phydro runs. Working for AU-ASM
jaideep777 May 30, 2024
e1204e1
CRITICAL BUGFIX: lines were silently deleted during merge, restore them
jaideep777 Jun 14, 2024
21055fb
minor bugfixes, prototype multi-site calib, add new cost fn for phydro
jaideep777 Jun 29, 2024
2be7e39
export phydro cost fn
jaideep777 Jun 29, 2024
098f789
bugfix in cost fn phydro
jaideep777 Jun 29, 2024
1de60cb
minor bugfix
jaideep777 Jun 29, 2024
16b8264
Merge pull request #213 from jaideep777/phydro_new
stineb Jul 9, 2024
905176f
trial run with phydro cost fn
jaideep777 Aug 3, 2024
a89eae1
add driver visualization in vignette; add vignette with new-format data
jaideep777 Aug 4, 2024
a65e110
Merge branch 'hotfix/newdataformat' into phydro_new
jaideep777 Aug 4, 2024
84beb01
updated vignettes for using pmodel and phydro with old and new data f…
jaideep777 Aug 4, 2024
c4ff2f4
gapfill and use forcing_acclim in phydro vignette
jaideep777 Aug 4, 2024
8535b09
add ET formulation for pmodel (similar to phydro); move soil LE calcs…
jaideep777 Aug 7, 2024
30e131c
Merge pull request #216 from jaideep777/phydro_new
stineb Aug 9, 2024
b3c77a6
nothing much, just for the looks
stineb Aug 9, 2024
a2dd4d7
implemented root zone water storage capacity-based soil moisture stre…
stineb Aug 9, 2024
2dbe7c8
Merge branch 'phydro' of github.com:geco-bern/rsofun into phydro
stineb Aug 12, 2024
500d79f
replace ET_* with T_* in phydro options; reserving ET for splash+phydro
jaideep777 Aug 13, 2024
8ae7fb3
add 3 ET options via: using_gs, using_pml
jaideep777 Aug 13, 2024
4d5950e
R interface for all ET options; update vignette to demonstrate
jaideep777 Aug 13, 2024
9d5fc70
remove nested functions
jaideep777 Aug 13, 2024
55c4dfa
delete unnecessary files from vignettes; use default settings with wa…
jaideep777 Aug 13, 2024
60575ed
add new format demo data
jaideep777 Aug 13, 2024
106a0e9
Merge pull request #219 from jaideep777/phydro_new
stineb Aug 14, 2024
cdcdefe
nothing much. just niceifications.
stineb Aug 14, 2024
c4f8cf3
implemented PML in waterbal routine (to be used in combination with p…
stineb Aug 15, 2024
5de81fb
tested: identical results for gs-coupled ET with diffusion (not PML).
stineb Aug 15, 2024
21f4e98
got PML with P-model Gs running. Complemented vignette. Performs very…
stineb Aug 15, 2024
a234d36
nothing much
stineb Sep 16, 2024
c1509f1
bugfix to make le follow aet consistently
stineb Sep 16, 2024
f9a6a20
added script for demo
stineb Sep 16, 2024
af63378
Update test-model-runs for PHYDRO
fabern Sep 16, 2024
1e4bb4e
fix warning: Unknown or uninitialised column: `forcing_acclim`
fabern Sep 16, 2024
6ff8a82
Deactivate printing soilmstress
fabern Sep 16, 2024
f1366e3
test: Add consistency check for LE and AET (and soil vs canopy)
fabern Sep 16, 2024
fb4b1ab
fix consistency check AET_tot = AET_soil + AET_canop
fabern Sep 16, 2024
15ca3b1
fix: reduce daet_soil and daet_e_soil proportionally when daet is red…
fabern Sep 17, 2024
e68edd5
Remove soilm_betao
fabern Sep 18, 2024
07603a7
Make model parameter check more robust
fabern Sep 18, 2024
6f6a6f2
Make parameter calibration more robust
fabern Sep 18, 2024
d69acda
Fix test-calibration-pmodel.R: add 'whc' and switch prior to uniform
fabern Sep 19, 2024
c836f63
Fix Vcmax25 tests in test-calibration-pmodel
fabern Sep 19, 2024
4674614
Make non-parallel runread_pmodel_f more robust
fabern Sep 19, 2024
08e9cb1
Make parallel runread_pmodel_f more robust and simple
fabern Sep 19, 2024
7fd4485
Make error message in required_param_names more explicit
fabern Sep 19, 2024
a598e0c
added minimal number to avoid division by zero for surface conductanc…
stineb Sep 20, 2024
51961c0
resolved merge conflict
stineb Sep 20, 2024
41f90ff
test: Add reference test-model runs for P and PM model
fabern Sep 23, 2024
40e2ada
test: Fix quantitative validation tests
fabern Sep 23, 2024
116d3d2
test: Skip biomee tests (TODO need to fix logp == -Inf)
fabern Sep 23, 2024
9c50c33
test: Reduce tolerance of regression tests
fabern Sep 23, 2024
9784913
docs: Clean comments (references)
fabern Sep 23, 2024
bee72f3
Merge branch 'phydro' into phydro-fix-le-output
fabern Sep 23, 2024
1e02539
Merge pull request #221 from geco-bern/phydro-fix-le-output
fabern Sep 23, 2024
7a44e5e
ci: Fix CI checks
fabern Sep 24, 2024
65d232c
Fix vignette on sensitivity
fabern Sep 24, 2024
8b55702
feat: Parallelize Bayesian calibration of sofun
fabern Sep 24, 2024
99b053a
refactor: Make likelihood computation more robust
fabern Sep 24, 2024
d06d211
fix: Vignette on sensitivity analysis
fabern Sep 24, 2024
9c6bdf1
test: Fix testing path for R CMD CHECK
fabern Sep 24, 2024
0fbb949
Attempt to fix R CMD Check by direct use of example driver
fabern Sep 25, 2024
d082cd9
Fix: analysis/01-sensitivity-analysis.R
fabern Sep 25, 2024
36c1259
docs: Fix example of calib_sofun()
fabern Sep 25, 2024
ebc788a
Fix errors in R CMD Check
fabern Sep 30, 2024
de83f04
Add dplyr:: and tidyr::
fabern Oct 2, 2024
d44e3f8
Add boilerplate code for phydro regression test (currently deactivate…
fabern Oct 2, 2024
da95557
Fix non initialized dpsi and psi_leaf variables.
Oct 3, 2024
d7287d9
Merge remote-tracking branch 'origin/phydro' into phydro
Oct 3, 2024
dff710a
test: Add memory leak tests for pmodel
fabern Oct 8, 2024
9c1e0f8
Merge branch 'master' into phydro
marcadella Oct 8, 2024
65e8b23
feat: revert: "Parallelize Bayesian calibration of sofun"
fabern Oct 9, 2024
738dd1d
Re-designed WHC: treated as site parameter, not model parameter struc…
stineb Oct 10, 2024
bfd458f
tests: Update tests for new WHC as site parameter
fabern Oct 10, 2024
8e1f281
tests: Use FR-Pue WHC in test-model-runs.R
fabern Oct 10, 2024
d0dc489
tests: Add reference test for calibrated params
fabern Oct 10, 2024
25b33b9
made vignette a bit nicer
stineb Oct 13, 2024
2b82b2e
removed stuff that was unnecessarily added to this vignette (visualis…
stineb Oct 13, 2024
bd7a57c
Merge branch 'phydro' of github.com:geco-bern/rsofun into phydro
stineb Oct 13, 2024
c052e92
renamed vignette demonstrating pmodel calibration
stineb Oct 13, 2024
8499575
revamped pmodel_use vignette and complemented with demo for ET and di…
stineb Oct 13, 2024
34b6531
using wind speed from forcing data frame for aerodynamic conductance
stineb Oct 15, 2024
6513b8e
scaling gs_acclim with PPFD; multiplying soil evaporation with 1.26 (…
stineb Oct 15, 2024
b6ab2d4
moved multiplication with PPFD to where gs_setpoint is calculated; in…
stineb Oct 15, 2024
6c38615
nothing
stineb Oct 28, 2024
2a26201
removed obsolete argument
stineb Nov 6, 2024
c443f78
add variable use_pml in gpp functions
FrancescoGrossi-unimi Nov 12, 2024
6936379
removed argument us_pml
stineb Nov 12, 2024
01c3d3e
Merge branch 'phydro' of github.com:geco-bern/rsofun into phydro
stineb Nov 12, 2024
dd586f9
removed argument us_pml
FrancescoGrossi-unimi Nov 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
src/*.o
src/*.so
src/*.dll
src/*.mod

# vignette builds
vignettes/*.html
Expand Down
10 changes: 8 additions & 2 deletions DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -61,14 +61,20 @@ LazyData: true
LazyDataCompression: xz
ByteCompile: true
NeedsCompilation: yes
RoxygenNote: 7.3.1
RoxygenNote: 7.3.2
Suggests:
covr,
cwd,
rcmdcheck,
testthat,
rmarkdown,
ggplot2,
knitr,
sensitivity
sensitivity,
hexbin,
khroma,
yardstick
Remotes:
geco-bern/cwd
VignetteBuilder: knitr
Encoding: UTF-8
1 change: 1 addition & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

export(calib_sofun)
export(cost_likelihood_biomee)
export(cost_likelihood_phydromodel)
export(cost_likelihood_pmodel)
export(cost_rmse_biomee)
export(cost_rmse_pmodel)
Expand Down
57 changes: 29 additions & 28 deletions R/calib_sofun.R
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
#' \item{\code{par}}{A list of model parameters. For each parameter, an initial value
#' and lower and upper bounds should be provided. The calibratable parameters
#' include model parameters 'kphio', 'kphio_par_a', 'kphio_par_b', 'soilm_thetastar',
#' 'soilm_betao', 'beta_costunitratio', 'rd_to_vcmax', 'tau_acclim', 'kc_jmax'
#' 'beta_costunitratio', 'rd_to_vcmax', 'tau_acclim', 'kc_jmax'
#' and 'rootzone_whc' , and (if
#' doing Bayesian calibration) error parameters
#' for each target variable, named for example 'err_gpp'. This list must match
Expand Down Expand Up @@ -49,11 +49,11 @@
#' kphio_par_a = 0,
#' kphio_par_b = 1.0,
#' soilm_thetastar = 0.6*240,
#' soilm_betao = 0.01,
#' beta_unitcostratio = 146,
#' rd_to_vcmax = 0.014,
#' tau_acclim = 30,
#' kc_jmax = 0.41
#' kc_jmax = 0.41,
#' whc = 432
#' )
#'
#' # Define calibration settings
Expand All @@ -68,12 +68,12 @@
#' sampler = "DEzs",
#' settings = list(
#' nrChains = 1,
#' burnin = 0,
#' iterations = 50 # kept artificially low
#' burnin = 0,
#' iterations = 50 # kept artificially low,
#' )
#' )
#' )
#'
#'
#' # Run the calibration for GPP data
#' calib_output <- rsofun::calib_sofun(
#' drivers = rsofun::p_model_drivers,
Expand Down Expand Up @@ -120,7 +120,7 @@ calib_sofun <- function(
# create bounds
lower <- unlist(lapply(settings$par, function(x) x$lower))
upper <- unlist(lapply(settings$par, function(x) x$upper))
pars <- unlist(lapply( settings$par, function(x) x$init))
pars <- unlist(lapply(settings$par, function(x) x$init))

out <- GenSA::GenSA(
par = pars,
Expand Down Expand Up @@ -149,39 +149,40 @@ calib_sofun <- function(
eval(settings$metric)(par = par,
obs = obs,
drivers = drivers,
...)
...) # the dots contain: par_fixed, targets, parallel, ncores
}

# reformat parameters
pars <- as.data.frame(do.call("rbind", settings$par), row.names = FALSE)

priors <- BayesianTools::createUniformPrior(
pars <- as.data.frame(do.call("rbind", settings$par)) # use rownames later on

# priors <- BayesianTools::createTruncatedNormalPrior(
# unlist(pars$mean), # NOTE: This needs a value otherwise: Error in `parallelSampler(1000)`: sampler provided doesn't work
# unlist(pars$sd), # NOTE: This needs a value otherwise: Error in `parallelSampler(1000)`: sampler provided doesn't work
# unlist(pars$lower), # As a workaround BayesianTools::createUniformPrior could be used
# unlist(pars$upper)
# # unlist(pars$init)
# )
priors <- BayesianTools::createUniformPrior( # workaround for TruncatedNormalPrior, this does not require mean and sd
unlist(pars$lower),
unlist(pars$upper),
unlist(pars$init)
unlist(pars$upper)
# unlist(pars$init)
)

# setup the bayes run, no message forwarding is provided
# so wrap the function in a do.call
setup <- BayesianTools::createBayesianSetup(
likelihood = function(
random_par) {
# cost(
# par = random_par,
# obs = obs,
# drivers = drivers,
# ...
# )
likelihood = function(random_par) {
do.call("cost",
list(
par = random_par,
par = setNames(random_par, rownames(pars)),
obs = obs,
drivers = drivers
))
},
prior = priors,
names = names(settings$par)
)
))},
prior = priors,
names = rownames(pars)#,
#parallel = TRUE,
#parallelOptions = list(variables = "all", packages = "all", dlls = NULL), # TODO: this default option might be tweaked
)

# set bt control parameters
bt_settings <- settings$control$settings
Expand All @@ -192,7 +193,7 @@ calib_sofun <- function(
sampler = settings$control$sampler,
settings = bt_settings
)

# drop last value
bt_par <- BayesianTools::MAP(out)$parametersMAP
bt_par <- bt_par[1:(length(bt_par))]
Expand Down
Loading
Loading