Merge pull request #60 from QuaSi-Software/feature_DateTime_timestamps

Enhance time handling and profile definition with datetime indexes and improved profile conversion. Version bump to v0.10.1 with the following changes: * Change handling and definition of timesteps by introducing datetime indexes for profiles and for internal handling: * Profile values in ReSiE are now defined as the mean/sum of the timespan following the current timestep. This affects mainly the weather data from a global weather file. If this is not used, any definition can be used, the provided profiles just have to be consistent. * Simulation start and end time have to be given as datetime now. * Simulation time step can now be given in multiple time formats: seconds, minutes, hours * Profiles now have to be referenced in time and have to cover the simulation start and end time. * Major restructuring of profile definition: Can now be startdate & timestepsize, startdate & timestamp or a datestamp with custom format. * Add parameter "data_type" in profiles and explicitly setting it as "intensive" or "extensive" values. "is_power" is no longer used. * Add parameter "time_shift_seconds" in profiles to manually shift profile data by a given time span * Add parameter "use_linear_segmentation" if linear and not stepwise interpolation should be used for segmentation. * Fix import of weather data to meet new time definition in ReSiE, especially for intensive values of EPW which are defined as values at the time indicated. * Add multi-year usage of weather data files. Weatherdata from weather file can have any year. * Improve segmentation and aggregation algorithms. Segmentation can now be either linear or stepwise (default) interpolation, depending on user input. * Add handling of daylight savings: Internally, ReSiE uses now local standard time. Profiles with a datestamp including DST are converted to local standard time. Output is always local standard time without DST. DST in EWP-Header will not be considered. * Add handling of leap days: They are ignored in all input and output. See documentation for details. * Line plot x-axis is now configurable to show seconds, hours or a datetime index. * Removed deprecated internal parameter "is_first_timestep" as this is now indicated by "time" that starts always at zero. * Add input of coordinates (optional) and reading and conversion of coordinates given in weather data files (default) * Add long wave irradiation as available profile for other components from weather file * Add and improve tests to cover profile conversion algorithms * Add default simulation parameters for tests * Add Julia packages Dates, TimeZones, Proj (for coordinate transformation) * For handling of timezones beyond 2038, files for a required recompilation of the package TimeZones and a description in the README has been included.
QuaSi-Software · Sep 6, 2024 · 02dffe0 · 02dffe0
2 parents 90611e2 + cd4f5b1
commit 02dffe0
Show file tree

Hide file tree

Showing 140 changed files with 11,577 additions and 10,521 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -4,6 +4,30 @@ In general the development follows the [semantic versioning](https://semver.org/
 ## Pre-1.0-releases
 As per the definition of semantic versioning and the reality of early development, in versions prior to 1.0.0 any release might break compatability. To alleviate this somewhat, the meaning of major-minor-patch is "downshifted" to zero-major-minor. However some breaking changes may slip beneath notice.
 
+### Version 0.10.1
+* Change handling and definition of timesteps by introducing datetime indexes for profiles and for internal handling:
+  * Profile values in ReSiE are now defined as the mean/sum of the timespan following the current timestep. This affects mainly the weather data from a global weather file. If this is not used, any definition can be used, the provided profiles just have to be consistent.
+  * Simulation start and end time have to be given as datetime now.
+  * Simulation time step can now be given in multiple time formats: seconds, minutes, hours
+  * Profiles now have to be referenced in time and have to cover the simulation start and end time.
+  * Major restructuring of profile definition: Can now be startdate & timestepsize, startdate & timestamp or a datestamp with custom format.
+  * Add parameter "data_type" in profiles and explicitly setting it as "intensive" or "extensive" values. "is_power" is no longer used.
+  * Add parameter "time_shift_seconds" in profiles to manually shift profile data by a given time span
+  * Add parameter "use_linear_segmentation" if linear and not stepwise interpolation should be used for segmentation.
+  * Fix import of weather data to meet new time definition in ReSiE, especially for intensive values of EPW which are defined as values at the time indicated.
+  * Add multi-year usage of weather data files. Weatherdata from weather file can have any year.
+  * Improve segmentation and aggregation algorithms. Segmentation can now be either linear or stepwise (default) interpolation, depending on user input.
+  * Add handling of daylight savings: Internally, ReSiE uses now local standard time. Profiles with a datestamp including DST are converted to local standard time. Output is always local standard time without DST. DST in EWP-Header will not be considered.
+  * Add handling of leap days: They are ignored in all input and output. See documentation for details.
+  * Line plot x-axis is now configurable to show seconds, hours or a datetime index.
+* Removed deprecated internal parameter "is_first_timestep" as this is now indicated by "time" that starts always at zero.
+* Add input of coordinates (optional) and reading and conversion of coordinates given in weather data files (default)
+* Add long wave irradiation as available profile for other components from weather file
+* Add and improve tests to cover profile conversion algorithms
+* Add default simulation parameters for tests
+* Add Julia packages Dates, TimeZones, Proj (for coordinate transformation)
+* For handling of timezones beyond 2038, files for a required recompilation of the package TimeZones and a description in the README has been included.
+
 ### Version 0.9.4
 * Improve geothermal probe:
   * fix interpolation of g-functions

diff --git a/Project.toml b/Project.toml
@@ -14,5 +14,7 @@ JuliaFormatter = "98e50ef6-434e-11e9-1051-2b60c6c9e899"
 Logging = "56ddb016-857b-54e1-b83d-db4d58db5568"
 PlotlyJS = "f0f68f2c-4968-5e81-91da-67840de0976a"
 Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
+Proj = "c94c279d-25a6-4763-9509-64d165bea63e"
 Roots = "f2b01f46-fcfa-551c-844a-d8ac1e96c665"
 Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
+TimeZones = "f269a46b-ccf7-5d73-abea-4c690281aa53"
diff --git a/README.md b/README.md
@@ -1,3 +1,5 @@
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.10492373.svg)](https://zenodo.org/doi/10.5281/zenodo.10492373)
+
 # Resie - simulation engine for district-scale networks of energy systems
 
 Use of Resie is described in more detail in the accompanying documentation. You can find a rendered version online at [the official readthedocs page](https://quasi-software.readthedocs.io). This document describes installation and contains useful information for developers, who wish to work with ReSiE.
@@ -22,10 +24,15 @@ Resie is released under the MIT license. You can find a copy of the license in f
 1. Exit out of the package REPL with shortcut `Ctrl+c`
 1. Exit out of the julia REPL with `exit()` or shortcut `Ctrl+d`
 
+To use profiles with a time-zone-aware datetime index beyond 2038, follow these steps:
+
+1. Copy the zip file found at `additional_files/move_to_user_julia_scratchspaces_and_unpack.zip` from the repository to your local Julia scratchspaces directory, typically located at `C:/users/user_name/.julia/scratchspaces`.
+2. Extract the content of the zip file directly into the `scratchspaces` folder. Ensure that the folder `f269a46b-ccf7-5d73-abea-4c690281aa53` is placed directly within `scratchspaces`, without any intermediate directories.
+
 If you wish to develop with this installation of ReSiE you should also perform the following inside the ReSiE root directory:
 
 1. Install the pre-commit framework: `pip install pre-commit`
-1. Install the pre-commit hooks into the ReSiE installation: `pre-commit install`
+2. Install the pre-commit hooks into the ReSiE installation: `pre-commit install`
 
 ## Usage
 

diff --git a/additional_files/move_to_user_julia_scratchspaces_and_unpack.zip b/additional_files/move_to_user_julia_scratchspaces_and_unpack.zip
diff --git a/examples/heating_and_cooling.json b/examples/heating_and_cooling.json
@@ -5,7 +5,7 @@
         "auxiliary_info_file": "./output/auxiliary_info.md",
         "sankey_plot_file": "./output/sankey_plot.html",
         "sankey_plot": "default",
-        "csv_output_keys": {},
+        "csv_output_keys": "nothing",
         "output_plot_file": "./output/output_plot.html",
         "output_plot": "all",
         "_output_plot": {
@@ -54,9 +54,11 @@
         }
     },
     "simulation_parameters": {
-        "start": 0,
-        "end": 604800,
-        "time_step_seconds": 900
+        "start": "01.01.2024 00:00",
+        "end": "07.01.2024 23:45",
+        "start_end_unit": "dd.mm.yyyy HH:MM",
+        "time_step": 15,
+        "time_step_unit": "minutes"
     },
     "order_of_operation": [
         "TST_TH_DEM_02 s_reset",
@@ -174,7 +176,8 @@
                 {
                     "name": "storage_driven",
                     "high_threshold": 0.95,
-                    "low_threshold": 0.2
+                    "low_threshold": 0.2,
+                    "storage_uac": "TST_TH_BFT_01"
                 }
             ],
             "power_th": 4500,

diff --git a/examples/multisector_district.json b/examples/multisector_district.json
@@ -114,9 +114,11 @@
         }    
     },
     "simulation_parameters": {
-        "start": 0,
-        "end": 31535100,
-        "time_step_seconds": 900
+        "start": "01.01.2024 00:00",
+        "end": "31.12.2024 23:45",
+        "start_end_unit": "dd.mm.yyyy HH:MM",
+        "time_step": 15,
+        "time_step_unit": "minutes"
     },
     "components": {
         "TST_DEM_TH_01": {

diff --git a/examples/simple_heat_pump.json b/examples/simple_heat_pump.json
@@ -4,6 +4,7 @@
         "auxiliary_info": true,
         "auxiliary_info_file": "./output/auxiliary_info.md",
         "sankey_plot": "default",
+        "csv_time_unit": "date",
         "csv_output_keys": {
 			"TST_HP_01": [
                 "m_h_w_lt1 IN",
@@ -60,9 +61,11 @@
         }    
     },
     "simulation_parameters": {
-        "start": 0,
-        "end": 603900,
-        "time_step_seconds": 900
+        "start": "01.01.2024 00:00",
+        "end": "07.01.2024 23:00",
+        "start_end_unit": "dd.mm.yyyy HH:MM",
+        "time_step": 900,
+        "time_step_unit": "seconds"
     },
     "components": {
         "TST_DEM_01": {

diff --git a/profiles/examples/district/4000hours_with_least_CO2_2022.prf b/profiles/examples/district/4000hours_with_least_CO2_2022.prf
@@ -1,5 +1,8 @@
-# time_step;900
-# is_power;true
+# data_type:                 intensive
+# time_definition:           startdate_timestamp
+# profile_start_date:        01.01.2024 00:00
+# profile_start_date_format: dd.mm.yyyy HH:MM
+# timestamp_format:          seconds 
 0	;	1
 900	;	1
 1800	;	1

diff --git a/profiles/examples/district/PV_Stuttgart_10_east_Wh_per_square_meter.prf b/profiles/examples/district/PV_Stuttgart_10_east_Wh_per_square_meter.prf
@@ -1,5 +1,8 @@
-# time_step;900
-# is_power;false
+# data_type:                 extensive
+# time_definition:           startdate_timestamp
+# profile_start_date:        01.01.2024 00:00
+# profile_start_date_format: dd.mm.yyyy HH:MM
+# timestamp_format:          seconds 
 0	;	0
 900	;	0
 1800	;	0

diff --git a/profiles/examples/district/PV_Stuttgart_10_west_Wh_per_square_meter.prf b/profiles/examples/district/PV_Stuttgart_10_west_Wh_per_square_meter.prf
@@ -1,5 +1,8 @@
-# time_step;900
-# is_power;false
+# data_type:                 extensive
+# time_definition:           startdate_timestamp
+# profile_start_date:        01.01.2024 00:00
+# profile_start_date_format: dd.mm.yyyy HH:MM
+# timestamp_format:          seconds 
 0	;	0
 900	;	0
 1800	;	0

diff --git a/profiles/examples/district/PV_Stuttgart_30_south_Wh_per_square_meter.prf b/profiles/examples/district/PV_Stuttgart_30_south_Wh_per_square_meter.prf
@@ -1,5 +1,8 @@
-# time_step;900
-# is_power;false
+# data_type:                 extensive
+# time_definition:           startdate_timestamp
+# profile_start_date:        01.01.2024 00:00
+# profile_start_date_format: dd.mm.yyyy HH:MM
+# timestamp_format:          seconds 
 0	;	0
 900	;	0
 1800	;	0

diff --git a/profiles/examples/district/ambient_temperature_TRY2015_Stuttgart.prf b/profiles/examples/district/ambient_temperature_TRY2015_Stuttgart.prf
@@ -1,5 +1,8 @@
-# time_step;900
-# is_power;true
+# data_type:                 intensive
+# time_definition:           startdate_timestamp
+# profile_start_date:        01.01.2024 00:00
+# profile_start_date_format: dd.mm.yyyy HH:MM
+# timestamp_format:          seconds 
 # data source: Deutscher Wetterdienst
 0	;	-0.3
 900	;	-0.3

diff --git a/profiles/examples/district/custom_g-function_geothermal_probe.txt b/profiles/examples/district/custom_g-function_geothermal_probe.txt
@@ -0,0 +1,40 @@
+G-Function calculated using pygfunction (method: detailed, 8 segments) for a rectangular probe field with 14 x 42 probes.
+The following parameters were used for the calculation:
+borehole depth: 25 m
+borehole spacing: 10m
+borehole radius: 0.110 m
+borehole buried depth: 0.5 m
+ground thermal diffusivity: 8.745e-7 m2/s    
+
+-- required parameter --           
+number_of_probes: 588 
+
+ -- data --
+***                
+-10.00145149; 0.11692829 
+-8.94596669; 0.41193769 
+-8.14783289; 0.72741733 
+-7.43977885; 1.04239484 
+-6.77167086; 1.35456167 
+-6.12303267; 1.6636677
+-5.48431457; 1.9692851  
+-4.8507645; 2.27040199 
+-4.2199379; 2.56536842 
+-3.59055521; 2.85187927 
+-2.96194048; 3.12710942 
+-2.33373489; 3.39514679
+-1.70574749; 3.70511843 
+-1.07787648; 4.16232799 
+-0.45006758; 4.85421046 
+0.17770816; 5.78990079 
+0.80546621; 6.85174471 
+1.43321481; 7.82223272
+2.06095838; 8.54142114 
+2.68869925; 9.00468145 
+3.31643868; 9.28612476 
+3.94417735; 9.45217429 
+4.57191561; 9.54634024 
+5.19965365; 9.59695471
+5.82739157; 9.62246368 
+6.45512943; 9.63445433 
+7.08286726; 9.63974592
diff --git a/profiles/examples/district/demand_electricity_85Living_15Business_Wh_per_square_meter.prf b/profiles/examples/district/demand_electricity_85Living_15Business_Wh_per_square_meter.prf
@@ -1,5 +1,8 @@
-# time_step;900
-# is_power;false
+# data_type:                 extensive
+# time_definition:           startdate_timestamp
+# profile_start_date:        01.01.2024 00:00
+# profile_start_date_format: dd.mm.yyyy HH:MM
+# timestamp_format:          seconds 
 0	;	0.399144707
 900	;	0.378652476
 1800	;	0.360870474

diff --git a/...es/examples/district/demand_heating_and_water_25Living_75Business_Wh_per_square_meter.prf b/...es/examples/district/demand_heating_and_water_25Living_75Business_Wh_per_square_meter.prf
@@ -1,5 +1,8 @@
-# time_step;900
-# is_power;false
+# data_type:                 extensive
+# time_definition:           startdate_timestamp
+# profile_start_date:        01.01.2024 00:00
+# profile_start_date_format: dd.mm.yyyy HH:MM
+# timestamp_format:          seconds 
 0	;	2.097296442
 900	;	2.114108251
 1800	;	2.119038111

diff --git a/...es/examples/district/demand_heating_and_water_85Living_15Business_Wh_per_square_meter.prf b/...es/examples/district/demand_heating_and_water_85Living_15Business_Wh_per_square_meter.prf
@@ -1,5 +1,8 @@
-# time_step;900
-# is_power;false
+# data_type:                 extensive
+# time_definition:           startdate_timestamp
+# profile_start_date:        01.01.2024 00:00
+# profile_start_date_format: dd.mm.yyyy HH:MM
+# timestamp_format:          seconds 
 0	;	1.865812603
 900	;	1.90987185
 1800	;	1.913829765

diff --git a/profiles/examples/district/demand_heating_and_water_temp_60_const.prf b/profiles/examples/district/demand_heating_and_water_temp_60_const.prf
@@ -1,5 +1,8 @@
-# time_step;900
-# is_power;true
+# data_type:                 intensive
+# time_definition:           startdate_timestamp
+# profile_start_date:        01.01.2024 00:00
+# profile_start_date_format: dd.mm.yyyy HH:MM
+# timestamp_format:          seconds 
 0	;	60
 900	;	60
 1800	;	60

diff --git a/profiles/examples/district/demand_heating_normalized.prf b/profiles/examples/district/demand_heating_normalized.prf
@@ -1,5 +1,8 @@
-# time_step;900
-# is_power;false
+# data_type:                 extensive
+# time_definition:           startdate_timestamp
+# profile_start_date:        01.01.2024 00:00
+# profile_start_date_format: dd.mm.yyyy HH:MM
+# timestamp_format:          seconds 
 0;0.183861802
 900;0.18820351
 1800;0.188593533

diff --git a/profiles/examples/general/dem_heat_nrg_var_hi-amp.prf b/profiles/examples/general/dem_heat_nrg_var_hi-amp.prf
@@ -1,5 +1,8 @@
-# time_step;900
-# is_power;false
+# data_type:                 extensive
+# time_definition:           startdate_timestamp
+# profile_start_date:        01.01.2024 00:00
+# profile_start_date_format: dd.mm.yyyy HH:MM
+# timestamp_format:          seconds 
 0;     0.085
 900;   0.097346221
 1800;  0.03988897

diff --git a/profiles/examples/general/dem_heat_temp_var_avg55.prf b/profiles/examples/general/dem_heat_temp_var_avg55.prf
@@ -1,5 +1,8 @@
-# time_step;900
-# is_power;true
+# data_type:                 intensive
+# time_definition:           startdate_timestamp
+# profile_start_date:        01.01.2024 00:00
+# profile_start_date_format: dd.mm.yyyy HH:MM
+# timestamp_format:          seconds 
 0;     54.46726684
 900;   54.38677618
 1800;  55.56247913

diff --git a/profiles/examples/general/src_heat_maxpow_dailyvar_lo-amp.prf b/profiles/examples/general/src_heat_maxpow_dailyvar_lo-amp.prf
@@ -1,5 +1,8 @@
-# time_step;900
-# is_power;true
+# time_definition:           startdate_timestamp
+# profile_start_date:        01.01.2024 00:00:00
+# profile_start_date_format: dd.mm.yyyy HH:MM:SS
+# timestamp_format:          seconds 
+# data_type:                 intensive
 0;0.498943359
 900;0.521484741
 1800;0.509648306