Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Features/#323 egon data link #328

Merged
merged 100 commits into from
Mar 31, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
100 commits
Select commit Hold shift + click to select a range
67449d2
added basic database engine to read local or remote (per ssh) database
khelfen Oct 7, 2022
3ca8908
added egon-data db read-in for SimBEV profiles
khelfen Oct 13, 2022
2c71864
Merge branch 'dev' into features/#323-egon-data-link
khelfen Oct 14, 2022
b3c8721
added egon-data db read-in for TracBEV data
khelfen Oct 14, 2022
18f266f
small bug fix
khelfen Oct 18, 2022
4b5e15b
added how-to
khelfen Oct 18, 2022
81d1bd8
minor fix
khelfen Oct 18, 2022
d38a725
Merge branch 'dev' of github.com:openego/eDisGo into features/#323-eg…
khelfen Oct 31, 2022
38cf468
changed queries to use saio
khelfen Nov 1, 2022
b2ef13b
minor change
khelfen Nov 2, 2022
c0c627d
added basic dsm structure and import
khelfen Nov 2, 2022
222ca6d
added basic dsm structure and import
khelfen Nov 2, 2022
eedc517
rollback changes
khelfen Nov 3, 2022
3ed88de
added wrapper function
khelfen Nov 3, 2022
20cd291
changed structure
khelfen Nov 3, 2022
fd3e63d
minor change
khelfen Nov 3, 2022
2c4ab35
Features/#323 dsm class (#327)
khelfen Nov 3, 2022
8535a25
added dsm to and from csv
khelfen Nov 3, 2022
61e1733
Merge branch 'features/#323-egon-data-link' of github.com:openego/eDi…
khelfen Nov 3, 2022
1e9ed6f
Merge branch 'features/#323-dsm-class' into features/#323-egon-data-link
khelfen Nov 3, 2022
ae75f7b
added pv rooftop import
khelfen Nov 4, 2022
5d53a68
restructured;
khelfen Nov 4, 2022
9da7e02
type hinting
khelfen Nov 4, 2022
a754bd0
Merge branch 'dev' into features/#323-egon-data-link
khelfen Nov 4, 2022
9238e61
import only charging infrastructure within grid district
khelfen Nov 7, 2022
92bf587
preprocess data
khelfen Nov 7, 2022
dbc0d62
determine intersecting weather cells
khelfen Nov 9, 2022
9cd178a
Merge branch 'dev' into features/#323-egon-data-link
khelfen Nov 10, 2022
ad3f350
import generator time series from egon data db
khelfen Nov 11, 2022
5a974f1
intergrate home batteries
khelfen Nov 11, 2022
0077c4f
determine the closest MV-LV station when importing lv generators
khelfen Nov 17, 2022
8372682
Merge branch 'dev' of github.com:openego/eDisGo into features/#323-eg…
khelfen Dec 22, 2022
292645b
Merge branch 'dev' of github.com:openego/eDisGo into features/#323-eg…
khelfen Jan 13, 2023
fa9b812
Merge branch 'dev' into features/#323-egon-data-link
khelfen Feb 3, 2023
47fcc11
Merge branch 'egon_dev' into features/#323-egon-data-link
birgits Mar 10, 2023
cf9b853
Remove stuff from autodoc
birgits Mar 10, 2023
6713d97
Remove file
birgits Mar 10, 2023
8968102
Use db module instead of egon_data_import
birgits Mar 10, 2023
c26908d
Remove unnecessary lines from doc rst files
birgits Mar 10, 2023
0ba4d8b
Remove old egon_data_import file
birgits Mar 10, 2023
7b9025e
Try fixing rtd compilation
birgits Mar 10, 2023
9f8b993
Adapt functions to get EV data from database and add tests
birgits Mar 12, 2023
754587d
Minor docstring changes
birgits Mar 12, 2023
25cc09e
Allow choosing whether to import all charging processes or only those…
birgits Mar 12, 2023
3926243
Adapt EDisGo.import_electromobility and its tests
birgits Mar 12, 2023
efa1bfb
Move assure_minimum_potential_charging_parks out of functions reading…
birgits Mar 12, 2023
52599ce
Bug fix concatenating geodataframes and add test
birgits Mar 12, 2023
8ce2689
Bug fix use new function name
birgits Mar 12, 2023
c19585d
Bug fix apply intersect in database
birgits Mar 12, 2023
c63dec3
Add test for getting weather cells from egon_data
birgits Mar 12, 2023
c6d0ebc
Bug fix handle gens without a geom
birgits Mar 12, 2023
a83f900
Fix test adapt to changed generator import
birgits Mar 12, 2023
8df43dc
Fix tests - adapt to new function names
birgits Mar 12, 2023
0eaee53
Set legacy_grids attribute in EDisGo to differentiate oedb data sources
birgits Mar 12, 2023
8ef5429
Adapt heat pump import to use helper functions from db module
birgits Mar 12, 2023
fb3f0c0
Adapt connect functions to allow for connection of storage units
birgits Mar 13, 2023
f476523
Implement getting home battery data from database and integrate into …
birgits Mar 13, 2023
c4dacb3
Adapt EDisGo.import_home_batteries
birgits Mar 13, 2023
e4d80c9
Adapt test to make sure fresh test dataframe is always used
birgits Mar 13, 2023
5e4b5bf
Add functions to get industrial and residential electricity load prof…
birgits Mar 15, 2023
968d3cb
Add function to get all electricity demand profiles from database
birgits Mar 15, 2023
f7840f0
Minor docstring changes
birgits Mar 15, 2023
f11b6c6
Change logging
birgits Mar 16, 2023
211c567
Make more robust in case index name is not index
birgits Mar 16, 2023
312cc1b
Add option to get electricity time series of conventional loads to ED…
birgits Mar 16, 2023
8642cb9
Rename retail to cts
birgits Mar 16, 2023
e4dd4b5
Adapt function to get feedin time series from egon_data
birgits Mar 16, 2023
0a9a8ea
Adapt functions to use timeindex helper function
birgits Mar 16, 2023
667758e
Change setting of time index of data imported from oedb
birgits Mar 16, 2023
5ffdf58
Add tests for getting feed-in time series from oedb
birgits Mar 16, 2023
75c3aba
Adapt docstring
birgits Mar 16, 2023
1861de2
Minor formatting changes
birgits Mar 16, 2023
83d4c8a
Fix module name
birgits Mar 16, 2023
b11fad2
Simplify import
birgits Mar 16, 2023
71842c5
Try fixing failing tests
birgits Mar 16, 2023
b5c07d4
Fix dtype when time series is imported from database as list
birgits Mar 17, 2023
eb74a31
Fix default value
birgits Mar 18, 2023
ae00203
Adapt functions to import new generators and add tests
birgits Mar 18, 2023
49fdc47
Remove links in first line of docstring to fix sphinx API doc
birgits Mar 18, 2023
466b50a
Minor doc change
birgits Mar 18, 2023
e44aa7a
Add test with previously integrated new PV rooftop plants to assert t…
birgits Mar 19, 2023
041f54a
Minor docstring fixes
birgits Mar 21, 2023
a583e7d
Adapt DSM class and add tests
birgits Mar 22, 2023
af993e9
Minor docstring changes
birgits Mar 22, 2023
2b0425a
Add DSM to EDisGo class and add tests
birgits Mar 22, 2023
53a605d
Refactor DSM import from database
birgits Mar 29, 2023
c23486c
Add import of DSM data to edisgo
birgits Mar 29, 2023
3777dfc
Change logging of generator import
birgits Mar 29, 2023
f97cf9e
Merge branch 'egon_dev' into features/#323-egon-data-link
birgits Mar 29, 2023
f7156a2
Import source_id of generators in ding0 import
birgits Mar 30, 2023
d33e851
Match pv rooftop plants based on source ID and add test
birgits Mar 30, 2023
5478484
Adapt tests to new test grid
birgits Mar 30, 2023
7fbe0eb
Change test grid
birgits Mar 30, 2023
4b2a893
Remove dash restrictions
birgits Mar 30, 2023
a38a058
Add check for internal links to dev notes
birgits Mar 31, 2023
383032e
Add changes to whatsnew
birgits Mar 31, 2023
d0bb863
Fix internal links
birgits Mar 31, 2023
d4a3201
Move pyyaml to keep alphabetical order
birgits Mar 31, 2023
c599b53
Adapt docstring
birgits Mar 31, 2023
b2547c8
Remove old option for DSM import to import pu timeseries
birgits Mar 31, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ repos:
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/psf/black
rev: 22.8.0
rev: 22.10.0
hooks:
- id: black
- repo: https://github.com/pycqa/flake8
Expand All @@ -19,14 +19,14 @@ repos:
- id: isort
name: isort (python)
- repo: https://github.com/asottile/pyupgrade
rev: v2.38.0
rev: v3.2.0
hooks:
- id: pyupgrade
#- repo: https://github.com/pycqa/pylint
# rev: pylint-2.6.0
# hooks:
# - id: pylint
- repo: https://github.com/kynan/nbstripout
rev: 0.6.0
rev: 0.6.1
hooks:
- id: nbstripout
6 changes: 6 additions & 0 deletions doc/dev_notes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -82,3 +82,9 @@ To manually check if external links in the documentation work, you can run the f

sphinx-build ./doc/ -b linkcheck -d _build/doctrees _build/html

Internal links can be checked adding -n option when building the documentation. This will
also raise warnings for type hinting, so it is a bit confusing, but can still be helpful.

.. code-block:: bash

sphinx-build -n -E -a -b html ./doc/ <outputdir>
2 changes: 1 addition & 1 deletion doc/quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ time series:
# load time series (scaled by annual demand)
timeseries_load = pd.DataFrame(
{"residential": [0.0001] * len(timeindex),
"retail": [0.0002] * len(timeindex),
"cts": [0.0002] * len(timeindex),
"industrial": [0.00015] * len(timeindex),
"agricultural": [0.00005] * len(timeindex)
},
Expand Down
1 change: 1 addition & 0 deletions doc/whatsnew/v0-3-0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Release date: <month> <day>, <year>
Changes
-------

* Added functionalities to obtain electromobility, DSM, storage and electricity timeseries data from oedb `#328 <https://github.com/openego/eDisGo/pull/328>`_
* Added functionalities to obtain heat pump data from oedb `#324 <https://github.com/openego/eDisGo/pull/324>`_
* Added functionality to resample and check integrity of flexibility bands `#341 <https://github.com/openego/eDisGo/pull/341>`_
* Added function to sort buses in lines dataframe such that bus0 is always the upstream bus `#335 <https://github.com/openego/eDisGo/pull/335>`_
Expand Down
751 changes: 531 additions & 220 deletions edisgo/edisgo.py

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions edisgo/flex_opt/costs.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ def grid_expansion_costs(edisgo_obj, without_generator_import=False):

Attributes
----------
edisgo_obj : :class:`~.self.edisgo.EDisGo`
edisgo_obj : :class:`~.EDisGo`
without_generator_import : bool
If True excludes lines that were added in the generator import to
connect new generators to the topology from calculation of topology expansion
Expand Down Expand Up @@ -213,7 +213,7 @@ def line_expansion_costs(edisgo_obj, lines_names=None):

Parameters
-----------
edisgo_obj : :class:`~.edisgo.EDisGo`
edisgo_obj : :class:`~.EDisGo`
eDisGo object
lines_names: None or list(str)
List of names of lines to return cost information for. If None, it is returned
Expand Down
6 changes: 3 additions & 3 deletions edisgo/flex_opt/reinforce_grid.py
Original file line number Diff line number Diff line change
Expand Up @@ -45,13 +45,13 @@ def reinforce_grid(
conducted and therefore which time steps to consider when checking
for over-loading and over-voltage issues.
It defaults to None in which case all timesteps in
timeseries.timeindex (see :class:`~.network.network.TimeSeries`) are
timeseries.timeindex (see :class:`~.network.timeseries.TimeSeries`) are
used.
Possible options are:

* None
Time steps in timeseries.timeindex (see
:class:`~.network.network.TimeSeries`) are used.
:class:`~.network.timeseries.TimeSeries`) are used.
* 'snapshot_analysis'
Reinforcement is conducted for two worst-case snapshots. See
:meth:`edisgo.tools.tools.select_worstcase_snapshots()` for further
Expand Down Expand Up @@ -96,7 +96,7 @@ def reinforce_grid(

Returns
-------
:class:`~.network.network.Results`
:class:`~.network.results.Results`
Returns the Results object holding network expansion costs, equipment
changes, etc.

Expand Down
22 changes: 18 additions & 4 deletions edisgo/io/db.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ def session_scope_egon_data(engine: Engine):

def sql_grid_geom(edisgo_obj: EDisGo) -> Geometry:
return func.ST_GeomFromText(
str(edisgo_obj.topology.grid_district["geom"]),
edisgo_obj.topology.grid_district["geom"].wkt,
edisgo_obj.topology.grid_district["srid"],
)

Expand All @@ -215,14 +215,28 @@ def get_srid_of_db_table(session: Session, geom_col: InstrumentedAttribute) -> i
return pd.read_sql(sql=query.statement, con=query.session.bind).iat[0, 0]


def sql_within(geom_col: InstrumentedAttribute, geom_shape: Geometry, srid: int):
def sql_within(geom_a: Geometry, geom_b: Geometry, srid: int):
"""
Checks if geometry a is completely within geometry b.

Parameters
----------
geom_a : Geometry
Geometry within `geom_b`.
geom_b : Geometry
Geometry containing `geom_a`.
srid : int
SRID geometries are transformed to in order to use the same SRID for both
geometries.

"""
return func.ST_Within(
func.ST_Transform(
geom_col,
geom_a,
srid,
),
func.ST_Transform(
geom_shape,
geom_b,
srid,
),
)
Expand Down
2 changes: 2 additions & 0 deletions edisgo/io/ding0_import.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,8 +93,10 @@ def sort_hvmv_transformer_buses(transformers_df):
# set up columns that are added in new ding0 version
grid.loads["building_id"] = None
grid.loads["number_households"] = None
grid.generators["source_id"] = None
else:
edisgo_obj.topology.buses_df["in_building"] = False
grid.generators = grid.generators.rename(columns={"gens_id": "source_id"})
edisgo_obj.topology.loads_df = grid.loads[edisgo_obj.topology.loads_df.columns]
# drop slack generator from generators
slack = grid.generators.loc[grid.generators.control == "Slack"].index
Expand Down
Loading