Add Documentation 2 (#334)

Add documentation that covers:

- configuration
- prediction

Add Python scripts that support automatic documentation generation for
configuration data defined with pydantic.

Adapt EOS configuration to provide more methods for REST API and
automatic documentation generation.

Adapt REST API to allow for EOS configuration file load and save.
Sort REST API on generation of openapi markdown for docs.

Move logutil to core/logging to allow configuration of logging by standard config.

Make Akkudoktor predictions always start extraction of prediction data at start of day.
Previously extraction started at actual hour. This is to support the code that assumes
prediction data to start at start of day.

Signed-off-by: Bobby Noelte <[email protected]>

docs/akkudoktoreos/configuration.md

@@ -0,0 +1,89 @@
+% SPDX-License-Identifier: Apache-2.0
+
+# Configuration
+
+The configuration controls all aspects of EOS: optimization, prediction, measurement, and energy
+management.
+
+## Storing Configuration
+
+EOS stores configuration data in a **key-value store**, where a `configuration key` refers to the
+unique identifier used to store and retrieve specific configuration data. Note that the key-value
+store is memory-based, meaning all stored data will be lost upon restarting the EOS REST server if
+not saved to the `EOS configuration file`.
+
+Some `configuration keys` are read-only and cannot be altered. These keys are either set up by other
+means, such as environment variables, or determined from other information.
+
+Several endpoints of the EOS REST server allow for the management and retrieval of configuration
+data.
+
+### Save Configuration File
+
+Use endpoint `PUT /v1/config/file` to save the current configuration to the
+`EOS configuration file`.
+
+### Load Configuration File
+
+Use endpoint `POST /v1/config/update` to update the configuration from the `EOS configuration file`.
+
+## Configuration Sources and Priorities
+
+The configuration sources and their priorities are as follows:
+
+1. **Settings**: Provided during runtime by the REST interface
+2. **Environment Variables**: Defined at startup of the REST server and during runtime
+3. **EOS Configuration File**: Read at startup of the REST server and on request
+4. **Default Values**
+
+### Settings
+
+Settings are sets of configuration data that take precedence over all other configuration data from
+different sources. Note that settings are not persistent. To make the current configuration with the
+current settings persistent, save the configuration to the `EOS configuration file`.
+
+Use the following endpoints to change the current configuration settings:
+
+- `PUT /v1/config`: Replaces the entire configuration settings.
+- `PUT /v1/config/value`: Sets a specific configuration option.
+
+### Environment Variables
+
+All `configuration keys` can be set by environment variables with the same name. EOS recognizes the
+following special environment variables:
+
+- `EOS_CONFIG_DIR`: The directory to search for an EOS configuration file.
+- `EOS_DIR`: The directory used by EOS for data, which will also be searched for an EOS
+             configuration file.
+- `EOS_LOGGING_LEVEL`: The logging level to use in EOS.
+
+### EOS Configuration File
+
+The EOS configuration file provides persistent storage for configuration data. It can be modified
+directly or through the REST interface.
+
+If you do not have a configuration file, it will be automatically created on the first startup of
+the REST server in a system-dependent location.
+
+To determine the location of the configuration file used by EOS, ask the REST server. The endpoint
+`GET /v1/config` provides the `config_file_path` configuration key.
+
+EOS searches for the configuration file in the following order:
+
+1. The directory specified by the `EOS_CONFIG_DIR` environment variable
+2. The directory specified by the `EOS_DIR` environment variable
+3. A platform-specific default directory for EOS
+4. The current working directory
+
+The first available configuration file found in these directories is loaded. If no configuration
+file is found, a default configuration file is created in the platform-specific default directory,
+and default settings are loaded into it.
+
+### Default Values
+
+Some of the `configuration keys` have default values by definition. For most of the
+`configuration keys` the default value is just `None`, which means no default value.
+
+```{eval-sh}
+./scripts/generate_config_md.py | ./scripts/extract_markdown.py --input-stdin --heading-level 1
+```

docs/akkudoktoreos/measurement.md

@@ -30,7 +30,7 @@ The measurement data must be or is provided in one of the following formats:
 
 A dictionary with the following structure:
 
-```JSON
+```python
     {
         "start_datetime": "2024-01-01 00:00:00",
         "interval": "1 Hour",

docs/akkudoktoreos/prediction.md

@@ -21,7 +21,7 @@ data is lost on re-start of the EOS REST server.
 Most predictions can be sourced from various providers. The specific provider to use is configured
 in the EOS configuration. For example:
 
-```plaintext
+```python
 weather_provider = "ClearOutside"
 ```
 
@@ -43,7 +43,7 @@ The prediction data must be provided in one of the following formats:
 
 A dictionary with the following structure:
 
-```JSON
+```python
     {
         "start_datetime": "2024-01-01 00:00:00",
         "interval": "1 Hour",
@@ -214,6 +214,67 @@ Configuration options:
 - `pvforecastimport_file_path`: Path to the file to import PV forecast data from.
 - `pvforecastimport_json`: JSON string, dictionary of PV forecast value lists.
 
+------
+
+Some of the configuration options directly follow the [PVGIS](https://joint-research-centre.ec.europa.eu/photovoltaic-geographical-information-system-pvgis/getting-started-pvgis/pvgis-user-manual_en) nomenclature.
+
+Detailed definitions taken from **PVGIS**:
+
+- `pvforecast<0..5>_pvtechchoice`
+
+The performance of PV modules depends on the temperature and on the solar irradiance, but the exact dependence varies between different types of PV modules. At the moment we can estimate the losses due to temperature and irradiance effects for the following types of modules: crystalline silicon cells; thin film modules made from CIS or CIGS and thin film modules made from Cadmium Telluride (CdTe).
+
+For other technologies (especially various amorphous technologies), this correction cannot be calculated here. If you choose one of the first three options here the calculation of performance will take into account the temperature dependence of the performance of the chosen technology. If you choose the other option (other/unknown), the calculation will assume a loss of 8% of power due to temperature effects (a generic value which has found to be reasonable for temperate climates).
+
+PV power output also depends on the spectrum of the solar radiation. PVGIS can calculate how the variations of the spectrum of sunlight affects the overall energy production from a PV system. At the moment this calculation can be done for crystalline silicon and CdTe modules. Note that this calculation is not yet available when using the NSRDB solar radiation database.
+
+- `pvforecast<0..5>_peakpower`
+
+This is the power that the manufacturer declares that the PV array can produce under standard test conditions (STC), which are a constant 1000W of solar irradiation per square meter in the plane of the array, at an array temperature of 25°C. The peak power should be entered in kilowatt-peak (kWp). If you do not know the declared peak power of your modules but instead know the area of the modules and the declared conversion efficiency (in percent), you can calculate the peak power as power = area * efficiency / 100.
+
+Bifacial modules: PVGIS doesn't make specific calculations for bifacial modules at present. Users who wish to explore the possible benefits of this technology can input the power value for Bifacial Nameplate Irradiance. This can also be can also be estimated from the front side peak power P_STC value and the bifaciality factor, φ (if reported in the module data sheet) as: P_BNPI  = P_STC * (1 + φ * 0.135). NB this bifacial approach  is not appropriate for BAPV or BIPV installations or for modules mounting on a N-S axis i.e. facing E-W.
+
+- `pvforecast<0..5>_loss`
+
+The estimated system losses are all the losses in the system, which cause the power actually delivered to the electricity grid to be lower than the power produced by the PV modules. There are several causes for this loss, such as losses in cables, power inverters, dirt (sometimes snow) on the modules and so on. Over the years the modules also tend to lose a bit of their power, so the average yearly output over the lifetime of the system will be a few percent lower than the output in the first years.
+
+We have given a default value of 14% for the overall losses. If you have a good idea that your value will be different (maybe due to a really high-efficiency inverter) you may reduce this value a little.
+
+- `pvforecast<0..5>_mountingplace`
+
+For fixed (non-tracking) systems, the way the modules are mounted will have an influence on the temperature of the module, which in turn affects the efficiency. Experiments have shown that if the movement of air behind the modules is restricted, the modules can get considerably hotter (up to 15°C at 1000W/m2 of sunlight).
+
+In PVGIS there are two possibilities: free-standing, meaning that the modules are mounted on a rack with air flowing freely behind the modules; and building- integrated, which means that the modules are completely built into the structure of the wall or roof of a building, with no air movement behind the modules.
+
+Some types of mounting are in between these two extremes, for instance if the modules are mounted on a roof with curved roof tiles, allowing air to move behind the modules. In such cases, the performance will be somewhere between the results of the two calculations that are possible here.
+
+- `pvforecast<0..5>_userhorizon`
+
+Elevation of horizon in degrees, at equally spaced azimuth clockwise from north. In the user horizon
+data each number represents the horizon height in degrees in a certain compass direction around the
+point of interest. The horizon heights should be given in a clockwise direction starting at North;
+that is, from North, going to East, South, West, and back to North. The values are assumed to
+represent equal angular distance around the horizon. For instance, if you have 36 values, the first
+point is due north, the next is 10 degrees east of north, and so on, until the last point, 10
+degrees west of north.
+
+------
+
+Most of the configuration options are in line with the [PVLib](https://pvlib-python.readthedocs.io/en/stable/_modules/pvlib/iotools/pvgis.html) definition for PVGIS data.
+
+Detailed definitions from **PVLib** for PVGIS data.
+
+- `pvforecast<0..5>_surface_tilt`:
+
+Tilt angle from horizontal plane.
+
+- `pvforecast<0..5>_surface_azimuth`
+
+Orientation (azimuth angle) of the (fixed) plane. Clockwise from north (north=0, east=90, south=180,
+west=270). This is offset 180 degrees from the convention used by PVGIS.
+
+------
+
 ### PVForecastAkkudoktor Provider
 
 The `PVForecastAkkudoktor` provider retrieves the PV power forecast data directly from

docs/index.md

@@ -14,6 +14,7 @@ welcome.md
 akkudoktoreos/about.md
 develop/getting_started.md
 develop/CONTRIBUTING.md
+akkudoktoreos/configuration.md
 akkudoktoreos/prediction.md
 akkudoktoreos/measurement.md
 akkudoktoreos/serverapi.md