Merge branch 'main' of https://github.com/pvlib/pvanalytics into snow

.github/pull_request_template.md

@@ -1,22 +1,17 @@
-## Description
-
-*Thank you for your contribution! Please provide a brief description of the problem and the proposed solution or new feature (if not already fully described in a linked issue)*
-
-## Checklist
-
-The following items must be addressed before the code can be merged. 
-Please don't hesitate to ask for help if you are unsure of how to accomplish any of the items. 
-*You are free to remove any checklist items that do not apply or add additional items that are 
-not on this list*
+<!-- Thank you for your contribution! Please don't hesitate to ask for help if you are unsure of how to accomplish any of the items. 
+You are free to strikethrough any checklist items that do not apply or to add additional items that are not on this list. -->
 
 - [ ] Closes #xxx
-- [ ] Added new API functions to `docs/api.rst`
-- [ ] Clearly documented all new API functions with [PEP257](https://www.python.org/dev/peps/pep-0257/) and [numpydoc](https://numpydoc.readthedocs.io/en/latest/format.html) compliant docstrings
+- [ ] Added tests to cover all new or modified code.
+- [ ] Clearly documented all new API functions with [PEP257](https://www.python.org/dev/peps/pep-0257/) and [numpydoc](https://numpydoc.readthedocs.io/en/latest/format.html) compliant docstrings.
+- [ ] Added new API functions to `docs/api.rst`.
+- [ ] Non-API functions clearly documented with docstrings or comments as necessary.
 - [ ] Adds description and name entries in the appropriate "what's new" file 
-      in [`docs/whatsnew`](https://github.com/pvlib/pvanalytics/tree/master/docs/whatsnew) 
+      in [`docs/whatsnew`](https://github.com/pvlib/pvanalytics/tree/main/docs/whatsnew) 
       for all changes. Includes link to the GitHub Issue with `` :issue:`num` `` 
       or this Pull Request with `` :pull:`num` ``. Includes contributor name 
       and/or GitHub username (link with `` :ghuser:`user` ``).
-- [ ] Non-API functions clearly documented with docstrings or comments as necessary
-- [ ] Added tests to cover all new or modified code
-- [ ] Pull request is nearly complete and ready for detailed review
+- [ ] Pull request is nearly complete and ready for detailed review.
+- [ ] Maintainer: Appropriate GitHub Labels and Milestone are assigned to the Pull Request and linked Issue.
+
+<!-- Please provide a brief description of the problem and the proposed solution or new feature (if not already fully described in a linked issue): -->

.github/workflows/lint-and-test.yml

@@ -8,7 +8,7 @@ jobs:
     strategy:
       matrix:
         os: [ubuntu-latest, macos-latest, windows-latest]
-        python-version: [3.7, 3.8, 3.9, "3.10"]
+        python-version: [3.7, 3.8, 3.9, "3.10", "3.11", "3.12"]
         requirements: ['']
         include:
           - requirements: "-r requirements-min.txt"

.github/workflows/pythonpublish.yml

@@ -1,7 +1,8 @@
 name: Upload to PyPi
 on:
-  release:
-    types: [published]
+  push:
+    tags:
+    - "v*"
 
 jobs:
   deploy:

.readthedocs.yml

@@ -1,16 +1,19 @@
 version: 2
 
+build:
+  os: "ubuntu-22.04"
+  tools:
+    python: "3.7"
+
 sphinx:
   configuration: docs/conf.py
 
 formats: all
 
 python:
-  version: 3.7
   install:
     - method: pip
       path: .
       extra_requirements:
         - doc
         - optional
-

LICENSE

@@ -1,6 +1,6 @@
 MIT License
 
-Copyright (c) 2020 pvlib
+Copyright (c) 2020, 2022 Alliance for Sustainable Energy, LLC and National Technology and Engineering Solutions of Sandia, LLC
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

README.md

@@ -1,5 +1,5 @@
 ![lint and test](https://github.com/pvlib/pvanalytics/workflows/lint%20and%20test/badge.svg)
-[![Coverage Status](https://coveralls.io/repos/github/pvlib/pvanalytics/badge.svg?branch=master)](https://coveralls.io/github/pvlib/pvanalytics?branch=master)
+[![Coverage Status](https://coveralls.io/repos/github/pvlib/pvanalytics/badge.svg?branch=main)](https://coveralls.io/github/pvlib/pvanalytics?branch=main)
 [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.6110569.svg)](https://doi.org/10.5281/zenodo.6110569)
 
 

docs/api.rst

@@ -65,6 +65,17 @@ clearsky insolation for the same day.
 
    quality.irradiance.daily_insolation_limits
 
+There is function for calculating the component sum for GHI, DHI,
+and DNI, and correcting for nighttime periods. Using this function, we can
+estimate one irradiance field using the two other irradiance fields.
+This can be useful for comparison, as well as to
+calculate missing data fields. 
+
+.. autosummary::
+   :toctree: generated/
+
+   quality.irradiance.calculate_component_sum_series
+
 Gaps
 ----
 
@@ -237,12 +248,16 @@ identification.
 Daytime
 -------
 
-Functions that return a Boolean mask indicating day and night.
+Functions that relate to determining day/night periods in a time
+series, and getting sunrise and sunset times based on the day-night mask
+outputs.
 
 .. autosummary::
    :toctree: generated/
 
    features.daytime.power_or_irradiance
+   features.daytime.get_sunrise
+   features.daytime.get_sunset
 
 Shading
 -------

docs/conf.py

@@ -19,7 +19,7 @@
 # -- Project information -----------------------------------------------------
 
 project = 'PVAnalytics'
-copyright = '2020, pvlib'
+copyright = '2020-2022, pvlib'
 author = 'pvlib'
 
 
@@ -53,7 +53,23 @@
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
-html_theme = 'alabaster'
+html_theme = 'pydata_sphinx_theme'
+
+html_theme_options = {
+    "github_url": "https://github.com/pvlib/pvanalytics",
+    "icon_links": [
+        {
+            "name": "PyPI",
+            "url": "https://pypi.org/project/pvanalytics/",
+            "icon": "fab fa-python",
+        },
+    ],
+    # "use_edit_page_button": True,
+    "show_toc_level": 1,
+    "footer_items": ["copyright", "sphinx-version", "sidebar-ethical-ads"],
+    "left_sidebar_end": [],
+}
+
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
@@ -84,7 +100,6 @@
 # settings for sphinx-gallery
 sphinx_gallery_conf = {
     'examples_dirs': ['examples'],  # location of gallery scripts
-    'nested_sections': False,
     'gallery_dirs': ['generated/gallery'],  # location of generated output
     # sphinx-gallery only shows plots from plot_*.py files by default:
     'filename_pattern': re.escape(os.sep),

docs/examples/clipping/clipping.py

@@ -35,7 +35,7 @@
 data['label'] = data['label'].astype(bool)
 # This is the known frequency of the time series. You may need to infer
 # the frequency or set the frequency with your AC power time series.
-freq = "15T"
+freq = "15min"
 
 data['value_normalized'].plot()
 data.loc[data['label'], 'value_normalized'].plot(ls='', marker='o')

docs/examples/day-night-masking/day-night-masking.py

@@ -33,9 +33,10 @@
 ac_power_file = pvanalytics_dir / 'data' / 'serf_east_1min_ac_power.csv'
 data = pd.read_csv(ac_power_file, index_col=0, parse_dates=True)
 data = data.sort_index()
+
 # This is the known frequency of the time series. You may need to infer
 # the frequency or set the frequency with your AC power time series.
-freq = "1T"
+freq = "1min"
 # These are the latitude-longitude coordinates associated with the
 # SERF East system.
 latitude = 39.742

docs/examples/gaps/data-completeness.py

@@ -34,7 +34,7 @@
 pvanalytics_dir = pathlib.Path(pvanalytics.__file__).parent
 file = pvanalytics_dir / 'data' / 'ac_power_inv_2173.csv'
 data = pd.read_csv(file, index_col=0, parse_dates=True)
-data = data.asfreq("15T")
+data = data.asfreq("15min")
 
 # %%
 # Now, we use :py:func:`pvanalytics.quality.gaps.completeness_score` to get the

docs/examples/gaps/stale-data.py

@@ -34,7 +34,7 @@
 pvanalytics_dir = pathlib.Path(pvanalytics.__file__).parent
 file = pvanalytics_dir / 'data' / 'ac_power_inv_2173_stale_data.csv'
 data = pd.read_csv(file, index_col=0, parse_dates=True)
-data = data.asfreq("15T")
+data = data.asfreq("15min")
 data['value_normalized'].plot()
 data.loc[data["stale_data_mask"], "value_normalized"].plot(ls='', marker='.')
 plt.legend(labels=["AC Power", "Inserted Stale Data"])

docs/examples/irradiance-quality/clearsky-limits-irradiance.py

@@ -29,7 +29,7 @@
 pvanalytics_dir = pathlib.Path(pvanalytics.__file__).parent
 rmis_file = pvanalytics_dir / 'data' / 'irradiance_RMIS_NREL.csv'
 data = pd.read_csv(rmis_file, index_col=0, parse_dates=True)
-freq = '5T'
+freq = '5min'
 # Make the datetime index tz-aware.
 data.index = data.index.tz_localize("Etc/GMT+7")
 

docs/examples/irradiance-quality/component-sum-irradiance.py

@@ -0,0 +1,119 @@
+"""
+Component Sum Equations for Irradiance Data
+===========================================
+
+Estimate GHI, DHI, and DNI using the component sum equations, with
+nighttime corrections.
+"""
+
+# %%
+# Estimating GHI, DHI, and DNI using the component sum equations is useful
+# if the associated field is missing, or as a comparison to an existing
+# physical data stream.
+
+import pvanalytics
+from pvanalytics.quality.irradiance import calculate_component_sum_series
+import pvlib
+import matplotlib.pyplot as plt
+import pandas as pd
+import pathlib
+
+# %%
+# First, read in data from the RMIS NREL system. This data set contains
+# 5-minute right-aligned data. It includes POA, GHI,
+# DNI, DHI, and GNI measurements.
+
+pvanalytics_dir = pathlib.Path(pvanalytics.__file__).parent
+rmis_file = pvanalytics_dir / 'data' / 'irradiance_RMIS_NREL.csv'
+data = pd.read_csv(rmis_file, index_col=0, parse_dates=True)
+
+# %%
+# Now generate solar zenith estimates for the location,
+# based on the data's time zone and site latitude-longitude
+# coordinates. This is done using the
+# :py:func:`pvlib.solarposition.get_solarposition` function.
+latitude = 39.742
+longitude = -105.18
+time_zone = "Etc/GMT+7"
+data = data.tz_localize(time_zone)
+solar_position = pvlib.solarposition.get_solarposition(data.index,
+                                                       latitude,
+                                                       longitude)
+
+
+# %%
+# Get the clearsky DNI values associated with the current location, using
+# the :py:func:`pvlib.solarposition.get_solarposition` function. These clearsky
+# values are used to calculate DNI data.
+site = pvlib.location.Location(latitude, longitude, tz=time_zone)
+clearsky = site.get_clearsky(data.index)
+
+# %%
+# Use :py:func:`pvanalytics.quality.irradiance.calcuate_ghi_component`
+# to estimate GHI measurements using DHI and DNI measurements
+
+component_sum_ghi = calculate_component_sum_series(
+    solar_zenith=solar_position['zenith'],
+    dhi=data['irradiance_dhi__7983'],
+    dni=data['irradiance_dni__7982'],
+    zenith_limit=90,
+    fill_night_value='equation')
+
+# %%
+# Plot the 'irradiance_ghi__7981' data stream against the estimated component
+# sum GHI, for comparison
+data['irradiance_ghi__7981'].plot()
+component_sum_ghi.plot()
+plt.legend(labels=["RMIS GHI", "Component Sum GHI"],
+           loc="upper left")
+plt.xlabel("Date")
+plt.ylabel("GHI (W/m^2)")
+plt.tight_layout()
+plt.show()
+
+# %%
+# Use :py:func:`pvanalytics.quality.irradiance.calcuate_dhi_component`
+# to estimate DHI measurements using GHI and DNI measurements
+
+component_sum_dhi = calculate_component_sum_series(
+    solar_zenith=solar_position['zenith'],
+    dni=data['irradiance_dni__7982'],
+    ghi=data['irradiance_ghi__7981'],
+    zenith_limit=90,
+    fill_night_value='equation')
+
+# %%
+# Plot the 'irradiance_dhi__7983' data stream against the estimated component
+# sum GHI, for comparison
+data['irradiance_dhi__7983'].plot()
+component_sum_dhi.plot()
+plt.legend(labels=["RMIS DHI", "Component Sum DHI"],
+           loc="upper left")
+plt.xlabel("Date")
+plt.ylabel("GHI (W/m^2)")
+plt.tight_layout()
+plt.show()
+
+# %%
+# Use :py:func:`pvanalytics.quality.irradiance.calcuate_dni_component`
+# to estimate DNI measurements using GHI and DHI measurements
+
+component_sum_dni = calculate_component_sum_series(
+    solar_zenith=solar_position['zenith'],
+    dhi=data['irradiance_dhi__7983'],
+    ghi=data['irradiance_ghi__7981'],
+    dni_clear=clearsky['dni'],
+    zenith_limit=90,
+    fill_night_value='equation')
+
+# %%
+# Plot the 'irradiance_dni__7982' data stream against the estimated component
+# sum GHI, for comparison
+data['irradiance_dni__7982'].plot()
+component_sum_dni.plot()
+plt.legend(labels=["RMIS DNI", "Component Sum DNI"],
+           loc="upper left")
+plt.xlabel("Date")
+plt.ylabel("DNI (W/m^2)")
+plt.tight_layout()
+plt.show()

docs/examples/metrics/variability-index.py

@@ -52,7 +52,7 @@
 # an hourly frequency.
 variability_index_series = variability_index(data['irradiance_ghi__7981'],
                                              clearsky['ghi'],
-                                             freq='1H')
+                                             freq='1h')
 
 # %%
 # Plot the calculated VI against the underlying GHI measurements, for the

docs/examples/pvfleets-qa-pipeline/README.rst

@@ -0,0 +1,5 @@
+PVFleets QA Examples
+--------------------
+
+These examples highlight the QA processes for temperature, power and irradiance data streams that are used in the NREL
+PV Fleet Performance Data Initiative (https://www.nrel.gov/pv/fleet-performance-data-initiative.html).