Calculate Grid Gross Loss

[grgmiller]

This PR is in response to #80 to add grid gross loss calculations to the project

The brief summary:

• downloads raw EIA-861 files since the tables we need are not yet in pudl
• adds a new module src.grid_gross_loss where GGL is calculated
• The method is "inspired" by egrid, but the actual implementation turned out to be quite different, for two main reasons:
  • eGRID uses the EIA's State Electricity Profiles as its source for this calculation, which are derived from EIA-861 data. We go straight to the raw EIA-861 data so that we can do our own aggregations
  • eGRID calculates GGL at the interconnect level. This code introduces a new method for calculating GGL at the BA level
• For now, I am just exporting the GGL values to the outputs folder, as I'm not quite sure where they should live in the results yet.

How does eGRID Calculate GGL?

• eGRID calculates GGL based on the EIA's State Electricity Profiles, specifically using Table 10, which includes calculated supply and disposition for each state. These values are calculated by EIA using the data reported in EIA-861 (which is reported by utility).
• The formula for GGL is GGL = Losses / (Disposition - Exports - Direct Use), where:
  • Disposition is equivalent to the total supply, including sales, losses, and self-use
  • Exports is electricity exported to other states. eGRID notes that it excludes exports due to a "slight double counting" issue, but I'm not really clear how exactly this causes an issue.
  • Direct use is the total amount of electricity consumed by the utility itself, and not sold for resale. eGRID interprets direct use to mean station use (ie the difference between gross and net generation), noting in their documentation that "direct use electricity is not transmitted through the grid and therefore does not have the potential to be lost". However, EIA notes in the footnotes to Table 10 that "Direct use is exclusive of station use." Thus I believe this to be a source of error in eGRID's method.
  • Losses are essentially total supply minus total sales or consumption by end uses
• Because state boundaries do not correspond exactly with interconnect boundaries, they have to allocate the data from the state profiles in proportion to the net generation reported in each state/interconnect in EIA-923.

How does this PR calculate GGL?

• My goal was to calculate GGL values for each BA, rather than at the interconnect level. Disposition data is reported for each utility in the EIA-861 "Operational Data" table. However, utilities sometimes serve load in multiple balancing authorities, so I needed a way to allocate the disposition data from each utility to each BA.
• Utilities separately report their total retail sales (which is a component of total disposition) in each BA in the EIA-861 "Sales_Ult_Cust" tables. I use this data to calculate which fraction of each utility's total retail sales are in each BA, and then distribute the utility disposition data to each BA based on this fraction.
• The formula that I use to calculate GGL is GGL = Losses / Disposition. I don't subtract out direct use (called "consumed by respondent without charge" in EIA-861) due to the note above, and I don't subtract out exports, because there is not a clear reason upon examining the data why this should be exlcluded from total disposition (although this is something that we should look into). I'm wondering if the total state export data in the state electricity profile is different from the "exchange energy delivered" data in EIA-861, and thus there might be a reason to treat it differently.
• This method results in GGL values between 0% and 10% for individual BAs, which seems reasonable.
• After this allocation, the total disposition and losses (for the whole country) still matches the total disposition and losses reported in the raw EIA-861 form.

Why are our calculations so different from eGRID?

• eGRID reports a national average GGL of 5.3%, with a range of 5.2%-5.6% across all interconnects. By comparison, the national average GGL we calculate is 2.7% with a range of 0% to 9.7%. It is concerning that the national average value that we calculate is about half of that that eGRID calculates.
• Even if I use the raw EIA-861 disposition data and try to calculate a national GGL using eGRID's formula (GGL = Total Energy Losses / (Total Disposition - Exchange Energy Delivered - Consumed by Respondent without Charge)) I still only arrive at a national average value of 2.8%.
• My suspicion is that there is double counting somewhere in one of our approaches
  • it is possible in my approach that I need to somehow adjust total disposition or losses. This would require either doubling total losses or halving total disposition. Total disposition in 2020 was about 7.5 billion MWh, which means to get a loss of 5%, I'd need to decrease disposition by about 3.8 billion MWh. The only other values in the disposition table that are close to this are retail sales (3.6 billion) sales for resale (3.6 billion). This makes me wonder if maybe the disposition value in the state profile excludes sales for resale.
  • After taking a look at the state profiles, it looks like these might exclude sales for resale in the total disposition numbers, while EIA-861 includes these resales in total disposition. The instructions for EIA-861 note that sales for resale includes "sales for resale to power marketers (reported separately in previous years), full and partial requirements customers, firm power customers, and non-firm customers". the EIA-861 table does include disposition data for retail power marketers (and wholesale power marketers), so I am wondering if there is some double-counting of disposition.

TODO

• Figure out where in the results we want the GGL file to live
• Figure out how GGL should be integrated into our consumed emissions calculations. One option would be to use it to adjust the consumed emissions values that come out of gridemissions (like consumed_ef / (1 - GGL)), or if these values could be integrated into the consumed emissions calculation itself (see Account for transmission losses / Grid Gross Loss #80)
• Set up a conversation with EIA to discuss the interpretation of the EIA-861 disposition data and whether exports should be excluded.

[grgmiller]

So it seems like using GGL = Losses / (Disposition - Resales) leads to a national average GGL value of 5.4%, but it also leads to really high values in certain BAs - e.g. CAISO now has a GGL of 17.4%, which seems unreasonably high. This suggests that perhaps there's something else going on with the numbers.