Read on Twitter
As we root for everyone to make it through the extremely challenging conditions today, there's clearly a temptation in my feed to figure out what the last few days means for ERCOT's market design. Warning: I would have made this thread shorter if I had more time.
I'll preface this with the note that based on the available information, we would currently be seeing rolling blackouts no matter what design choices had been made for resource adequacy. Some more on the current situation from @gilbeaq here: https://twitter.com/gilbeaq/status/1361311892132794370?s=20
Obviously a lot of people will try to use this event as a referendum on energy-only markets vs capacity markets. Here I'm instead focusing on the question: given that we have an energy only market, how do we know it's working?
As in summer 2019, the question is something like: is this just the kind of tail event you have to live with in an "economically optimal" system, or is it evidence that the system isn't delivering the desired "economically optimal" results?
In summer 2019, you could tell a plausible story that the high prices and emergency declarations were just a normal oscillation for an energy-only market operating about as hoped.
It remains to be seen how much load will need to be shed today, but it seems to me that this story will be much more difficult to maintain.
So here are some reasons the story might be wrong. This will take the form of a peer review of this report: http://www.ercot.com/content/wcm/lists/219844/2020_ERCOT_Reserve_Margin_Study_Report_FINAL_1-15-2021.pdf, which I take to be the basic model of how ERCOT is "supposed" to work.
If the market design is working, the report estimates that ERCOT will have about 4,500 MWh of unserved energy per year (expected unserved energy (EUE), my preferred reliability metric)
ERCOT reports that "about 10,500 MW of customer load was shed at the highest point." If they stay at anything close to that level for the entire AM, or into the PM, obviously that will shatter several years' worth of the EUE estimate.
A more recent estimate from @MikeZaccardi puts the current value at closer to 20 GW, but obviously the totals for the day won't be available for a while. https://twitter.com/MikeZaccardi/status/1361331538726711306?s=20
Comment #1: The report calculates EUE incorrectly. This is the EUE if the system maintains equilibrium capacity at all times. But as they make clear elsewhere, in an energy-only market you expect some oscillation around that equilibrium capacity.
The first technical issue with this is that EUE is a nonlinear function of the reserve margin. So oscillations around the equilibrium capacity imply that there will be greater unserved energy than the amount suggested by the report.
The second technical issue with this estimate is that correlated outages are quite difficult to measure, and it seems likely to me that the methodology underestimates the probability of the sort of event we're seeing this week.
Comment #2: the report estimates an equilibrium reserve margin of 12.25%. Why might the actual number be lower? One reason is they use a weighted average cost of capital based on PJM, a market that is uniquely committed to de-risking investment in generation.
It seems inappropriate to me to use an *average* cost of capital in PJM to represent the *marginal* entrant in ERCOT. ( @JesseJenkins and I hope to elaborate on this point in a paper that we can hopefully share soon; until then, refer to http://rdcu.be/bVQJn .)
Comment #3: what the report refers to as "economically optimal" reflects a misunderstanding of the loss of load probability.
In order to calculate the "economically optimal" reserve margin, the simulations rely on a particular version of the operating reserves demand curve (ORDC), which builds on estimates of the loss of load probability (LOLP) and value of lost load (VOLL).
Estimates of the VOLL vary quite widely, so that's one source of uncertainty. But I think the bigger mistake here is that they treat the estimated LOLP as actual fact. I think that's highly suspect (I talk a bit about this in http://www.optimization-online.org/DB_HTML/2019/10/7414.html, forthcoming).
Without a much more detailed simulation resulting in actual estimates of lost load, I don't think using the estimated LOLP is a sound way of assessing "economic optimality."
It would be a little unsatisfying if I closed this thread without some speculation on, if it's not working, what do we do about it? Normal caveats apply that we will hope to learn more about the performance of the system as time goes by.
I would make two points. One need relates to the three comments above: technically, making sure we are modeling correlated outages and simulating the system as accurately as possible; economically, making sure systems are in place to manage financial risk.
A second point is that we could move toward the (in my view, widely held) intuition that maybe in the case of electricity we should be looking at something other than expected value as our measure of "optimal."
Looking at Figure ES-10 in the report ( http://www.ercot.com/content/wcm/lists/219844/2020_ERCOT_Reserve_Margin_Study_Report_FINAL_1-15-2021.pdf), it simply isn't that expensive to target a, say, 15% reserve margin instead of the 11% that the study deems "economically optimal."
To end on a slightly more philosophical note, @clarkamiller pointed out yesterday that, ideologically, people concerned with economic efficiency tend to favor both a) the energy-only market design and b) low reserve margins https://twitter.com/clarkamiller/status/1360956232836677633?s=20
So I hope this thread helps drive a little wedge in between those two aspects of the debate, and encourages people to the view that the argument for low reserve margins being "economically optimal" is far weaker than the arguments for energy-only markets.
