Read on Twitter
Since inquiring minds want to know, I’m inspired to offer a tweetorial on how statisticians can interpret the weather. 1/n https://twitter.com/daniela_witten/status/1343245828945833988
Let’s say the news reports that there is a 30% chance of snow. If you’re not a statistician, this statement might sound fine. If you are a statistician, the more you think about this statement, the more you will get frustrated… 2/n
Fortunately, some statisticians and lots of excellent weather and climate scientists have thought about this A LOT. Tldr: it means after some adjustment 30% percent of numerical weather simulations for the area modeled predicted snow. But let’s go deeper. 3/n
To have some variety in examples, or because I have slides already made 
, let’s say that rather than wondering where the probability estimate of snow comes from, you’re interested in understanding where the probability of sea ice (frozen ocean water) presence comes from. 4/n
, let’s say that rather than wondering where the probability estimate of snow comes from, you’re interested in understanding where the probability of sea ice (frozen ocean water) presence comes from. 4/n
The ideas are the same for snow, rain, and other variables Current forecasts are based on numerical predictions systems.These deterministic predictions are physics-based simulations of how the atmosphere, ocean, and other components of the Earth system are expected to evolve. 5/n
These prediction systems take as inputs information about the state of the world at the time the forecast is issued, i.e., water temperature, wind direction, etc. Then, the simulations are evolved forward to the time at which a forecast is needed. 6/n
You can think about this loosely as numerically solving massive systems of non-linear differential eqs. Weirdly, if you run two simulations of a non-linear system forward that start with even just slightly different states, the simulations will end up varying, often by lots 7/n
This famous idea is chaos theory. Scientists use this idea to get a set of possible realizations of what weather could unfold. Multiple simulations are run with a slightly varied conditions. This collection of simulations is called an ensemble 8/n
So, is the probability of sea ice presence just the proportion of ensemble members that predict sea ice presence? Unfortunately, not. Numerical prediction system may have biases and/or not be calibrated. 9/n
Biases refer to systematic errors in the estimates the numerical prediction system makes. So, if, for example, a numerical prediction system on average expects sea ice too often, then the prediction system is biased. 10/n
Calibration gets at not just if the prediction system is accurate on average, but also if the spread of the ensemble members is correct. We say a forecasting system is calibrated if events predicted with probability p occur p proportion of the time for all p. 11/n
We can assess calibration, with a reliability diagram. Assume we have some training data. We can group together all the events that were predicted to occur with various probabilities and plot these probabilities against the proportion of the time those events occurred. 12/n
If we had a perfectly calibrated prediction system, all these points would lie on the line y = x. You can see that doesn't happen for my sea ice data, indicating that the prediction system I'm evaluating here is not calibrated. 13/n
Fortunately, stat methods are applied to the outputs of forecasting systems before forecasts are issued. These methods correct biases and improve the calibration of the output of these numerical prediction systems. 14/n
So, what does it mean when you hear that the probability of sea ice presence is 0.3? It basically means that in 30% percent of possible physical simulations of the Earth system, sea ice is predicted. However, there's some stat modeling going on too to get to that estimate. 15/n
How do those stat methods for bias correction and calibration work? It's an active research area. Generally, it depends a lot on what meteorological variable(s) are being considered and what the forecast goals are. 16/n
I've thought a lot about how to do this for sea ice. For details see my papers with @AdrianRaftery1 and @drseaice in Journal of Climate and soon-to-appear in AAOS 17/17 https://journals.ametsoc.org/view/journals/clim/30/23/jcli-d-17-0185.1.xml?tab_body=fulltext-display
https://imstat.org/journals-and-publications/annals-of-applied-statistics/annals-of-applied-statistics-next-issues/
https://imstat.org/journals-and-publications/annals-of-applied-statistics/annals-of-applied-statistics-next-issues/
