Read on Twitter
Win Probability, Explained 
What is win probability? How does it work? Is it ever "correct"?
If you ever find yourself asking these questions, this thread is for you
What is win probability? How does it work? Is it ever "correct"?
If you ever find yourself asking these questions, this thread is for you
1/ Win probability (abbreviated "WP") is the likelihood that a team will win a particular game, expressed as a percentage.
50% WP: If the game is played 1000 times, the team will win ~500
10% WP: If the game is played 1000 times, the team will win ~100
50% WP: If the game is played 1000 times, the team will win ~500
10% WP: If the game is played 1000 times, the team will win ~100
2/ Importantly, WP is not a statement about whether a team WILL or WON'T win a game. A team having a 30% WP means they're MORE LIKELY TO LOSE THAN WIN but not that THEY'RE GOING TO LOSE.
3/ There are, generally speaking, two general types of WP models:
- Pre-game WP models
- In-game WP models
- Pre-game WP models
- In-game WP models
4/ Pre-game WP models predict the probability that a team will win based on how good each of the teams are, home field advantage, and any other information that might be knowable *before the game*.
As should be obvious, these models calculate WP before the game is played.
As should be obvious, these models calculate WP before the game is played.
5/ For example, a pre-game 
WP model might calculate Team A's WP against Team B by taking into account the following:
- Team A is strong on offense and mediocre on defense
- Team B is weak on offense and defense
- Team B is home
- It'll be raining during the game
WP model might calculate Team A's WP against Team B by taking into account the following:- Team A is strong on offense and mediocre on defense
- Team B is weak on offense and defense
- Team B is home
- It'll be raining during the game
6/ In-game WP models calculate a team's WP while the game is being played. These models often take into account the same things that pre-game models do (e.g. team strength), but also take into account the game situation.
7/ For example, an in-game WP model might calculate Team A's WP by taking into account that:
- Team A is strong on offense, OK on defense
- Team B is weak on offense & defense
- Team B is home
- It's raining
- Team A is up 21-7
- Team B has the ball, 3rd & 8 from Team A's 40
WP model might calculate Team A's WP by taking into account that:- Team A is strong on offense, OK on defense
- Team B is weak on offense & defense
- Team B is home
- It's raining
- Team A is up 21-7
- Team B has the ball, 3rd & 8 from Team A's 40
/8 So how do these models actually work? Well, that depends...
There's more than one way to build a WP model, ranging from simple to incredibly complicated. They're all trying to predict the same thing, the probability that a team wins, they just do it in different ways.
There's more than one way to build a WP model, ranging from simple to incredibly complicated. They're all trying to predict the same thing, the probability that a team wins, they just do it in different ways.
9/ The most simple version of a WP model doesn't take a statistics degree to understand.
Let's say that we want to predict the in-game win probability for Team C, an NBA team currently beating Team D by 8 points with 3 minutes left in the 3rd quarter...
Let's say that we want to predict the in-game win probability for Team C, an NBA team currently beating Team D by 8 points with 3 minutes left in the 3rd quarter...
10/ To calculate the WP, we find all past NBA games where a team was beating its opponent by 8 points with 3 minutes left in the 3rd quarter.
Let's say there are 100 of such games in the last 20 years, and the team leading by ended up winning 85 of them.
Let's say there are 100 of such games in the last 20 years, and the team leading by ended up winning 85 of them.
11/ From this, we'd deduce that Team C has an 85% WP because 85% of teams that have been in this situation ultimately won.
Of course, this doesn't account for a number of factors, including team strength. We'd want to layer that information in for a better prediction.
Of course, this doesn't account for a number of factors, including team strength. We'd want to layer that information in for a better prediction.
12/ More advanced models riff on this theme, using information on the outcomes of past games to make predictions about future games or games in-progress...
13/ But more advanced models take into account additional information (including information from games that don't exactly resemble the game in question) and use more sophisticated techniques to build predictions.
14/ Are these models ever tested?
If they're good, yes. A good WP will have been tested against actual historical games to determine how well it would have predicted win probabilities before and/or during those games.
If they're good, yes. A good WP will have been tested against actual historical games to determine how well it would have predicted win probabilities before and/or during those games.
15/ I see different models showing different probabilities. Why are they different?
Different models produce different results because they take into account different factors (one might account for weather while another doesn't) or are built using a different techniques.
Different models produce different results because they take into account different factors (one might account for weather while another doesn't) or are built using a different techniques.
16/ Okay, but which model is right?
No model is "right," per se. WP can't be precisely known and all models that try to quantify it are imperfect estimates (though in some cases, very good imperfect estimates).
"All models are wrong, but some are useful"
-George E.P. Box
No model is "right," per se. WP can't be precisely known and all models that try to quantify it are imperfect estimates (though in some cases, very good imperfect estimates).
"All models are wrong, but some are useful"
-George E.P. Box
17/ That said, there are ways to determine which models tend to be more accurate over time.
For more on analyzing the accuracy of WP models, check out this piece from @StatsByLopez: https://statsbylopez.com/2017/03/08/all-win-probability-models-are-wrong-some-are-useful/
For more on analyzing the accuracy of WP models, check out this piece from @StatsByLopez: https://statsbylopez.com/2017/03/08/all-win-probability-models-are-wrong-some-are-useful/
18/ Interested in getting a bit more technical? Check these out:
Stephen Hill's WP model walk-through: http://medium.com/@technocat79/building-a-basic-in-game-win-probability-model-for-the-nfl-54600e57fe1c
@recspecs730's
model description: http://lukebenz.com/post/ncaahoopr_win_prob/
Stephen Hill's
WP model walk-through: http://medium.com/@technocat79/building-a-basic-in-game-win-probability-model-for-the-nfl-54600e57fe1c@recspecs730's
model description: http://lukebenz.com/post/ncaahoopr_win_prob/
19/ Well that's all for today - If you have any questions or if there's a metric you'd like me to cover in the future, drop a reply below!
