Read on Twitter
My thoughts on Paul Maurice's crusade against public analytics:
1. I love it. It's hilarious seeing stats nerds get offended by his quotes. But this is exactly how I'd want my boss to defend me against public scrutiny. Plus he still offers much insight into proprietary analytics https://twitter.com/WiebesWorld/status/1357412960138788865
1. I love it. It's hilarious seeing stats nerds get offended by his quotes. But this is exactly how I'd want my boss to defend me against public scrutiny. Plus he still offers much insight into proprietary analytics https://twitter.com/WiebesWorld/status/1357412960138788865
2. Proprietary data IS more accurate and more granular than public data.
I've worked a bit with some proprietary data from @HockeyAnalytics for the Big Data Cup. My early conclusion is that an xG model built off proprietary data is ~7% more accurate than one from NHL's PBP data
I've worked a bit with some proprietary data from @HockeyAnalytics for the Big Data Cup. My early conclusion is that an xG model built off proprietary data is ~7% more accurate than one from NHL's PBP data
3. With that being said, data is only as useful as the data engineers, data scientists, and data analysts who handle it.
One elite data scientist can do a lot more with the NHL's PBP data than a team of snake oil salesmen can do with whatever proprietary data WPG has access to.
One elite data scientist can do a lot more with the NHL's PBP data than a team of snake oil salesmen can do with whatever proprietary data WPG has access to.
4. That is not to say WPG's data guys are bad. Maurice has said in the past that WPG's xG model is 85% accurate. I believe this is his interpretation of a 0.85 AUC.
My model is around 0.785 AUC (all sits) for this season. So, Maurice's value is reasonable & their model is good.
My model is around 0.785 AUC (all sits) for this season. So, Maurice's value is reasonable & their model is good.
5. Even with more accurate/granular data + an elite data team there can be bias & echo chambers. Teams may create environments where analytics guys tell coaches what they want to hear, not what they need to.
Here's SJS head video coach admitting they've created this environment:
Here's SJS head video coach admitting they've created this environment:
6. Not every team has this environment. I don't know if WPG does.
But if their data guys say they're a good play driving team at 5v5, something is off somewhere. They were -260 Fenwick+/- and -4 Goal+/- at 5v5 last year with a Vezina winning & Hart worthy year from Hellebuyck.
But if their data guys say they're a good play driving team at 5v5, something is off somewhere. They were -260 Fenwick+/- and -4 Goal+/- at 5v5 last year with a Vezina winning & Hart worthy year from Hellebuyck.
7. This is the strength of public hockey analytics. They're an open source library full of smart people who work together (sometimes indirectly, despite competing with one another) to be less wrong about hockey. Nobody is working to confirm a coach's pre-conceived notion.
Should I write a full form article on Medium about this topic, with a more in-depth dive into each of these concepts and further analysis on public Vs. proprietary analytics?
