Some confessions re: @NateSilver538 and @FiveThirtyEight

1. Headlines like this are a weasel move, but I don't necessarily mind it (in a vacuum).

My major objection to 538/Silver is that if Trump does win, they point to this headline and say "I told you". Same if Trump loses. https://twitter.com/NateSilver538/status/1323089760190124032

2. In terms of modeling and aggregating polls, 538 is probably the best out there.

I know from modeling other kinds of events -- not elections -- that this takes a lot of work and isn't easy.

3. If I were to try modeling elections based on polls -- I have no plan to do so (see #2) -- I expect I'd end up with very similar forecasts to 538/Economist.

So I'm not claiming they're doing anything "wrong" or that I could beat them at their own game.

4. Except in case of 0% and 100% probabilities, there's no such thing as "getting it right" in an absolute sense. In the same way, there's no such thing as "getting it wrong".

All that makes sense is relative accuracy compared to another forecaster.

My problem, as I've said many times, is
- they misrepresent the game they're playing and
- the game they play isn't really worth playing.

On the 2nd point: The goal is to forecast the winner.

Not: forecast the election based only on [some pre-defined type of data].

As @SimonDeDeo wrote earlier today. If the latter is the game, where the "type of data" is limited mostly to polls, then how much value is 538 adding beyond RCP average? https://twitter.com/SimonDeDeo/status/1322765502431768576

Yes, 538 forecast is much more sophisticated than RCP average, but if the ultimate headline based on 90% forecast is "Trump can still win", has all the effort been worth it?

And has the effort actually produced an insight that most people can't already see for themselves?

On the first point, if the upshot of their analysis is "x% events happen all the time", when x is any small number (10% this year), they are setting up a situation in which they're right no matter what happens.

Yes, I know that x% events happen all the time. But if this is (year after year) the ultimate conclusion of the analysis, I'm afraid it doesn't do much good in clarifying anything.

My preference for prediction markets is that at least there
(i) the derived % means something -- someone actually paid that price to trade and
(ii) the people stand to gain/lose based on trades.

PMs can get it wrong too, but at least there's a consequence.

A trader can convince himself he "got the right price" even after losing -- and he can be right about that -- but still suffers the loss on that trade. Traders can stay delusional longer than they stay solvent.

This is why I'm so curious about the relative performance between modelers and prediction markets.

I've pre-registered the analysis to make sure no weasel moves post election.

We'll see what happens.

https://researchers.one/articles/Models-vs-Markets-Forecasting-the-2020-US-election/97212967bc124e2bf3af31a5

https://pivs538.herokuapp.com/