Read on Twitter
Looking at state level polls and what makes an A+ rating on 538.
In the tipping point state of Pennsylvania Muhlenberg are rated "A+". They're one of six out of over 400 polling firms in the US with this top rating.
This is their methodology:
In the tipping point state of Pennsylvania Muhlenberg are rated "A+". They're one of six out of over 400 polling firms in the US with this top rating.
This is their methodology:
538 have a dated preference for live telephone polls.
I used to advocate for them but with declining response rates we need to be more cautious. In 2016 non-response of non-college graduates skewed the results.
This means that the polls may need to do lots of weighting.
I used to advocate for them but with declining response rates we need to be more cautious. In 2016 non-response of non-college graduates skewed the results.
This means that the polls may need to do lots of weighting.
Unfortunately weighting isn't free, it reduces the effective sample size and introduces a design effect error.
Because this poll's sample size is already small (419 [and worryingly including 107 landlines]) you end up with a fairly useless margin of error of 5.5%.
Because this poll's sample size is already small (419 [and worryingly including 107 landlines]) you end up with a fairly useless margin of error of 5.5%.
With 94% and 96% retention the overall result is predicated on the tiny number of switchers and their weights.
The outcome that this projects is almost entirely the result of a greater appeal for Biden among those that didn't vote etc last time.
The outcome that this projects is almost entirely the result of a greater appeal for Biden among those that didn't vote etc last time.
Overall, a lot in this poll is predicated on a very small number of cases.
With that in mind it's seems remarkable that the Muhlenberg series has been so stable: 48-44, 51-44, 49-43 since August.
On the other hand the volatile Iowa Selzer poll (also A+) has drawn scrutiny.
With that in mind it's seems remarkable that the Muhlenberg series has been so stable: 48-44, 51-44, 49-43 since August.
On the other hand the volatile Iowa Selzer poll (also A+) has drawn scrutiny.
Which brings me to the other reason why 538 rate Muhlenberg so highly - their record. And to be fair the record is good by US state level polling standards.
But if you were to calculate the RMSE of the error in the margin of victory you'd end up with +/- 9.6% (95% CI).
But if you were to calculate the RMSE of the error in the margin of victory you'd end up with +/- 9.6% (95% CI).
Overall I'm not suggesting not to trust the polls here though I am pouring caution over some of them individually.
I would also say that the ratings from 538 should be treated with a dollop of salt in favour of scrutinising the details of each poll where available.
I would also say that the ratings from 538 should be treated with a dollop of salt in favour of scrutinising the details of each poll where available.
