Read on Twitter
So: Is there statistical evidence of irregularities in voting results from the 2020 election? A student in last night's class with @sarahcroco and #jakanathomas mentioned that Internet data pundits had found such evidence, using Benford's Law. A thread. /1
First, Benford's Law is a cool statistical regularity. It states that, in many naturally-occurring sets of numbers, the leading digit is more likely to be small than large. You should see 1 as a leading digit about 30% of the time, for example. /2
This happens with phenomena as diverse as population sizes, the surface areas of rivers, and, as Javier Marquez Pena demonstrates in one of @statmodeling's blog posts, postal addresses. /3 https://statmodeling.stat.columbia.edu/2013/06/01/benfords-law-and-addresses/
Enter the 2020 Presidential election, which many people on the American right have called into question. The reply from the left (and the center, and sometimes from elsewhere on the right) has been that there's no evidentiary basis for such a claim. /4
"Aha," say proponents, "but there is. The voting returns for Biden/Harris from various precincts violates Benford's Law!" And a look at some precinct-level data like these* do indeed show what looks like gross irregularities. /5
* (I'm not linking to the original source for a couple of reasons. First, it's not hard to find, and second, I don't want to give these arguments any SEO oxygen, because they're dangerous and, as I'll show momentarily, wrong.)
When this came up, @SarahCroco and I were, first and foremost, delighted that our students were aware of Benford's Law! Woo hoo!! And when Benford's Law came up, the same name came to our lips immediately: Walter Mebane. /6
Professor Mebane is the guy to listen to on election fraud. He teaches a class in election forensics, has written software, and very much knows his stuff. Consider this informative (and very funny) writeup of the 2016 Russian Duma election. /7 https://www.washingtonpost.com/news/monkey-cage/wp/2017/01/11/when-the-russians-fake-their-election-results-they-may-be-giving-us-the-statistical-finger/
Professor Croco pointed out that he'd written a brief memo on the 2020 election and concluded that no irregularities have occurred. Which, indeed, he has—probably to send to reporters who call him to ask about this. I had to read it. /8 http://www.umich.edu/~wmebane/inapB.pdf
What does the paper say? First, it points out that, in precinct-level elections, the distribution of the first digits is not a useful indicator of election fraud. There are other useful indicators, which he examines. But the first digit isn't one of them. /9
A 2011 paper in @polanalysis by Deckert et al. backs up this point. "[C]onformity with and deviations from Benford's Law follow no pattern. ... Its 'success rate' either way is essentially equivalent to a toss of a coin." /10 https://www.cambridge.org/core/journals/political-analysis/article/benfords-law-and-the-detection-of-election-fraud/3B1D64E822371C461AF3C61CE91AAF6D
As Prof. Mebane puts it, "Clearly the first digits of the Biden/Harris counts will most frequently be 3, 4 or 5. That non-Benford’s Law pattern simply reflects the distribution of precinct sizes (presuming turnout did not vary that much across the city)." /11
He then goes on to run a variety of election forensics tests that *do* work on precinct-level data. He finds a few statistical irregularities, which you're bound to find if you run enough tests—as xkcd memorably demonstrated. /12 https://xkcd.com/882/
But looking at the outcome of multiple forensic tests, he concludes that "the particular datasets examined in this paper give essentially no evidence that election frauds occurred." He also notes, responsibly, that these are preliminary results, which they are. /13
Looking at the variety of sources that have forwarded the Benford's Law claim, I can't blame the student for being sucked in. They use statistics. They sound authoritative. One even comes from an actual mathematician—who misses the (non-mathematical) reason the test fails. /14
And most denizens of the internet can't readily refute such claims, or if they can, won't take the time. The information environment that we all live in really, truly sucks. It is often *designed* to suck, in ways that we find engaging. Because attention=profit. /15
A different student in the same class asked what people with limited time and attention could do to insulate themselves from misinformation. Insulating them is a job that responsible people used to do. And many, like Professor Mebane, still do. /16
But the attention economy incentivizes irresponsibility, rabble-rousing, and bullshit. As one @nytimes employee put it in this candid piece, "Hate drives readership more than any of us care to admit." /17 https://nymag.com/intelligencer/2020/11/inside-the-new-york-times-heated-reckoning-with-itself.html
To address the student's question, what can we do? It's a really hard question. Faculty can double down on the parts of our courses that help people sort fact from BS. We can all try to be savvier consumers, and more humble in light of widespread attempts to deceive. /18
Most of all, though, I think we have to demand that government and industry address the attention economy in ways that curb its destructiveness. Because the Trump years and the 2020 election are nothing if not a case study in how devastating its dynamics can be. /fin
