“These caveats do not mean faster data should be ignored, but that we need to know its weaknesses while marvelling at its speed.” @ChrisGiles_ yes and yes https://www.google.com/amp/s/amp.ft.com/content/366653da-fc7b-4f3d-bf2f-ef95dfc18041

since 2016 I worked on private big data project at @federalreserve we transformed card transactions from First Data into retail sales series. 5 Fed economists, an excellent research assistant (now Stanford PhD econ studies), a brilliant Palantir programmer, (now at Philly Fed)

our methods paper shoes hoe we started with garbage data, for statistical purposes (left) and got to close correspondence with national, monthly retail sales from Census Bureau. only then did we use our daily, geographic detail for Fed policy work https://www.federalreserve.gov/econres/feds/files/2019057pap.pdf

my last forecast in 2018, wrote Tealbook, economic summary for FOMC before each monetary policy meeting, we included real-time analysis on consumer spending from Hurricanes Harvey and Irma. we get our data within 3 days! years of painstaking work, got policymakers fast macro read

during government shutdown, when Census could not publish official retail sales and economy was in a rough patch, our data gave FOMC high quality info on retail spending

not at Fed anymore so don’t know how our data are being used now .. but I’m sure they are. FOMC must make decisions in the moment. they cannot wait for the data dust to settle. why they invested big dollars and years of staff time to tapping into private big data, pays off

another project started same time at Fed in 2016 uses ADP payrolls data to understand what’s happening in the labor market ... their high quality data (after years of with too) now shows wide spread wage cuts, never seen since Great Depression https://www.brookings.edu/bpea-articles/covid-19-and-labor-markets/