A thread at the intersection of the 3 most interesting topics in the world: #Privacy, #DigitalAdvertising, #IncrementalityTesting.
TL.DR: Take care that cookie replacements don’t bias your ad incrementality tests. 1/10

Our story begins with browsers' (Safari, then Firefox) decision to block third party cookies. They did so (ostensibly) to protect consumer privacy and perhaps because they believed that advertisers could get on just fine without cookie IDs. 2/

Nonetheless, advertisers value consumer identity because it allows them to target, measure, & optimize ads. Advertisers value this a lot: ads without cookies fetch ~40-70% less revenue. E.g. Firefox & Safari users’ ads are much less valuable. 3/ https://twitter.com/garjoh_canuck/status/1166037247386370048?s=20

#AdTech is responding to this move by replacing cookie ids with (hashed) email ids. So, now websites ask users to log in and may share hashed emails with adtech (through back-end server calls rather than routing through the browser). 4/
More here: https://adage.com/article/digital/inside-facebooks-push-get-advertisers-plug-directly-its-ad-servers/2290606

It’s worth pausing here to note the irony that pseudonymous IDs passed through the browser (where user could intervene) are replaced with hashed PII (with consent) passed through APIs. #privacy
I guess economic incentives (when understood) are such that “life finds a way.” 5/

Now, we turn to #incrementality testing. Advertisers should be running experiments to learn their ad ROI. Existing tools make this easy: e.g. Facebook’s Conversion Lift, Google’s Conversion Lift (uses our Ghost Ad methodology). 6/ https://twitter.com/garjoh_canuck/status/1322287393010077696?s=20

If you examine an outcome for which the user provides their email (e.g. sign-up, purchase), then you should be on solid ground because this outcome is equally observable for treatment and (holdout) control users. 7/

The slippage happens between when a user arrives at the site and when (if) she logs on. At this moment, she has a 1st party relationship with the site (via a cookie id) but cannot be connected to 3rd party data (identifying ad views). 8/

One solution here is to use "click IDs": e.g. the site routes a click on a Facebook ad back to Facebook, such that Facebook can match its user to the website’s 1st party cookie ID. Now, Facebook can observe site visits for clicking users. 9/

The problem: ad clicks *only* happen in the treatment group. If you want to measure effect of ads on site visits, the experiment will overestimate the true lift because we track visits for *more* users in treatment (log-in + click ID users) than in control (log-in only). 10/10

I think the audience for this thread is @rickbruner @adamheimlich @EconInformatics @Myles_Younger @enub @eleafeit @ProfHoban and that’s maybe it