I'm going to be thinking about this paper for a good long while... how to make a transformer #AI that has good #privacy properties.

Naively, I think it's de facto impossible without compromising performance, hence utility in a competitive ecosystem. https://twitter.com/colinraffel/status/1339012222811598848

The fundamental problem is that you use transformers because they know a lot of input data. That's what they are good for!

Now you come along and say, "hey, we want an algorithm that knows a lot of things, but not specific things that compromise human privacy or IP".
Tough ask.

So some folks do the responsible thing and make transformer AIs that *do* try to respect privacy... and they have to go head to head in the market with those that don't, or that do it less so.
The algorithm doesn't care, it's just running code. So it's human systems.

The end result is one of a few plausible outcomes. Either everyone gets the unlocked AIs and privacy erodes to chosen plaintext attacks, commercial code uses privacy-preserving AI while intelligence and AI researchers get unlocked code, or no one uses AI.

The third one seems…unlikely at this point. That leaves (1) or (2)... and we're back to a @doctorow meets @DavidBrin future where there is a war going on, but it's over sousveillance and what algorithms you are allowed to own.

The stakes are real, and are high.
My performance at work (I'm a futurist working for government) has already improved due to my use of GPT-3 for work purposes. Yours has too, because of autocomplete.

As these systems get more integrated into Word, Google Docs, Sheets, and PowerPoint, you'll see significant performance differences between the haves and have-nots, and then between the trained and trained-nots.

Add a little bit of programming to a GPT-3 used in a loop, and you can generate a list of a thousand English idioms in a half-hour, and edit them in another 30 minutes.

AI produces content, what, 1000 times faster than humans?
Editing becomes the limiting factor.

A lot of it, today, is not good. You have to edit like crazy, which is still faster than original work, but maybe only four or five times faster.

Consider the performance differences between two employees, one of whom is operating four times faster than the other.

All this to say, it would be worth thinking through an influence cascade on this. @supergovernance @wendyinfutures @avantgame @ArmineYalnizyan