Read on Twitter
We just discussed @timnitGebru @emilymbender @mcmillan_majora and (sch) @mmitchell_ai s Stochastic Parrots paper at #DataEthicsClub. I'm going to attempt to summarise our discussion, but I'd love for the other attendees (or other readers) to jump in with their own points.
Here's a link to the paper before I begin: http://faculty.washington.edu/ebender/papers/Stochastic_Parrots.pdf
The first thing is to say that there is so much content in this paper! @alastairtanner said we could have discussed it for 3 hours, and I agree!
A lot of the discussion in my breakout room focused on the titular "stochastic parrots": if models are just parroting back what they see, what are the implications of this? What ARE we measuring? And how can these models be fooled?
We agreed with the paper's argument that large models amplify society's existing problems, due to the impossible task of curating such large data sets.
This could make a worrying feedback loop, with language models being fed as input the output of other language models. A total garbage-go-round.
There is an example in the paper of model error causing trouble: a man was arrested after a language model was used to translate his Facebook post which said "good morning" in Arabic was wrongly translated as "attack them".
We felt that more examples like this seem likely to happen. We drew parallels to the fact that image classification models are easy to fool: https://www.nature.com/articles/d41586-019-03013-5. Changing an image slightly can massively effect an AI's classification.
Some participants acknowledged that they'd been drawn in by AI hype into overestimating what these models can do. One person mentioned that learning how to trick a chatbot had opened his eyes to their downfalls.
We also spoke about the culture in predictive modelling for papers that are simply looking for a high percentage accuracy and little else, reminding me of Jurassic Park "Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should".
Unpopular opinion alert: I question the scientific merit of this kind of prediction as a product ML. What is the scientific community supposed to have learned if there is not an improvement in a predictive task? My feelings intensify when the code is not open!
We spoke about the perverse incentives for publication and what stops us Data Scientists from engaging with experts in politics or language, and building this in to their research.
It's partly because Data scientists (like me) lack training in this arena. Even if we have good intentions and respect our humanities colleagues, sometimes we just don't understand their work and it can be difficult to find the right vocabulary to have a productive discussion.
We are also separated into our faculty silos which can create additional bureaucratic challenges to working together. And there can be some tension in trying to work together, too, partly due to Data Scientists behaving like this: https://xkcd.com/1831/ .
For example, I went to a sociology of AI talk from and someone from sociology stood up and said without any prompting something like "Not all Data Scientists are pro-capitalism." I really didn't think this needed saying, so it was eye opening!
We mostly agreed that the contents of the paper wasn't particularly controversial, making it all the more shocking that it was blocked from publication and eventually led to @timnitGebru 's IMO very unfair and worrying dismissal.
I.e. the environmental and financial arguments weren't surprising. Some quotes did really put these arguments in perspective, though: "Training a single BERT base model (without hyperparameter tuning) on GPUs was estimated to require as much energy as a trans-American flight"
And to be clear, not surprising != not worth discussing. Someone made the good point "If it's not surprising, then is that reflected in how people are behaving?" Spoiler: no.
This apparent lack of controversy made us think that the Jurassic Park quote represented the truest worry that Google might have: being forced to consider whether or not this work should be done.
There were lots of other points that I'd like to explore more, e.g. the inbuilt positivistic assumptions of NLP and ML, and the missed opportunity costs of focusing on big language models over other things.
I hope other #DataEthicsClub attendees will add their thoughts!
I hope other #DataEthicsClub attendees will add their thoughts!
Next time we voted to discuss @Abebab's paper, towards a relational ethics. Join our mailing list: http://eepurl.com/hjkmnX everyone is welcome!
