Read on Twitter
1/25
A great and thought-provoking thread by @EpiEllie about current science skepticism and its proper explanation. This is what I've mainly been interested in for the last 6 months. Let's talk about science literacy and what makes science reliable! *THREAD* https://twitter.com/EpiEllie/status/1342931552146235402
A great and thought-provoking thread by @EpiEllie about current science skepticism and its proper explanation. This is what I've mainly been interested in for the last 6 months. Let's talk about science literacy and what makes science reliable! *THREAD* https://twitter.com/EpiEllie/status/1342931552146235402
2/25
Here's how I understand Dr. Murray's main claim. (Happy to be corrected!) There exists a popular view of what the difference between scientists and science-skeptical people consists in, that is essentially wrong. It goes like this.
Here's how I understand Dr. Murray's main claim. (Happy to be corrected!) There exists a popular view of what the difference between scientists and science-skeptical people consists in, that is essentially wrong. It goes like this.
3/25
Scientists rely for their beliefs on careful observation and analysis by a proper method, that maximizes the objectivity and reliability of conclusions. Science skeptics don't care for these methods, and also don't use scientifically credible sources for their information.
Scientists rely for their beliefs on careful observation and analysis by a proper method, that maximizes the objectivity and reliability of conclusions. Science skeptics don't care for these methods, and also don't use scientifically credible sources for their information.
4/25
How is this wrong (or at least too simple)? 1st, for the vast bulk of their beliefs, scientists rely on the work of others--they read the published peer-reviewed reports, rather than make the systematic observations and analyses themselves.
How is this wrong (or at least too simple)? 1st, for the vast bulk of their beliefs, scientists rely on the work of others--they read the published peer-reviewed reports, rather than make the systematic observations and analyses themselves.
5/25
2nd, science skeptics regularly do exactly that! They are perfectly capable of referring to scientific articles. What they are skeptical about is _not_ proper scientific methods--they just think that the results support their view, against that of the scientific mainstream.
2nd, science skeptics regularly do exactly that! They are perfectly capable of referring to scientific articles. What they are skeptical about is _not_ proper scientific methods--they just think that the results support their view, against that of the scientific mainstream.
6/25
If science skeptics do care about proper empirical methods and regularly refer to scientific publications, then what is the relevant difference between them and scientists, that explains their differing views?
If science skeptics do care about proper empirical methods and regularly refer to scientific publications, then what is the relevant difference between them and scientists, that explains their differing views?
7/25
Dr. Murray's answer as I understand it is, firstly, that due to professional incentives to publish as much as possible, scientists are liable to cut methodological corners and submit a lot of material of poor quality. "Our scientific system is fundamentally broken."
Dr. Murray's answer as I understand it is, firstly, that due to professional incentives to publish as much as possible, scientists are liable to cut methodological corners and submit a lot of material of poor quality. "Our scientific system is fundamentally broken."
8/25
Secondly, this results in a huge spread of published conclusions, such that you can likely find _some_ study to support almost any outlier view.
Secondly, this results in a huge spread of published conclusions, such that you can likely find _some_ study to support almost any outlier view.
9/25
The real difference between scientists and science skeptics is that scientists understand the situation I just described, and they therefore know that they must read scientific publications in a critical way and--crucially--they have the expertise required to do that.
The real difference between scientists and science skeptics is that scientists understand the situation I just described, and they therefore know that they must read scientific publications in a critical way and--crucially--they have the expertise required to do that.
10/25
Dr. Murray suggests that fighting skepticism then entails decreasing the number of poor-quality publications, by educating scientists, and making the public better at reading science. The first part she understands better than I do--I want to focus on the last part.
Dr. Murray suggests that fighting skepticism then entails decreasing the number of poor-quality publications, by educating scientists, and making the public better at reading science. The first part she understands better than I do--I want to focus on the last part.
11/25
Making the public better at reading science seems to be about imbuing "science literacy." The notion of science literacy, and its perceived importance, can be traced back over a hundred years, to writings such as John Dewey's How We Think (1910).
Making the public better at reading science seems to be about imbuing "science literacy." The notion of science literacy, and its perceived importance, can be traced back over a hundred years, to writings such as John Dewey's How We Think (1910).
12/25
Recently (e.g. in @OECDEduSkills's Pisa 2018 document), the motivation for teaching science literacy has been to enable citizens to participate fruitfully and democratically in a society largely driven and influenced by science. This indeed seems important.
Recently (e.g. in @OECDEduSkills's Pisa 2018 document), the motivation for teaching science literacy has been to enable citizens to participate fruitfully and democratically in a society largely driven and influenced by science. This indeed seems important.
13/25
Historically, science literacy has been taught mainly by making students emulate the activities of scientists. This could mean knowing what an RCT is, how to interpret a confidence interval and maybe even about effect sizes and R^2. But this just won't do for our purposes.
Historically, science literacy has been taught mainly by making students emulate the activities of scientists. This could mean knowing what an RCT is, how to interpret a confidence interval and maybe even about effect sizes and R^2. But this just won't do for our purposes.
14/25
If science literacy means understanding general scientific concepts and methods, then it seems obvious that this will never suffice for critically reading actual scientific publications, because doing that requires a deep understanding of the particular domain under study.
If science literacy means understanding general scientific concepts and methods, then it seems obvious that this will never suffice for critically reading actual scientific publications, because doing that requires a deep understanding of the particular domain under study.
15/25
Writing a PhD thesis usually gives you the equipment needed to start to acquire the domain-specific scientific knowledge required for reading published science critically--for being truly science literate.
Writing a PhD thesis usually gives you the equipment needed to start to acquire the domain-specific scientific knowledge required for reading published science critically--for being truly science literate.
16/25
"Science literacy" as traditionally conceived is by its nature generic, and stems from a narrow and individualistic conception of scientific method. It's invaluable for distinguishing science from obvious non-science, but clearly insufficient for assessing actual research.
"Science literacy" as traditionally conceived is by its nature generic, and stems from a narrow and individualistic conception of scientific method. It's invaluable for distinguishing science from obvious non-science, but clearly insufficient for assessing actual research.
17/25
If we can't realistically expect non-experts to acquire the ability to assess published research, no matter how much general conceptual understanding of science is taught, where does that leave us?
If we can't realistically expect non-experts to acquire the ability to assess published research, no matter how much general conceptual understanding of science is taught, where does that leave us?
18/25
After all, in a democratic society the people ultimately must decide (by way of their elected officials) what strategies to pursue, based on value judgments and our scientific understanding of the situation at hand.
After all, in a democratic society the people ultimately must decide (by way of their elected officials) what strategies to pursue, based on value judgments and our scientific understanding of the situation at hand.
19/25
The answer lies in an understanding of what it is about science that makes it our most reliable source of knowledge about the natural world. It is, ultimately, not the RCTs or even formal causal models that ensures the relative reliability of science--it's the peer-review.
The answer lies in an understanding of what it is about science that makes it our most reliable source of knowledge about the natural world. It is, ultimately, not the RCTs or even formal causal models that ensures the relative reliability of science--it's the peer-review.
20/25
I mean peer-review in the broadest sense, just the critical assessment of the works of other scientists that were the centerpiece in the first half of this thread. Scientists have incentives to publish quickly--but they also have incentives to find errors in others' work.
I mean peer-review in the broadest sense, just the critical assessment of the works of other scientists that were the centerpiece in the first half of this thread. Scientists have incentives to publish quickly--but they also have incentives to find errors in others' work.
21/25
Our most reliable process for generating knowledge about the natural world is a collective process, the output of which can be found in high-quality meta-analyses and eventually in text books and the very scientific canon--not in individual publications.
Our most reliable process for generating knowledge about the natural world is a collective process, the output of which can be found in high-quality meta-analyses and eventually in text books and the very scientific canon--not in individual publications.
22/25
As non-experts, we need to be able to tell science from non-science, but also where to find the reliable output of science: not in individual publications or claims by individual experts, but in the output of this collective process as a whole.
As non-experts, we need to be able to tell science from non-science, but also where to find the reliable output of science: not in individual publications or claims by individual experts, but in the output of this collective process as a whole.
23/25
This view of the scientific process as a collective process has been developed most famously in philosophy by Helen Longino, and more recently in Naomi Oreskes's book Why Trust Science (2019). The general field of social epistemology is also important.
This view of the scientific process as a collective process has been developed most famously in philosophy by Helen Longino, and more recently in Naomi Oreskes's book Why Trust Science (2019). The general field of social epistemology is also important.
24/25
It is instrumental that we can also make a judgment as to the credibility of a particular scientific field. Tools for making such judgments are developed within meta-science, e.g. by @METRICStanford and at @cochranecollab. The public needs access to more such information.
It is instrumental that we can also make a judgment as to the credibility of a particular scientific field. Tools for making such judgments are developed within meta-science, e.g. by @METRICStanford and at @cochranecollab. The public needs access to more such information.
25/25
Science skepticism is thus most effectively combated, I think, by making science education more about science's collective method, and about the conditions under which a research field as a whole is credible. Much to be done then, both in science education and meta-science.
Science skepticism is thus most effectively combated, I think, by making science education more about science's collective method, and about the conditions under which a research field as a whole is credible. Much to be done then, both in science education and meta-science.
