The Automated Screening Working Group (which includes @sciscore) has pooled tools and expertise to tackle the problem of research quality in fast-evolving, non-peer-reviewed publications, i.e., #COVID #preprints. This group asks, can robots review manuscripts?

Thread 1/6

The good news is, yes they can, at an incredible speed even. It takes 6 sec to run 6 tools (a full list at https://scicrunch.org/ASWG/about/COVIDPreprint). The report is pushed to @hypothes_is so you can see it for every #COVID19 @biorxivpreprint and @medrxivpreprint. So all good?

Thread 2/6

Not exactly: 13.6% of preprints shared open data and 14.3% open code. 27% contained an ethics approval statement for human or animal research, the rest likely secondary or tertiary analyses, modelling studies or cell line studies that do not require approval.

Thread 3/6

Despite known sex differences in COVID-1910, only 20% of all COVID-19 preprints, and 38% of preprints with an ethics approval statement, address sex as a biological variable. For published work in PubMed Central, this is 55%.

Thread 4/6

Statements regarding sample size calculations (1.4%), blinding (2.7%) and randomization (11.4%) were very low, also among preprints including human ethics statement (2.4%, 5.4% and 12.6%, respectively).

Thread 5/6

All of this was published in @NatureMedicine yesterday in a Letter to the Editor: https://www.nature.com/articles/s41591-020-01203-7
A great first result as far as we're concerned, more to follow soon!

Thread 6/6