Read on Twitter
DNNs perform well on a range of medical diagnosis tasks, but do they diagnose similarly to humans?
In breast cancer screening, DNNs use different features than radiologists. Some are spurious, while others may represent new biomarkers.
https://arxiv.org/abs/2011.14036
1/9
In breast cancer screening, DNNs use different features than radiologists. Some are spurious, while others may represent new biomarkers.
https://arxiv.org/abs/2011.14036
1/9
Drawing inspiration from the natural image domain, we use perturbation robustness as a lens to compare human and machine perception. We demonstrate the importance of drawing separate conclusions for subgroups that differ in their diagnostic approach.
2/9
2/9
In our breast cancer screening case study, we compare radiologists and DNNs with respect to their robustness to low-pass filtering (blurring), drawing separate conclusions for microcalcifications and soft tissue lesions.
3/9
3/9
We apply probabilistic modeling in order to isolate the effect that low-pass filtering has on the predictive confidence and class separability (correctness) of radiologists and DNNs.
4/9
4/9
For microcalcifications, DNNs use a different set of high frequency components than radiologists. These could be spurious, but could also be new biomarkers. For soft tissue lesions, DNNs use spurious high frequency components ignored by radiologists.
5/9
5/9
The divergence on soft tissue lesions is only observable through subgroup analysis. If we aggregate microcalcifications and soft tissue lesions, we artificially inflate the similarity between radiologists and DNNs. This can potentially be very harmful in the medical domain.
6/9
6/9
We expect our comparison framework to be applicable to other medical diagnosis tasks and imaging modalities, and urge others to incorporate subgroups in order to draw correct conclusions.
The code for this paper is available at https://github.com/nyukat/perception_comparison.
7/9
The code for this paper is available at https://github.com/nyukat/perception_comparison.
7/9
Coauthors: @kudkudakpl, W. Oleszkiewicz, C. Chacko, @Rehrenpreis, @nazsamr, C. Chhor, @EricKimMD, @DrJiyonLee, K. Pysarenko, B. Reig, H. Toth, D. Awal, @LindaDu10, A. Kim, J. Park, @DanielSodickson, L. Heacock, @DrLindaMoy, @kchonyc, @kjgeras
9/9
9/9
