Read on Twitter
One week has passed since our caribou-wolf rebuttal was published. In that time, the paper has been accessed almost 4000 times and widely covered in the media. Commentary on Twitter has ranged from blindly supportive to highly opposed. Some reflections. /1 https://bit.ly/3j00Om9
One of the common interpretations has been that predator control does not work ever, in any form. This was not what we said. We addressed a flawed statistical analysis that claimed that “adaptive management” explained trends in caribou vital rates. /2
Based on our reanalysis of the published data, adaptive management treatments do not perform any better than a null model, as explained in the paper; see also @ViktoriaWagner1's recent thread. /3 https://twitter.com/ViktoriaWagner1/status/1284529592745656321
Some people have taken to Twitter to attempt to explain why our analysis results are flawed. @JasonTFisherLab may have been first up, and in his new thread has some interesting points. /4 https://twitter.com/JasonTFisherLab/status/1283174234164871168
@AdamTFord‘s thread on Friday attracted lots of attention + seemed to convince some folks that we made an error, while also advancing the claim that wolf control explains vital rates. But this thread has a couple problems. Let’s examine them. /5 https://twitter.com/adamTford/status/1284013405423071232
Adam claimed we dropped “Boreal” (n=1) in the paper but not the scripts. But he's wrong. He is conflating the AIC and post-hoc pairwise t-test. The only place we excluded boreal was in the t-test, because you can't run the test on a ‘group’ of n=1. /6
In fact, the data we used for model selection (our Table 1A) match Serrouya’s exactly; download the data and scripts from Figshare and try it out yourself. We only added a null. And performance of treatment and null does not differ. /7
Next, @adamTford used our supposed exclusion of "Boreal" to justify dropping Translocation, and merging "Wolf reduction" with "Wolf reduction + penning" (which were separate treatment levels in Serrouya). /8
We caution against retroactively reclassifying treatments or removing data to improve performance for a favoured predictor. The pitfalls of using marginal significance to ‘chase small effects hidden in noisy data’ are well known: /9 https://www.nature.com/news/scientific-method-statistical-errors-1.14700
P-fishing can be an educational experiment in the context of a Twitter thread, but we are discussing a published PNAS paper that's been cited as a policy guide. Decisions like this require sound science. /10
We went out of our way to make sure the original authors knew that the null performed as well as treatment and that they had opportunity to address these issues. Eight months later, their paper was still up. — We never wanted this debate but what would you do in our case? /11
We have four more points in our paper. Turns out, lots of knobs that can be turned in modeling like this, including what popns to include, which start/end years to use for lambda, censusing uncertainty, other mitigation influencing vital rates... /12
…not to mention the not-insignificant problem that we can’t reproduce the geospatial estimates of how much habitat was 'altered' + the data product used from the outset had some big limitations. These issues need to be addressed, too, not just 'the stats'. /13
Mountain caribou conservation matters. We encourage more people to look at the data, look at our scripts more carefully, and alert us to anything we could have done better. We got into this because we genuinely want policy-makers & the public to be well-informed. /14
We are grateful for the constructive conversations that are happening. If some people feel like their work was strongly criticized, that is because it was. It's never fun but this is how science and evidence-based adaptive mgmt. should work. /END
