Paper day: "A cautionary tale in fitting galaxy rotation curves with Bayesian techniques: does Newton's constant vary from galaxy to galaxy?"

SPOILER: the answer is "no" because we do respect Betteridge's law.

All work by @PengfeiLi0606. Read it here: https://arxiv.org/abs/2101.11644 

Backstory: over the past two years several studies argued that -in a MOND context- a single acceleration scale a0 cannot be used to fit every galaxy in the SPARC database, contrary to what we previously found ( https://arxiv.org/abs/1803.00022 ). If true, this would rule out MOND outright.

Unfortunately, fitting rotation curves is not a trivial exercise: the nuisance parameters are degenerated and the formal uncertainties cannot always be taken literally (it ain't like running a controlled experiment in a physics lab!). So adding a free parameter can be dangerous.

Bayesian statistic helps a lot in breaking parameter degeneracies but one has to be careful in using empirically-motivated, informative priors, otherwise some of the fitting parameters may go astray for trivial improvements in the fit quality (uncertainties ain't equal to errors)

So we present a "reductio ad absurdum": what happens if we use the same (not-so-appropriate) priors & techniques from studies claiming a variation in a0, but we let Newton's gravitational constant G to vary from galaxy to galaxy instead?

Here's the result: G varies and several galaxies are incompatible with the expected value (from the Solar System) at more than 3 sigma. Taking this at the face value, the same (not-so-appropriate) approach that rules out MOND also rules out Newtonian dynamics. Oh, damn!

The real issue is that adding G (or a0) as a free parameter doesn't really improve the fits. One can get similar fits with a narrow informative prior on G (or a0) or just by keeping it fixed. Below the reduced Chi^2 is used just as a zeroth order statistic to check fit quality...

...but if one scrolls through each and every fit by eye will reach the same conclusion. This is probably the moral of this tale: Bayesian statistic is incredibly helpful but cannot substitute our brain & our experience-built appreciation of the imperfect nature of the input data.