Back in https://notebook.drmaciver.com/posts/2020-02-16-14:22.html I talked about "ghost knowledge" - knowledge that exists within expert communities but is never written down and basically doesn't exist for you unless you have access to those communities.

After a conversation with a friend recently I've realised that the problem is actually worse than I'd realised: Even the knowledge that is written down is hard to safely engage with without access to the expert community and that ghost knowledge.

And the reason for this is that credibility information is almost always ghost knowledge. e.g. People within an academic community know that a particular journal is generally solid, or that another one is where you put stuff you don't want the reviewers to scrutinise.

If I read a paper without that credibility information I may come away believing false things (even if I'm competent to evaluate the argument of the paper! It's hard work, I make mistakes, and I need to take e.g. experimental data on faith).

And this credibility information is rarely written down, and even when it is that problem just goes meta: How do I know the written sources are credible?

e.g. I once stumbled into a spat between two academics - one whose book I really liked, the other who thought the book in question was absolute trash. Both from the outside seem reasonably credible, and I don't really know how to distinguish the two.

Not all of this is resolvable by having access to the community, but a lot is - you can get second opinions on credibility, and credibility knowledge tends to be widely known within the community even if nobody writes it down - but without someone to ask it's a guessing game.

This is also why heuristics like https://twitter.com/DRMacIver/status/1351618847246331904 are important. It's not that there's no good data driven social science - of course there is - it's that it's almost impossible to figure out what it is from the outside because of the lack of good credibility information.