Academic publishing is a mess and it makes culture wars dumber

Well, academic publishing is a mess for many reasons, mostly to do with predatory publishers.

But I think here you mean peer-review is a mess. The thing is, peer review will always be a mess, because research itself is messy. There’s this pervasive myth outside academia that peer-review is supposed to be some gold standard that only lets truth though. But it never has been, and never can be. Peer review is much more about making sure papers are clear and detailed enough so that others can understand and replicate the work. The whole point is to get new work out into the research community so that others can poke and prod it. When we train PhD students, we teach them to sort through the peer reviewed literature in their area, and learn how to pick out the handful of papers that are actually worthwhile; those which further the field. This sorting the wheat from the chaff cannot be done at peer review time, because it takes many studies (and hence many papers) to develop and evaluate theories. Researchers have to take a holistic view of all the work published in their field to make sense of it.

Contrast this with the popular view in the media that each new paper presents some new truth. So playing this system for shock value is a ridiculously easy game. But it doesn’t tell us anywhere near as much about the peer review system as those outside academic seem to think it does.

21 Likes