I am pleased to learn that the Dutch Universities start looking at rankings of a more scientific way. It is long overdue that we take scientific peer review of the indicators used in those rankings seriously, instead of hiding beyond fud around the decline of quality of research.

So, what defines the quality of a journal? Or better, of any scholarly dissemination channel? After all, some databases do better peer review than some journals. Sadly, I am not aware of literature that compares the quality of peer review in databases with that in scientific journals. Also long overdue, in my opinion.

I hope the Open Science community will help shape these scholarly dissemination channels, journals included. Some ideas, the outlet:

  • encourages post-publication peer review
  • communicates the post-publication peer review
  • allows updating easily small fixes and clarifications (no hiding behind the version-of-record)
  • ensures supp info / additional files undergo the same level of peer review
  • use modern solutions for communication (like semantic web technologies)
  • have clear licenses for all aspects of the research output
  • actively fight against visual representation only, but provides all data
  • guarantees that supp info / additional files are archived, as the output itself
  • adopts, promotes, requires community standards (including global, unique identifiers)

Okay, these items are pretty broad. Many of them are part of FAIR, but that should not surprise you, because FAIR are just applying traditional scholarly approaches, like properly keeping notebooks. It’s just a bit more “digital” then we have been taught.

Do we know how to do this? Yes, pretty much. This is not a technical exercise, but one of social change and particularly willingness. Basically, if you want to keep the current way of doing things, the declare you want unreproducible, low quality research reporting. That’s your academic freedom, of course. If I were a funder or a university, I would also expect a bit more in return for my money.

Let me stress, glossy articles are fine! You do not have to stop that. Media appearances, key notes, these are all also fine. They are, however, complementary. We should not continue the habit of fancy narratives as replacement for quality research dissemination. Do both, if you must.

This post was originally posted as https://chem-bla-ics.blogspot.com/2023/07/journal-rankings.html.