I wanted to see what I could recover, and here I describe what I did and what could be done next.

A list of all articles

The first step is actually to create a list of all articles published in the IJC and collect as much metadata about them as possible. With just over 100 articles, I decided to use Wikidata, as a machine-readable database, supporting the curation and reporting. I wanted at least two independent sources, and for Wikidata, use public resources. That means, while Web of Science does have a list of all articles, I only used this for validation, and not as information source. Instead, I used citations to IJC articles and, of course, the Internet Archive (IA). It turns out a query like this does wonders (well, for the abstracts; I did not find full-texts archived on IA):

https://web.archive.org/web/*/http://www.ijc.com/abstracts/*

I found that all but one article had the abstract archived in the IA. Here’s an example:

This gave my a lot of information to add to Wikidata. Title, publication date, volume, article number, keywords, an absstract, and, of course, the list of authors. Some authors I know personally, many I did not. But it did allow me to enter all articles to Wikidata along with the authors and “author” (P50) or “author name string” (P2093).

The article authors

It also turned out that multiple authors listed their IJC article on their public ORCID profile. That greatly helped identification. I managed to link many authors to mostly existing Wikidata items:

I already mentioned that I used Wikidata to collect this information. Besides the interactive visualization with Scholia, it also gave me the option to track my progress with SPARQL queries. For example, this query helped me do that author FAIR-ification:

You can see here two columns with author information, one for P50 and the other for P2093. There is quite some identification left to be done, and additional information is welcome:

Sources

So, that brings us to this list of sources:

• Internet Archive: abstracts and metadata
• ORCID profiles: ORCIDs of (some) authors
• Google Scholar: metadata and citations
• Web of Science: independent list for external validation

Because there is plenty of work left to be done and I hope the collected information will further spread in library collections, I added sources as much as possible. This query lists for all articles the Web of Science identifier (recorded so that everyone can check the consistency), the link to the Internet Archive-d abstract page, and a link to a known full text (five).

If you wonder, neither OpenAlex or Europe PMC have a full list.

What’s next?

I do not have a formal training in archiving, but I am happy with the minimal viable metadata collection. I know more can be done (and love to hear your pointers and suggestions): more author identies, better coverage of keyword annotation, etc. But I think an important addition is adding citations to and from the IJC articles are important. The journal predates efforts like the I4OC and Open Citations, so I may have to manually recover citations from Google Scholar. I will have to report on that later. But you can enjoy the citations that are already there. And now that we have sufficient metadata, I can use this to find more full texts.

Btw, I have made contact with Prof. Steven Bachrach, who founded the journal and was the Editor-in-Chief.

But in the most recent iteration, triggered by some work for WikiPathways, I was using Europe PMC to find articles that mention WikiPathways and then search in the full text for the string WP, as a trigger for the possible mention of WikiPathways pathway identifiers, which look like WP4846. The use of compact (resource) identifiers (see doi:10.1038/sdata.2018.29) is minimal, but at least some articles use identifiers.

That allows me to extend our WikiPathways knowledge graph of articles citing specific pathways. At the time of writing, we collected 2509 citations from 440 different articles to 883 different pathways. Now, I want to blog about that more, but it’s related to an observation.

Information loss

Now, back in the late ninities I learned about GNU/Linux and after playing with Red Hat and Suse, I settled for Debian. One of the things I learned is that, generally, information corruption (like data loss) is an absolute red flag, a no-go, a total showstopper.

And then we have this in publishing, the one area where data corruption must also be a no-go:

In this image, the left side shows a screenshot of the publisher version of the article and on the right side the version in Pubmed Central (PMC). PMC has been an important project to archive full text versions of articles:

11.2 million articles are archived in PMC.

So, this is really bad! The archived version is not really useful. As a human I already struggle to read the degraded image, let alone an algorithm.

Does that matter? Yes, projects like the awesome Pathway Figure OCR (see doi:10.1186/s13059-020-02181-2) depend on images to be FAIR enough to extract information. (Side note: yes, these images should be vector graphics, but commercial publishers decided about twenty years ago that they could not care enough.)

At this moment, I do not know where the information is lost. Maybe PubMed Central is storing the images in a low resolution. Maybe the publisher provides PMC with a low resolution image. But to me, this must be solved as soon as possible. This is utterly unacceptable.

I wonder what the authors of the article (doi:10.1186/s13287-025-04166-z) I took as example think of this.

Mark Dingemanse wrote up a post [i]n praise of niche papers, suggesting people to highlight papers that are not cited enough (as proxy for not getting enough attention). They write:

Let’s define niche papers informally as work to be proud of even if it managed to remain a bit obscure; good work that would deserve more readers. Niche papers may not contain the most flashy results. They may not appear in the most glamourous venues. They may be book chapters. They don’t easily gather drive-by citations.

Why I found this post interesting

Before I move on to highlighting niche papers (from our group and from others), I want to ponder a bit more about the rest. The first I learned is that the citation count to articles is a bad measure for the impact (2006 pondering): articles using your work may get more citations than your own article. For example, the first paper (doi:10.1021/CI025584Y) about the open science cheminformatics about the Chemistry Development Kit (CDK) was originally cited less than the paper about the BRENDA enzyme database (doi:10.1093/NAR/GKH081) using the CDK for fingerprint calculations (to compare and search enzyme substrates), and later much less than MZmine (doi:10.1186/1471-2105-11-395) (see this Scholia page):

I think we should with limiting ourselves to papers and book chapters. We must extend out notion of research output, anyway, starting with data and software. This is part of defining what niche is, imo.

Second reason why I liked Mark’s post is the drive-by citations, which he references to a 2009 post by andrewperrin which defined such a citations as

references to a work that make a very quick appearance, extract a very small, specific point from the work, and move on without really considering the existence or depth of connection between the student’s work and the cited work.

This is someone I noted too when analyzing citations to the aforementioned CDK paper. Particularly in the early days, it was cited a lot in a similar way: it was not using the CDK, but ascribed some authority to the paper in a very quick appearance, without really considering the cited work. The Citation Typing Ontology (CiTO, doi:10.1186/2041-1480-1-s1-s6) has cito:citesAsAuthority for that (not exactly the same thing, and maybe CiTO should have cito:driveByCitation too). And they happen a lot, and in the past I have guestimated them to make up 20-35% of the citations to an article, and I postulate that high-journal-impact-factor journals amass a higher ratio than specialistic (niche?) journals.

With FAIR citations (see this post) we can visualize that ratio, here in this Scholia page:

It is also obvious that the first CDK paper introduced a new method. But the pattern is not limited to this paper, and with just over 2000 citation intentions, we start of get some idea of this pattern:

My contributed Niche Papers

That brings me to a first neglected paper, David Shotton’s original conference proceedings CiTO, the Citation Typing Ontology (doi:10.1186/2041-1480-1-S1-S6), another paper where citing articles are more cited than the original:

A second example is cited even less (only 36 times according to OpenAlex), but a wonderful early example of machine learning of a massive amount of data: Genome‐Scale Classification of Metabolic Reactions: A Chemoinformatics Approach (doi:10.1002/anie.200503833) by Diogo Latino and João Aires‐de‐Sousa. My 2006 blog post about their article did not make a difference. And this is remarkable if you look at home many articles are published now yearly in similar efforts.

From our group, I think the impact of Ryan Miller’s Understanding signaling and metabolic paths using semantified and harmonized information about biological interactions (doi:10.1371/journal.pone.0263057 is not fully appreciated yet. This paper describes and validates work by Ryan, Martina Kutmon, Answesha Bohler, and Andra Waagmeester on modelling biological interaction in a FAIR way. It builds on earlier work, like the WikiPathways RDF work by Andra (doi:10.1371/journal.pcbi.1004989), but zooms in on the interactions and develops method to assess the quality of the FAIR modelling of them. This provides us with a method to evaluate later analyses where these interactions are used.

A second paper from our group which I expected to get more attention is a paper by Ammar (doi:10.1186/s13321-023-00701-3) where he looked into personalized binding affinities. That is, drugs may bind better to their targets for some people than for other (and therefore work better for some people than for other), and his analysis suggests they impact can be significant. We will learn in time.

Things are coming together, and while commercial publishers (SpringerNature, Elsevier, MDPI, Frontiers, etc) are focused on profit (“shareholder value”) instead of the community they serve, Open Science is providing working, real-world, inexpensive, superior FAIR solutions for scientific dissemination. Maybe European universities are not convinced yet (see Björn’s post), but it is happening.

Two things that are happening are OpenAlex and OpenCitations. CiTO adoption not so much yet, but I am not giving up yet. Simply because Open Science doesn’t go away and everything can be picked up tomorrow. Each holiday I am picking up the Citation Typing Ontology and this holiday the use of nanopublications for CiTO intent annotation of April this year.

Yesterday, I played with the nanopublication templates used by NanoDash, got to the triples of it, and ended up using the web interface to create a derived template from Tobias’ template from April. What makes this nanopublication template special is that it uses the CiTO ontology.

The difference is that the original template used ScholarlyWork as type for the citing resource, while the derivative uses CreativeWork from the schema.org namespace, allowing things like this:

• article to software release: example nanopub
• article to blog: example nanopub
• blog to article: example nanopub
• blog to blog: example nanopub

The last three are possible because of the Rogue Scholar DOIs. Let’s continue with the fourth example, the blog to blog citation. While an URL is a unique, global identifier, the digital preservation depends on a lot of things. On the other hand, a DOI with the associated metadata is easier to preserve. For example, because it can be spread more easily than the digital object itself.

So, when Martin Fenner and I started archiving the Depth-First blog of Rich Apodaca to digitally preserve his blog, it also automatically gave the blog posts DOIs. This makes the blog more FAIR, just like it does for my blog. And being more FAIR, we can use the DOIs for other things too, like blog to blog citations with CiTO intent annotation, as nanopublications. (Technically, any Springer Nature journal can do this, but they found reasons to not do it.)

So, let’s take this blog post. I have today updated this to not use depth-first.com URLs but, following Martin’s example, use the DOIs for those posts instead.

And when I make a nanopublication out of this, I can add the citation intent, and then it looks like this:

For some reason, the DOIs do not show up as references as they do for this post for the DOIs of the posts of Martin Paul Eve, Björn Brembs, and Martin Fenner. It does for this post citing Depth-First.

So, from now on, I will use DOIs when citing other blog posts, and I hope many other blogs will start using Rogue Scholar or some other service to generate DOIs for single blog posts. I also have to figure out if I want to use DOIs to link to posts in my own blog. And hopefully, OpenCitations will soon accept citations provided by nanopublications. With or without CiTO intent annotations, whatever comes first. Oh, and I cannot wait to see the citations who up in Altmetric.com :)

Let’s see where this is going.

So, with “version of record” defined, why do we have only one in publishing?

There is absolutely no reason not to have multiple versions of the same narrative, as long as they are clearly tagged. Open Science has been doing this for two decades, but publishers have been slacking. Retractions are updated versions of the same article, as are corrections, corrigendum, and errata. It is not that publishers do not know how to do it. Hesitently, they are accepting that preprints exist, but publishers tend to frame that as inferior versions. There was a paper earlier this month that looked into how much the versions are really different, and when I find it again, I will add the link.

Of course, money, control, power likely have a role here. And historic reasons too, I guess. After all, when you have to print a journal issue and send them by horse carriage to universities around the world, making updates is indeed not trivial.

Twitter or X (or Mastodon or Bluesky)

But as an openscientist, I have the urge to keep research output relevant. We do this for data, we do this for community standards, and we do this for research code. Routinely. Again, for decades. Must open access not learn from open science here?

I asked that recently on Mastodon. Specifically, should the Ten simple rules for getting started on Twitter as a scientist article (doi:10.1371/journal.pcbi.1007513) not be updated? Looking at the number of scientific papers that discuss social media in scientific communication, ten tips sound to me to be timeless. And I was interested in why or why-not the paper should be updated. Content-wise, a trivial update would be to update the name to X, which is the name of what was formerly known as Twitter. But then, updating the paper to replace Twitter by Bluesky or Mastodon would not be that much work either.

The discussion brought up various aspects of this question (and hereby thanks to all who joined the discussion!). Is it worth it? Is it legal? Should it be an update, or just a new paper? Who should do it? Do scholars have some responsibility to keep there research relevant? If I string-replace Twitter with X, how do I make clear who the original authors are, and what my role is? How do we get PLOS to point to the new version (surely not as corrigendum)? I do not have the answers.

But I do see differences between different types of research output and that makes these question an essential part of Recognition and Rewards. If it some types of output have different rules, then we do not give all scholars the same chance of recognition. Of course, this is the current situation, and just reflects that academia still has much to do to adopt Open Science.

All of these depend on the publisher to support these efforts. But the big publishers are not good at this (see also doi:10.5281/zenodo.4926031https://doi.org/10.5281/zenodo.4926031) and/or prefer to make 20-30% profit first.

This opens room for innovative publishers. We have F1000Research pushing open peer review, and PenSoft pushing a new editor, Overleaf bringing collaborative online editing of LaTeX, [Qeios] experimenting with a wider range of output types, and the Journal of Open Source Software (JOSS) pioneering a more open platform for the whole editing process.

And, of course, we have Diamond Open Access publishers that do not get enough visibility, like SciPost and Beilstein for natural sciences and JOSS for open source.

Open Journal Systems

And there is the Open Journal Systems (OJS), another editor manager, one that has been around for some time now. We use OJS for the CDK News newsletter. Big news this week was that OJS has been selected as infrastructure to underping the Open Research Europe publishing platform, something running on F1000Research, bought up up Taylor&Francis in 2020.

I need to catch up with where the OJS is technically. Do they support Markdown submissions? Do they export JATS? Do they support CiTO annotations? But this needs editors and journals to expect these things. Unfortunately, many journals have a limited expectation of digitial knowledge dissemination, and it’s still PDF galore.

Better Publishing

This brings me to the following: should the Dutch universities continue to fund the publisher business, stakeholder profit, or should we invest in open infrastructure to benefit our own core business: research and education. I think you understand what my position is on this. The current big deals we have with the big publishers are not actually really in our benefit and with the upcoming defunding we have to use every euro carefully. And then I prefer to fund a young researcher instead of publisher stakeholders.

I hope you are willing the read the following petition to the Dutch negotiators to very carefully consider what their priorities are and who they represent. You can sign anonymously (if you fear backslash) and you can just read the details behind this well-written petition: there are many references at the bottom to support the statements I make here, and more.

But I really, really hope you wish a better future for knowledge dissemination. Just think of your next Reviewer 2, that you pay the publisher to have Reviewer 2 scold at you, or the time spent on reference formatting, just because the publisher prefers profit over usability.

Join and sign!

So, in Wikidata, you can capture this like this:

So, what defines the quality of a journal? Or better, of any scholarly dissemination channel? After all, some databases do better peer review than some journals. Sadly, I am not aware of literature that compares the quality of peer review in databases with that in scientific journals. Also long overdue, in my opinion.

I hope the Open Science community will help shape these scholarly dissemination channels, journals included. Some ideas, the outlet:

• encourages post-publication peer review
• communicates the post-publication peer review
• allows updating easily small fixes and clarifications (no hiding behind the version-of-record)
• ensures supp info / additional files undergo the same level of peer review
• use modern solutions for communication (like semantic web technologies)
• have clear licenses for all aspects of the research output
• actively fight against visual representation only, but provides all data
• guarantees that supp info / additional files are archived, as the output itself
• adopts, promotes, requires community standards (including global, unique identifiers)

Okay, these items are pretty broad. Many of them are part of FAIR, but that should not surprise you, because FAIR are just applying traditional scholarly approaches, like properly keeping notebooks. It’s just a bit more “digital” then we have been taught.

Do we know how to do this? Yes, pretty much. This is not a technical exercise, but one of social change and particularly willingness. Basically, if you want to keep the current way of doing things, the declare you want unreproducible, low quality research reporting. That’s your academic freedom, of course. If I were a funder or a university, I would also expect a bit more in return for my money.

Let me stress, glossy articles are fine! You do not have to stop that. Media appearances, key notes, these are all also fine. They are, however, complementary. We should not continue the habit of fancy narratives as replacement for quality research dissemination. Do both, if you must.

One of these latter two was not a more traditional paper, but a different kind of research output: a definition, about “Drive-by Curation” (doi:10.32388/KBX9VO). Now about this output type, collaboratively working on definitions is something core to ontology development (e.g. see doi:10.1186/s13326-015-0005-5), but there is a clear need to discuss terminology. The GRACIOUS project in the EU NanoSafety Cluster also recognized this and set up a tool for this, their Terminology Harmonizer (doi:10.1016/j.impact.2021.100366).

This GRACIOUS tool, much more than what Qeios does, helps users. Unfortunately, and why how these topics nicely come together, writing definitions, thinking about when some zeta potential is different from another zeta potential, and the (drive-by) community curation, it needs transparency. I understand it, but landing on a login page is for me a recipe for a silent death as it disallows people to learn, without making an (time) investment first. That is what Qeios does differently: it is more FAIR.

So, that brings me to my last point in this post. Jente Houweling and I wrote up a definition for “Research Output Management” (doi:10.32388/ZNWI7T), based on our discussions about her research insights. See the screenshot below.

It has been reviewed internally, and by one independent peer (doi:10.32388/C3SJTN). But we would love to hear your review too. Just follow the instructions online. We are looking forward to reading your thoughts and to refining our definition.

Linkrot, the phenomenon that URLs are not persistent, has been studied, including the in scholarly settings (see 1998, 2000, 2003, 2006, 2008, 2014, 2015, 2000, 2021, and probably many more). Indeed, scholarly publishers started introducing the following: URLs should be accompanied with an “accessed on” statement. Indeed, you can find this in many bibliographic formatting standards.

Indeed, this must change, and we already have a solution since 1996: the Internet Archive (tho the archive goes back much longer). I call all publishers to change their “Accessed on” to “Archived on”. Two simpel solutions that can compliment each other:

Authors archive upon submission

This solution is simply introduced by updating author guidelines. Surely it will take a bit of time for bibliography software to be updated, and for the time being we still write “Accessed on” until there is proper support of “Archived on”.

Journals archive upon acceptance

This solution looks for all URLs in journal articles and archives them. It doesn’t matter if the author already did this, because the Internet Archive has no trouble handling this:

BTW, projects like Wikipedia have automated the process of archiving URLs and I see no reason why publishers could not do this.