![\"\"](\"/assets/images/jats_analysis.png\")
About fifteen months ago a new project started: One Million IUPAC names:
\n\n\n\n\nThus, the idea came up, can we create a set of 1 million unique IUPAC names found in literature?
\n
We started out with using Europe PMC to get JATS XML files for the full texts of open access articles.\nParsing the XML is easy and the text paragraphs are passed through OSCAR and OPSIN. That has not changed.
\n\nWhat did change last weekend is something I had long on my todo list (but life interfered). The first approach\nwas to ask for named entities using the Europe PMC APIs. But I quickly realized that with OSCAR and OPSIN we could\nget more names out of the articles. The next step was to move from Google Colab to a command line script.\nThat gave another boost, as explained in this second post in the series.\nWe reached 200 thousand names in june 2025\nbut then things slowed down again in the growth. Two months later\nwe only had 75 thousand more. However, plenty of discussion was happening and there turned out to be\nother, larger collections of IUPAC names under an open license. Millions of names, actually.
\n\nBut another problem emerged. We were still using the Europe PMC API and were basically asking for open access\narticles between two dates. Practically, the API could answer requests between 1 and max 3 days. Beyond that,\ntimes outs and 404s became an issue. Moreover, because these dates are publications dates and not the dates\non which the JATS were deposited, I had to got back to previous months and redo the queries. That gave another\n5 thousand names since last August. Something had to change.
\n\nEurope PMC, however, also provides the JATS XML files as download on their FTP site.\nAlready that august 2025 I had\na prototype and knew it would change the game. These gzipped XML files are about 150 to 250 MB. Unzipped, about 1 GB each.\nBetter, these files are based on Europe PMC identifiers, hopefully resolving the issue with using dates in the queries.
\n\nNow, parsing a 1 GB XML files is a total non-issue. I have done it plenty of times before. Just use a\nSimple API for XML (SAX) parser. This is a streaming parser\ngiving you full control of how to parse things. It is ideal for this siutation: you just keep the current\nparagraph of text in memory and release that when done with that paragraph. That is, you do not have to read\nthe full file in memory, just the bits you are interested in. I used this for my Chemical Markup Language\npatches for Jmol and JChemPaint back in the nineties.
\n\nLast weekend I finally made the jump. Use SAX to extract the <p> elements one by one, running OSCAR on\nthem, filter with OPSIN, output that name, and clear the memory. Effectively, each gzipped file processes\nwith a Groovy script in about 1 to 2 hours.
The output is a mesmerizing stream of scientific literature (which I will use until someone points me to a Java\nCLI library that creates a Matrix-style falling letters equivalent), tho less so as a static image:
\n\n
In this plot, an x means a new article to be processed. Each . and o that follows is a single <p>\nelement and the difference is that an o means at least one IUPAC name was detected in the paragraph.
Each gzipped file gives 400 to 500 new IUPAC names. Indeed, going from 288 thousand to 300 thousand\nwas a matter of a day and a half. And earlier this afternoon we passed the 400 thousand IUPAC names.\nWith about 230 gzipped files. Now, I am going back in time, and the sizes of these files are shrinking:\nAnother 500 files and the size has dropped to around 125 MB, so a rough estimate suggests that\nwe will end up with 650 to 700 thousand names this way. This will be completed in a few weeks (and mostly\nbecause I need to focus first on other things again, because I can use our computing cluster do this).
\n\nRegarding the original goal, fortunately, we are still publishing at a higher rate every year, and\nmore and more articles are available as open access. So, I still have good hopes we will reach the\n1 million IUPAC names. Also, keep in mind, we know how to boost this by simple name variations to\nseveral millions, even with the 400 thousand\nwe have today.
\n\nOh, and our next milestone will be in the pocket\nbefore I visit Christoph Steinbeck’s cheminformatics team in Jena!
", "summary": "About fifteen months ago a new project started: One Million IUPAC names:", "image": "https://chem-bla-ics.linkedchemistry.info/assets/images/jats_analysis.png", "date_published": "2026-05-02T00:00:00+00:00", "date_modified": "2026-05-02T00:00:00+00:00", "tags": ["iupac","textmining","xml","europepmc"], "authors": [ { "name": "Egon Willighagen", "url": "https://orcid.org/0000-0001-7542-0286" } ] } ] }