Jekyll2026-04-19T09:50:36+00:00https://chem-bla-ics.linkedchemistry.info/feed/by_tag/wikidata.xmlchem-bla-icsChemblaics (pronounced chem-bla-ics) is the science that uses open science and computers to solve problems in chemistry, biochemistry and related fields.Egon WillighagenSWAT4HCLS 2026 Amsterdam this week2026-03-22T00:00:00+00:002026-03-22T00:00:00+00:00https://chem-bla-ics.linkedchemistry.info/2026/03/22/swat4hcls-2026-amsterdam-this-weekTomorrow, SWAT4HCLS 2026 will start, again in Amsterdam. The first SWAT4LS I attended was also in Amsterdam, and the second meeting in Amsterdam I was also there. And I was in Cambridge (see this post), Antwerp (no post), and at least to one of the two Leiden meetings (also no posts, it seems).

I am looking forward to meet old friends, new friends (some whom I never met in person), and recent collaborators (that I never met in person). For those who will not be in Amsterdam, you can follow the meeting on social media with the hashtag #swat4hcls. And there is also this BioHackrXiv Fediwall, for those in the fediverse.

Scholia demo

I will give a demo to update people on the work in the Scholia project with Daniel Mietchen, Peter Patel-Schneider, Konrad Linden, Johannes Kalmbach, Lars Willighagen, Wolfgang Fahl, and Hannah Bast (also keynote in Amsterdam) to update the SPARQL queries we use to visualize data in Wikidata to SPARQL 1.1 so that it can run on Qlever. The abstract can be found in Wikimedia Commons.

This was the outcome of many years figuring how to ensure Scholia could remain working. The Wikidata RDF graph split has given us many headaches, so many that just before christmas it became it could be possible to survive the split, I was so happy, I realize I want to share this news. So, we teamed up and wrote this demonstration contribution abstract. Thanks to everyone who made this happen! Just to be clear, we are not done yet. The system is not running outside the Wikimedia Foundation platforms.

One of the reviewer comments requested a Scholia page for the meeting. It has not been updated for the accepted speakers, but you can look at pages for past meetings to get an idea what you will find.

SWAT4HCLS Biohackathon 2026

There will also be a biohackathon again, of course, with the option for BioHackRxiv reports. There are already several pitches, including one that I submitted about Scholia.

]]>Egon WillighagenRescuing Scholia #2: getting closer2025-12-31T00:00:00+00:002025-12-31T00:00:00+00:00https://chem-bla-ics.linkedchemistry.info/2025/12/31/rescuing-scholia-2-getting-closeThree weeks ago, I wrote a the post Rescuing Scholia: will we make it in time?, where I sketched a future without Scholia. Scholia, started almost 10 years ago and I think it is worth keeping around longer.

Fortunately, it looks like we will have a working replacement in time before the WDQS instance with all the Wikidata triples in a single SPARQL endpoint goes down, likely in a week or so (even tho we may be behind the citation peak).

The work of the past year helped, for exampe, making it easier to configure Scholia for a different endpoint and the asynchronous loading of panels (reducing the stress on the SPARQL end point). Already in September, Prof. Hannah Bast started a branch for the transition and various hackathons this autumn, and the work by Konrad Linded who explored and addressed some of the hurdles to take. The tips and suggestions from Hannah and RobinTF really made a difference. And also a huge thanks to Daniel who kept relentlessly pushing this forward.

When I posted my will we make it post, there was a demo instance and a spreadsheet showing the state of each query. The instance showed no human-readable labels. This was because the WDQS wikibase:label service was used a lot, and there is no replacement for that. Getting labels for all relevant items is possible, but makes the queries a lot heavier and made even more queries run out of memory. Various solutions were discussed, Finn indicated he preferred a macro solution, which Lars implemented, and saw some tweaks after that. Then followed a long series of patches by particularly Peter to update all the SPARQL queries to have them use the new labels macro. But plenty of other things were fixed or newly implemented, such as Wolfgang’s /backend page.

So, with one week to go, we need your help: as the weekly Wikidata Status Update already indicated:

this month’s Scholia hackathon has moved Scholia closer to its planned switch to a QLever backend. Beta testers can assist by exploring the interim QLever-backed Scholia instance and reporting any issues.

And thanks to Adriano and others who already have!

Now, we are not done yet. The real instance at scholia.toolforge.org has seen ridiculous abuse by scrapers (and the main instance is regularly unusable, to be honest), and we have no idea the new setup is powerful enough. And we need to point to the new servers anyway. So, plenty of work is left to be done in the next few days.

But we are getting close. So, please give qlever.scholia.wiki a go, and let us know your observations. As Linus’s law writes:

Given enough eyeballs, all bugs are shallow.

]]>Egon WillighagenRescuing Scholia: will we make it in time?2025-12-08T00:00:00+00:002025-12-08T00:00:00+00:00https://chem-bla-ics.linkedchemistry.info/2025/12/08/rescuing-scholiaWhat started out in 2016 on Twitter became a (small) award winning decade long collaborative project. Unfortunately, the future is not clear. We are at odds if it will survice the growth of Wikidata and in particularly the SPARQL graph split. To be clear, the choice for Blazegraph initially worked great, but after it was bought by a big company, developed halted. Very unfortunate for Wikidata. Unlike earlier, we no longer have funding, and rewriting Scholia at this scale takes a good bit of effort. We already held a few hackathons.

So far, we have been able to continue to use a legacy SPARQL endpoint with all the data, but in exactly one month that endpoint will be sunset. And we are not ready.

Rescuing Scholia

Daniel and Lane have been leading an effort to rescue Scholia. The hackathons were part of this effort. It seems that QLever is the only route left. Earlier efforts to rewrite the more than 350 Scholia SPARQL queries to support the graph split have basically failed. The complexity is far too high. QLever, however, provides the full graph and since recently full SPARQL 1.1 support. That is also not enough to reproduce the full Scholia functionality, but it seems to get us far. Importantly, the data may not update as frequently as the WDQS, and that is another complexity to take into account. Particularly, all the 404 pages.

So, in the next weeks, we have to complete rewriting all those queries as queries that QLever can handle. A team of people have done great work already, including Konrad.

I hope we make it in time.

]]>Egon WillighagenThe Internet Journal of Chemistry2025-08-11T00:00:00+00:002025-08-11T00:00:00+00:00https://chem-bla-ics.linkedchemistry.info/2025/08/11/the-internet-journal-of-chemistryThe Internet Journal of Chemistry (IJC, issn:1099-8292) was one of the first scientific journals to get published on the world wide web (part of the Internet), see doi:10.1080/00987913.2000.10764578. Issues were published from 1998 to 2004. But because it predates systematic archiving of webpages by libraries, a lot is lost. The nature of the journal, however, makes it unique, and quite a number of articles are cited a lot, and should be part of the scientific record. But I soon realized it actually is quite hard to track down content of the journal. I knew some articles have been author accepted manuscripts online. One of that was my own first (and single) author-article, self-archived on Zenodo (doi:10.5281/zenodo.1495470), green open access style.

I wanted to see what I could recover, and here I describe what I did and what could be done next.

A list of all articles

The first step is actually to create a list of all articles published in the IJC and collect as much metadata about them as possible. With just over 100 articles, I decided to use Wikidata, as a machine-readable database, supporting the curation and reporting. I wanted at least two independent sources, and for Wikidata, use public resources. That means, while Web of Science does have a list of all articles, I only used this for validation, and not as information source. Instead, I used citations to IJC articles and, of course, the Internet Archive (IA). It turns out a query like this does wonders (well, for the abstracts; I did not find full-texts archived on IA):

https://web.archive.org/web/*/http://www.ijc.com/abstracts/*

I found that all but one article had the abstract archived in the IA. Here’s an example:

This gave my a lot of information to add to Wikidata. Title, publication date, volume, article number, keywords, an absstract, and, of course, the list of authors. Some authors I know personally, many I did not. But it did allow me to enter all articles to Wikidata along with the authors and “author” (P50) or “author name string” (P2093).

The article authors

It also turned out that multiple authors listed their IJC article on their public ORCID profile. That greatly helped identification. I managed to link many authors to mostly existing Wikidata items:

I already mentioned that I used Wikidata to collect this information. Besides the interactive visualization with Scholia, it also gave me the option to track my progress with SPARQL queries. For example, this query helped me do that author FAIR-ification:

You can see here two columns with author information, one for P50 and the other for P2093. There is quite some identification left to be done, and additional information is welcome:

Sources

So, that brings us to this list of sources:

  • Internet Archive: abstracts and metadata
  • ORCID profiles: ORCIDs of (some) authors
  • Google Scholar: metadata and citations
  • Web of Science: independent list for external validation

Because there is plenty of work left to be done and I hope the collected information will further spread in library collections, I added sources as much as possible. This query lists for all articles the Web of Science identifier (recorded so that everyone can check the consistency), the link to the Internet Archive-d abstract page, and a link to a known full text (five).

If you wonder, neither OpenAlex or Europe PMC have a full list.

What’s next?

I do not have a formal training in archiving, but I am happy with the minimal viable metadata collection. I know more can be done (and love to hear your pointers and suggestions): more author identies, better coverage of keyword annotation, etc. But I think an important addition is adding citations to and from the IJC articles are important. The journal predates efforts like the I4OC and Open Citations, so I may have to manually recover citations from Google Scholar. I will have to report on that later. But you can enjoy the citations that are already there. And now that we have sufficient metadata, I can use this to find more full texts.

Btw, I have made contact with Prof. Steven Bachrach, who founded the journal and was the Editor-in-Chief.

]]>Egon WillighagenPFAS in the blood of the Dutch population2025-07-06T00:00:00+00:002025-07-06T00:00:00+00:00https://chem-bla-ics.linkedchemistry.info/2025/07/06/pfas-in-the-blood-of-the-dutch-populationA recent report by the Dutch RIVM, PFAS in the blood of the Dutch population (doi:10.21945/RIVM-2025-0094), writes that seven PFAS compounds are found in blood samples of all tested people. Another nine compounds are found in at least 1-in-10 people. Because there is relevant data in the report on the 28 studied PFAS compound, I wanted to have the report more FAIR than it is on the website. Why this report? Well, the chemistry and the history is fascinating and brutal (I like this Veritasium video).

The history tells me that our society may sound woke and leftish, in reality it is a continous fight for basic human rights. (Something that plenty have been saying for years.) In this case, a healtht life is the human right.

So, what can I do to make this report more FAIR?

Findable in Wikidata

Since this report has been mentioned in the news, it clearly is notable. The simplest thing to do is thus to just add it Wikidata. Because the DOI of the report had not been recorded yet, I could not let Scholia do it for me. But doing it manually is only a bit more work: Q135222054. The provided metadata on the RIVM website is minimal.

But we can do more. Particularly, because I want people to find this report when they look info knowledge about the 28 studied chemicals, I added main subject annotation using the information in Table 1 in the report. Using Scholia and the CAS registry number in the table, I crosscheck the information in Wikidata is consistent with the report (and visa versa). It was. I then added the Dutch name and acronym for most of them. Some already had the name as in the Table. That gives us a nice “Topic scores” plot for the Scholia page of the report:

The central PFAS bubble is also only one main subject but larger because many the specific PFAS compounds are subclassing PFAS. And you may also note many smaller bubbles. These actually come from main subject annotations of articles cited from the report. Because I added a few of them too. Not all, because many are not in Wikidata (yet).

Findable in WikiPathways

But since 16 of these compounds are readily found in human blood samples, that is handy knowledge when doing metabolomics (on blood samples). Or (and I leave that to later blog post), we can map the experimental data for Dordrecht versus the rest of The Netherlands to the PFAS compounds. That is relevant to research by VHP4Safety. There are many ways to see if you have PFAS in your dataset, but since we have many controlled lists of genes in metabolites, I added one for common PFAS in human blood samples. Well, the 16 common in Dutch blood samples:

Each metabolite here is annotated with their Wikidata identifier, allowing us to map experimental data on top of it. And we get links out to other databases almost for free:

And the link to Wikidata actually links to Scholia, so for the PFOA in the above example, we can quickly see the boiling point, decomposition point, and melting point of this PFAS. And literature with undoubtedly even more knowledge about this PFAS:

Now, these two steps were mostly manual: drawing WP5579 in WikiPathways and adding the report annotations (main subject and cites) in Wikidata.

Findable in the VHP4Safety Compound Wiki

As part of the VHP4Safety project, I am collecting information on chemicals studied in the context of toxicology, safety, and risk assessment. Often specific collections of compounds studied as a whole. This report is such a collection and provides experimental data on these compounds. So, I want this report to be findable for the VHP4Safety Compound Wiki too. Creating the collection is a manual step: Q5145.

Now, because both Wikidata and our VHP4Safety Compound Wiki (a Wikibase instance) are semantic and support, I can use SPARQL to create instructions to link the 28 compounds to the new collection. Now, arguably, that can be done manually too, and maybe faster, for larger collections this is harder. So, I dug up my earlier notes and got some useful things together.

This query lists all 28 PFAS linked to the report in Wikidata:

SELECT ?pfas ?pfasLabel WHERE {
  wd:Q135222054 wdt:P921 ?pfas .
  ?pfas wdt:P31 wd:Q113145171 .
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],mul,en". }
}

Using federation powers, I can use this for a SPARQL query, and return the results in QuickStatements that say this VHP compound is part of the VHP collection:

PREFIX wb: <https://compoundcloud.wikibase.cloud/entity/>
PREFIX wbt: <https://compoundcloud.wikibase.cloud/prop/direct/>

SELECT (SUBSTR(STR(?cmp),45) AS ?qid) ?P21 WHERE {
  ?cmp wbt:P5 ?wikidata .
  SERVICE <https://query.wikidata.org/sparql> {
    wd:Q135222054 wdt:P921 ?pfas .
    ?pfas wdt:P31 wd:Q113145171 .
    BIND (substr(str(?pfas),32) AS ?wikidata)
  }
  BIND ("Q5145" AS ?P21)
}

I actually had to add 5 PFAS compounds in the VHP4Safety Compound Wiki first. That follows the same procedure for how I have been adding chemical compounds to Wikidata (see also this preprint). The input cas.smi has the (missing) SMILES, Wikidata QID, and English label:

C(CS(=O)(=O)O)C(C(C(C(C(C(F)(F)F)(F)F)(F)F)(F)F)(F)F)(F)F       Q27063662       6:2 Fluorotelomer sulfonate
CN(CC(=O)O)S(=O)(=O)C(C(C(C(C(C(F)(F)F)(F)F)(F)F)(F)F)(F)F)(F)F Q126605979      MeFHxSAA
CN(CC(=O)O)S(=O)(=O)C(C(C(C(F)(F)F)(F)F)(F)F)(F)F       Q126682412      MeFBSAA
C(=O)(C(C(F)(F)F)(F)OC(C(C(F)(F)F)(F)F)(F)F)O[H]        Q29387971       2,3,3,3-tetrafluoro-2-(heptafluoropropoxy)propanoic acid
C(C(C(=O)O)(F)F)(OC(C(C(OC(F)(F)F)(F)F)(F)F)(F)F)F      Q81981675       4,8-Dioxa-3H-perfluorononanoic acid

For reference, this is the command line I used to create QuickStatement instructions:

groovy createWDitemsFromSMILES.groovy -w compoundcloud.wikibase.cloud -c Q2368 -d P5 -l -i wikidata -a P11

Final remark

Are these 16 the only PFAS in our body? With 28 studied out of a potential seven million, I doubt it.

]]>Egon WillighagenRetracted articles cited in Wikipedia2025-06-08T00:10:00+00:002025-06-08T00:10:00+00:00https://chem-bla-ics.linkedchemistry.info/2025/06/08/retracted-articles-cited-in-wikipediaLast week, the Wikidata and Sister Projects event tooks place. The presentations are recorded, and I strongly encourage you to check the schedule. One presentation I liked (there are more), was the one by Mike Peel with the title “Best practices for reusing Wikidata’s data in the Wikimedia Projects”. At some point he walks us through the {{Cite Q}} template, around 26:07.

I learned that this template will highlight when an article cited in Wikipedia is actually retracted (withdrawn or replaced). Now, for the past months, I have been using the Crossref API to the Retraction Watch Database and annotated thousands of articles as retracted, using URLs from the database as reference. I use this script.

So, that means that this work actually has had a massive impact. Perhaps thousands of (English) Wikipedia readers have seen the results from running that script. That is pretty awesome! This is why we do open science.

But it made me also wonder something else. The Retraction Watch Database has over 60 thousand articles and Wikidata only about 22 thousand (at the time of writing). What if Wikipedia has an article not in Wikidata? Well, obviously, it cannot use {{Cite Q}}. But wouldn’t we want to have that article in Wikidata? Clearly, the article is notable; at least, in Wikipedia notability-sense. So, I was wondering, of those 40 thousand retracted articles not in Wikidata, how many are cited in English Wikipedia (to start with).

So, I wrote a first script that lists DOIs in the Retraction Watch Database (via the CrossRef API downloaded list) that are not found in Wikidata. A second script uses a Scholia (doi:10.1007/978-3-319-70407-4_36) SPARQL query develop by Finn Nielsen that uses wikibase:mwapi to do an efficient DOI search.

The results are fascinating and this is the list of DOIs found:

Searching 10.1016/j.engfailanal.2021.105457 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Filippo_Berto
Searching 10.1002/14651858.CD002291 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Antipruritic
Searching 10.1007/s12115-020-00496-1 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Lawrence_Mead
Searching 10.1515/9783110619768 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Dead_Eagle_Owl
Searching 10.1016/j.sbi.2022.102426 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Deborah_F._Kelly
Searching 10.1371/journal.pone.0240851 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Industrialization_of_China
Searching 10.1080/00927670309601525 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Rouben_Azizian
Searching 10.1080/14693062.2016.1179616 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Benjamin_K._Sovacool
Searching 10.1038/s41390-022-02127-3 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Harald_Walach
Searching 10.1001/jamapediatrics.2021.2659 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Harald_Walach
Searching 10.1109/CCECE.2007.335 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/List_of_scientific_misconduct_incidents
Searching 10.1002/14651858.CD001834 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Meningococcal_vaccine
Searching 10.1007/s11356-021-16530-6 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Kamran_Bagheri_Lankarani
Searching 10.1016/j.biortech.2023.129044 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Ashok_Pandey
Searching 10.1002/14651858.CD003614 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Trimetazidine
Searching 10.1002/14651858.CD003808 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Hall_Technique
Searching 10.1002/14651858.CD003747 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Venous_thrombosis
Searching 10.1002/14651858.CD003711 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Myocarditis
Searching 10.1002/14651858.CD003225 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Prevention_of_migraine_attacks
Searching 10.1002/14651858.CD003226 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Prevention_of_migraine_attacks
Searching 10.1002/14651858.CD003498 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/A2_milk
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Autism
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Autism_therapies
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Casein
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Casomorphin
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Causes_of_autism
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Gluten-free%2C_casein-free_diet
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Gluten-free_diet
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Opioid_excess_theory
Searching 10.1093/restud/rdy054 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Managerial_economics
Searching 10.1039/d1nr00388g in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Deborah_F._Kelly
Searching 10.1139/p90-116 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Canada%27s_Stonehenge
Searching 10.3791/64256 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Thomas_J._Webster
Searching 10.1016/j.biortech.2022.127565 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Ashok_Pandey
Searching 10.1016/j.anbehav.2015.04.001 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Stegodyphus_dumicola
Searching 10.1016/j.tafmec.2022.103573 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Filippo_Berto
Searching 10.1155/2022/7002630 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Dopamine_receptor_D3
Searching 10.1007/s11223-017-9884-2 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Filippo_Berto
Searching 10.1016/j.forsciint.2024.112115 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Peter_A._McCullough
Searching 10.1002/14651858.CD002778 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Temporomandibular_joint_dysfunction
Searching 10.1016/j.jcv.2022.105248 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Kay_Davies
Searching 10.1002/14651858.CD002916 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Bleomycin
Searching 10.1007/s11756-021-00841-7 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/2021_in_archosaur_paleontology
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Argentinadraco
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Wellnhopterus
Searching 10.4132/KoreanJPathol.2009.43.4.306 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Cho_Kuk
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Cho_Min_academic_credentials_scandal
Searching 10.1503/cmaj.80742 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/LetUsTalk
Searching 10.1002/14651858.CD004125 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Granisetron
Searching 10.1111/ffe.12616 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Filippo_Berto
Searching 10.1007/s00366-008-0118-x in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Kamran_Daneshjoo
Searching 10.1002/jmv.28097 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Noora_%28vaccine%29
Searching 10.1126/science.1070563 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Sch%C3%B6n_scandal
Searching 10.1246/cl.170853 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Enantioselective_Iridium-Catalyzed_C-H_Borylation
Searching 10.1016/j.marpol.2017.06.032 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Ray_Hilborn
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Tony_J._Pitcher
Searching 10.1016/j.ijfatigue.2021.106450 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Filippo_Berto
Searching 10.1016/j.crphar.2022 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Desidustat
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Emoxypine
Searching 10.1007/s10479-023-05261-1 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Artificial_intelligence_marketing
Searching 10.1257/aer.20210369 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Dividend_tax
Searching 10.1109/AIMSEC.2011.6010222 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/SAP_CRM
Searching 10.1002/14651858.CD001103 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Hydrocolloid_dressing
Searching 10.1007/978-3-642-27708-5 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/IP_Multimedia_Subsystem
Searching 10.1351/PAC-CON-08-12-06 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Parthenocissus_tricuspidata
Searching 10.1002/advs.202204315 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Tetrataenite
Searching 10.1353/sho.2011.0038 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Maus
Searching 10.1111/jpim.12058 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Ulrich_Lichtenthaler
Searching 10.1136/bcr-2021-241572 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/COVID-19_proxalutamide_trial_in_Brazil
Searching 10.1038/s44160-022-00068-7 in Wikipedia...
  ... found in this Wikipedia page: https://en.wikipedia.org/wiki/Single-layer_materials
]]>Egon WillighagenNew preprint: “Scholia Chemistry: access to chemistry in Wikidata”2025-05-25T00:00:00+00:002025-05-25T00:00:00+00:00https://chem-bla-ics.linkedchemistry.info/2025/05/25/new-preprint-scholia-chemistry-access-to-chemistry-in-wikidataTwo week ago I uploaded a paper that has been in the works for some time. In fact, I first mention it as conference paper for the special issue of the 11th International Conference on Chemical Structures, you know, the meeting held in 2018, of which the 13th edition starts in 7 days. I had a poster at that conference which I described in this blog post.

In turn, that poster described work of at least three years, going back to adding identifiers in 2015 and chemical structures in early 2016. I started using scripts two months later. This helped a lot with migrating pKa data from a custom Semantic MediaWiki installation to Wikidata and with adding thousands of EPA CompTox identifiers in 2017.

But that 2018 conference paper never happened. Because Scholia did. And even on the ICCS poster, Scholia was used to visualize chemistry data in Wikidata. To be honest, not just that, of course. About a year ago I had a serious go at finishing the paper, and it was sent around to co-authors. But I realized at the time, that the paper was lacking some good suggestions how the peer review our actual contributions to Wikidata. I could hardly expect readers of the paper browse the individual histories of all, by then, 1.3 million chemical compounds. And during the holidays I collected a few tools, which I had lined up to add to the manuscript.

However, another thing happened, the COVID-19 pandemic. While all the experience helped a lot with getting knowledge together around SARS-CoV-2, it also made something else clear: the software behind Wikidata does not scale well (enough). This lead to plans to split the RDF graph representation into two separate SPARQL endpoints. And that breaks many, if not most, of Scholia’s SPARQL queries, including those for the chemistry aspects. The situation in Summer 2024 was that there was a significant chance Scholia would not survive the split. And the Scholia Chemistry paper had to wait. You cannot publish an article of which the website is gone before it is formally accepted.

Let me make clear, this graph split is not solved and the risk is not gone. But a serious of unfunded, weekend hackathons allows us to refactor Scholia to give us a chance. It started with making Scholia more configurable. We had the first hackathons in October and November, and I had four more hackathon weekends this April.

The graph split into a main graph and a scholarly graph happened on May 9. Currently, we have been granted extra time and can use a legacy server with the full graph, but a lot less hardware, so slower. A final patch, merged in last week, allows us to define which SPARQL endpoint a query should run. So, each time we port a SPARQL query, we can directly update Scholia, making the migration somewhat more manageable.

But, with those uncertainties out of the way, it was time to finish the Scholia Chemistry paper!

The preprint (doi:10.26434/chemrxiv-2025-53n0w) brings 10 years of research together, and describes details of the used methods not formally peer-reviewed before. We describe in detail how chemical structures are added, the choices of Wikidata on how to represent chemical structures, how we curate the quality, and how we visualize chemical structures and data with Scholia. As you can expect, the Chemistry Development Kit has an important role, along with the InChI.

The paper introduces three new Scholia aspects for chemicals, chemical classes, and elements. Each aspect is a template for a page with information about molecular entities and chemical substances, compound classes (like fatty acids), and elements (like carbon). Each template provides relevant information. Of course, any compound, class, or element can also still be opened in the Scholia “topic” aspect, listing relevant literature.

With this paper we aim to show that Wikidata is a innovative platform that meets the needs for a chemical structure database, with detailed data provenance, and scalable community curation.

I welcome your strongest peer review on the preprint. I don’t liking settling for anything less. Here’s the abstract:

Sharing knowledge on chemicals in the digital age has been the playground of databases such as the Chemical Abstract Services and PubChem. Wikipedia complements this field by providing context to chemicals aimed at a broad audience, but is not easily read by machines. Wikidata was started as a database service to improve the machine readability of the knowledge captured in Wikipedia. Wikidata has an open license, application programming interfaces, and a strong provenance model. Scholia uses the features to provide access to chemical knowledge. This study reviews the chemistry in Wikidata, shows how thousands of new chemicals were added, extends Wikidata with new properties for chemical representation and external links to additional databases, and shows how we extended Scholia to represent the chemistry in Wikidata.

Thanks to Finn, Denise, Daniel, and Adriano!

]]>Egon WillighagenRetracted articles in Wikidata2025-02-16T00:00:00+00:002025-02-16T00:00:00+00:00https://chem-bla-ics.linkedchemistry.info/2025/02/16/retraction-data-in-wikidataA good number of years ago, a colleague and I explored if we could get access to the Retraction Watch Database, but we could not afford it. We have been using data on retractions for curate our databases, like WikiPathways. A database should not contain knowledge based on (only) a retracted article. Wikidata, btw, has a small number (499) of statements supported by retracted articles. Similarly, it turns out that I am citing retracted articles in two papers (and a preprint of one of them).

Wikidata has a good number of retracted articles in their database (some 21 thousand at the time of writing). A lot of this data comes from CrossRef, that recently acquired the Retraction Watch Database (doi:10.13003/c23rw1d9)) and started providing the content as FAIR and Open data. With a Bacting-based script I am regularly updating Wikidata with annotations from CrossRef, giving a rich dataset in Wikidata around the queries. Over the past few years I have written various SPARQL queries to show the results which today I collected under a single home:

]]>Egon WillighagenAdding citations between existing articles in Wikidata2024-09-07T00:00:00+00:002024-09-07T00:00:00+00:00https://chem-bla-ics.linkedchemistry.info/2024/09/07/wikidata-citationsScholarly articles provide context to the factualness of statements in Wikidata, similar to the [citation needed] in Wikipedia. And just like the cited references in each scholarly article itself. The citation network is general seen as an essential part of (doing) science, even without citation intention annotation. Nowadays, citations are mostly open, but this took very serious lobbying by the Initiative for Open Citations and not every publisher reacted immediately. But now that they are open, projects like OpenCitations are making this citation network FAIR.

Therefore, when an article is cited as reference in Wikidata, I think that the articles (and other research output) cited in that article is part of the reference. After all, it is really hard to understand any article without the details in the cited articles. So, getting these citations between article into Wikidata deepens the knowledge captured by Wikidata. Of course, Wikidata is also one of the few places where we can capture the citation intentions at all.

Adding these citations manually is cumbersome but sometimes needed as these citations are not open or not FAIR yet. Fortunately, in many cases we can automate the process, for which I wrote a Bacting-cased script. Until recently, the script takes as input a single DOI or a list of DOIs as input, and for each DOI looks up in OpenCitations if it cites other article DOIs and is cited by other DOIs. For the cited and citing DOIs it checks if those are in Wikidata and (only) if they are in Wikidata, then it create QuickStatements. The result can look like this:

The script also needs a OpenCitation token, which you can get here. This is how I run the code from the command line (with the token in the TOKEN environment variable), for a single DOI:

groovy quickstatements.groovy -t ${TOKEN} -d 10.1002/JLAC.18721620110 | tee output.qs

A list of DOIs is provided as a text file, with one DOI on one line. I then use the -l parameter (oh, here DOIs of works by Shyamala Gopalan, mother of Kamala Harris):

groovy quickstatements.groovy -t ${TOKEN} -l harris_dois.txt | tee output.qs

But last weekend I created a new feature. To enrich the profiles of authors, for example Nobel Prize winners, mothers of, or famous chemists, previously I would create a list of DOIs, now I have the script do that:

So, today I could add the citation network for any arbitraty author, e.g. Carolyn Bertozzi, I just pass the Wikidata QID:

groovy quickstatements.groovy -t ${TOKEN} -a Q7442 | tee output.qs

I can imagine that in the future the script will have more such options, to do the same for many authors at some affiliation, or all DOIs for a certain journal.

]]>Egon WillighagenScholia configurability2024-08-23T00:00:00+00:002024-08-23T00:00:00+00:00https://chem-bla-ics.linkedchemistry.info/2024/08/23/scholiaScholia is a visual layer on top of Wikidata providing a rich user experience for browing scholarly research related knowledge. I am using the combinatie for various things, including exploring new research topics (a method, compound, or protein I do not know so much about yet), indexing notable research output (including citations), progress of Citation Typing Ontology uptake, etc. This weekend I hope to send around the final draft for the Scholia Chemistry paper.

Scholia has received a fair share of scholarly and social attention. The Scholia paper has been cited over 100 times and the websites received about 200 thousand page views each day (though we do not know how to get Toolforge to give us sufficient insight into the how and what of that count). There is a Wikipedia template to link to Scholia and some of projects I am involved in link Scholia for articles, such as WikiPathways.

With that, there is also interest in using it for other Wikibases and perhaps even random SPARQL endpoints. These things are not trivial, as Scholia uses complementary APIs, various URL patterns for some of the functionality, and generally, all SPARQL queries are tweaked to the Wikidata Blazegraph SPARQL endpoint to ensure results are returned in reasonable time. But that last requires use of Blazegraph extensions to the SPARQL standard.

All this requires Scholia to become more independent, in a better model-view-controller model. And that actually turns out very important at this moment. That is, Wikidata is not a RDF-first database, but a Wikibase-based store. Whenever an edit is made, RDF is generated and the SPARQL endpoint is updated. Now, the number of edits in Wikidata is enormous and the notion that the SPARQL endpoint is often minutes at most behind is a huge accomplishment. But the Blazegraph platform cannot keep up with Wikidata. Blazegraph is open source, but has been bought up and development stopped from one day to another.

Therefore, a split of the Wikidata SPARQL platform is planned. This split will put one part of the knowledge in on endpoint and the other half in the other. Any query that needs information from both graphs, will have to do a federated SPARQL query. Basically, there are very few Scholia queries that do not rewriting. My first rewrite actually failed, because the rewriting is not obvious and quickly times out. To some extend, this is because now lots of results of subqueries need to be send over the network from one endpoint to the other. When the combined query basically covers half of each endpoint, that’s a lot of network traffic.

An immediate use case of the configuration is therefore running Scholia against the current three endpoints: the current official endpoint, and the two split endpoints under development. With a recent patch Finn and I worked on, this configuration looks like this (and saved as scholia.ini:

[query-server]
# Wikidata:
#sparql_endpoint = https://query.wikidata.org/sparql
#sparql_editurl = https://query.wikidata.org/#
#sparql_embedurl = https://query.wikidata.org/embed.html#

# Wikidata Split Main
sparql_endpoint = https://query-main.wikidata.org/sparql
sparql_editurl = https://query-main.wikidata.org/#
sparql_embedurl = https://query-main.wikidata.org/embed.html#

# Wikidata Split Scholar
#sparql_endpoint = https://query-scholarly.wikidata.org/sparql
#sparql_editurl = https://query-scholarly.wikidata.org/#
#sparql_embedurl = https://query-scholarly.wikidata.org/embed.html#

So, right now, we can test the impact of the split with Scholia and this patch. We would fire up a local instances of Scholia, running against one of the split endpoints, and use the Toolforge instance as baseline.

Now, on my system I need to use Python virtualenv so, I first start a Scholia venv:

source ~/.venvs/scholia/bin/activate

After that, I can select an other endpoint, e.g. the main Wikidata split endpoint (query-main-experimental.wikidata.org) were it not they are currently offline as part of the transition and run Scholia on a unique port:

scholia run

Then I can have two browser windows along side and compare Scholia pages againt the current Scholia instance and when running against another SPARQL endpoint. For now, I can test how well Scholia runs on the QLever instance of Wikidata (superfast and updated data once a week). Here the configuration I have is not entirely complete, and many SPARQL queries do not work against QLever, including anything with graphical depiction. But that said, I can use this configuration:

[query-server]
# QLever
#sparql_endpoint = https://qlever.cs.uni-freiburg.de/api/wikidata
#sparql_editurl = https://qlever.cs.uni-freiburg.de/wikidata/?query=
#sparql_embedurl =

Then, I can compare, for example, the chemicals statistics the main Scholia with one running against QLever:

This query ran without modification. For other queries rewriting is needed, but with this setup we can at least quickly see the differences in the results.

]]>Egon Willighagen