Scholia has not been upgraded. It has more then 350 SPARQL queries, and each one has to be tested separately and updating every query is not trivial. Together with Daniel, Finn, and others, I have hacked up patches last year to:

• configure Scholia for the endpoint to use
• create pages for many Scholia SPARQL queries

This month I continued working on the second, and I:

• added titles for chemistry aspect panels
• use the legacy SPARQL endpoint

That second also indicated that the legacy server has more limited resources and users will more quickly run into error messages that too many queries are run in parallel. Now, users can rerun the query, but then the results table contains the previous error message. Second, you want to run the queries for one aspect to not run all at the same time, but have Scholia send of the query when the panel becomes visible (and scrolling a page takes a bit of time).

For these issues, I wrote these two patches (yet to be approven and merged):

• delete the previous error message
• lazy load the table and iframe panels

Now, the iframes already had some aspects of lazy loading, but it turned out that it was mostly lazy display, and the queries were still run as soon as possible. This last patch challenged my JavaScript skills and I learned Intersection Observer API, a browser technology that allows the browser to see what part of the webpage you are looking at right now. Yeah, I can easily see how that does user profiling, but in this case it is just used to fire of the SPARQL query when it become relevant. It uses an additional callback function, so I had to make sure Jekyll/Liquid creates custom callback functions for each panel. Actually, I intended to show the code here, but I am not entirely sure how to escape the code so that Jekyll does not try to run the instructions. For now, you have to check the PR.

Scholia Chemistry paper

The other things I have been doing, is finally finish up the Scholia Chemistry paper. That actually depended on the maturing of various tools, me figuring out how to characterize the actual amount of content contributed to Wikidata and how to make that transparent, and more recently, the above to be able to convince readers Scholia will not die with the graph split. With the above pages, we have, I think, sufficient guarantee it will be around for another few years, at least.

This paper, which I hope to finish the final draft today, applying some good feedback from co-author last weekend, is the final bit of work done on the Alfred P. Sloan Foundation grant.

We intend to put the paper up as preprint soon and then submit it do a Diamond Open Access journal, one that supports CiTO citation intent annotation.

The list of supported research output types in ISAAC is a bit dated, but includes scientific articles, books and monographs, book chapters, PhD theses, conference proceedings papers, professional publications, publications aimed at a broad audience, patents, contracts, and other. With Recognition & Rewards in mind, this list should be more diverse. And clearly missing are software and data, because these are already supported by global unique identifiers and dedicated efforts for software citations and data citations. FAIR has progressed a lot for these two types.

When entering new research output to a project in ISAAC, you get asked to fill out various HTML forms. For articles, each author is a separate HTML form. All in all, quite a bit of work. But with unique identifiers and open APIs, it is a waste of research funding to have to enter this all by hand. Some time earlier I heard of browser plugins in the USA that automagically filled out those forms, and realized I wanted that too.

Fortunately, Lars Willighagen had done much of the work already with citation.js (doi:10.7717/peerj-cs.214), a JavaScript library that can convert formats like BibTeX into formatted references (with Citation Style Language and citeproc-js), but also can support various identifiers to fetch bibliographic metadata. All we needed is to integrate that. And so the ISAAC Chrome extension was born. But the history, technology, and use has not been written up, while we have a solid base of some 50 users who regularly use it. And one user wrote:

Werkt geweldig. [..] dit de enige manier waarop publicaties redelijk ingevoerd kunnen worden.

Actually, maybe we should rename the extension to ISAAC Browser Extenaion, because it also works in Brave and Edge.

2025 updates

The last update had been a while, and we got reports of some changes on the ISAAC database side, and we could confirm at least one of the HTML form identifiers had changed, so we fixed filling out the Open Access status of output. This is released as version 1.5.0 (doi:10.5281/zenodo.14562484).

Another change is that the ISAAC database now supports listing the ORCID identifier of authors, and this metadata is increasingly available from research output metadata, and a single line change was enough for Lars to update the extension to automatically fill that out too. This is FAIR in action. This version is released as version 1.6.0 (doi:10.5281/zenodo.14562504) and should be available from the webstore soon.

How it works

While the ISAAC database does not have an API, at least we found sufficient hooks in the HTML to get a reproducible workflow. The foundation of the browser extension is global unique identifiers, and it supports DOIs, ISBNs, and PubMed identifiers for research output. For authors, it supports the ORCID. To fetch the metadata, it uses online resources Crossref, DataCite, and mEDRA, Google Books and OpenLibrary, PubMed and Unpaywall. The first three to fetch metadata for DOIs, the next two for ISBN numbers, and PubMed for, well, PubMed identifiers. Based on the retrieved metadata it determines which type of research output it should fill out the HTML for.

Unpaywall is used to see if the output is, for example, published in a purely open access venue (like a CC-BY-only journal like eLife or Nature’s Scientific Data), or published in a hybrid journal. The ISAAC database does not have the option to drop a URL (which could be automated with Unpaywall too), but does allow uploading documents into their database. This last is left to the user.

How to use it

Users would install the browser extension and this would add an add-on icon to the toolbar. The icon shows the various colors of Open Access with an I, for identifier. The user would then login on the ISAAC database, open their project grant page, and navigate to the Product tab:

To use the extension, the user would take the following steps. First, click the “Toevoegen” button, green-blue with white letters in the above screenshot. This would give a page like this:

Second, and optionally, click one of the types. The metadata retrieved by the extension contains sufficient information to make the right guess, so that this step is optional. If you find that the metadata is wrong and the wrong guess was made, in this step you can first manually indicate the research output type.

Third, one the page with the above screenshot (or, optionally, after indicating the output type), click the ISAAC Chrome Extension icon in the browser toolbar to give a popup dialog:

Fourth, select the identifier type (DOI, ISBN, or PubMed) and give the identifier itself, and then click Search. For example, for a DOI, it would look like this:

Fifth, the plugin will then guide you through the ISAAC HTML forms, just like you would do manually, but with the important difference that some forms show in different order. But rest assured, it will not submit anything before your final approval. For example, for a journal article I would immediate fo to the HTML form for the first author, which, for a random article, could look like this:

By clicking “Verder”, the browser extension allows you to add missing metadata (for example, the ORCID is not listed for this author in the CrossRef metadata and the gender is not shared by the publisher), and sometimes you may find yourself needing to correct metadata.

Sixth, after going through all author pages, you will return to the main product form, which will look something like this (for a random paper):

Here you can add the final missing information and upload additional files, like a PDF of the article. In the above screenshot we find some required (red asterix) missing. In this case, the DOI referred to an article published as “as soon as publishable” and the page numbers and issue is simply not known yet. You can also see the Unpaywall metadat in action here.

Seven, like before, the final submission of this new output is done manually. The ISAAC Chrome Extension requires that manual step; on purpose: you are in control.

However, we very easily jump to functions. In fact, we use so many libraries nowadays, this linearity is not so clear anymore. Things just happen with magic library calls. But at the same time, the library calls make our life a lot easier: by using functions, we group functionality in easy to read and easier to understand blobs.

OK, the previous example showed that we could use the HTML @onClick attribute to provide further detail. But I did not show how. This is how:

html += "Name: <span onClick=\"showDetails('" +
  escape(dataJSON) + "\')\">" + 
  response[i].prefLabel + "</span>";

This code adds the @onClick attribute and a function call to the showDetails() method which takes one parameter, where we pass escaped JSON. That is non-trivial, I understand, and may be due to my limited knowledge of JavaScript. The escaping of the JSON is needed to make quotes match in the generated HTML. In the function later, we can unescape it and get the original JSON again. Importantly, the dataJSON data contains all the details I like to show.

Now, this functions needs to be defined. Yes, plural, because two functions are used in this code snippet: showDetails() and escape(). The last is defined by one of the used libraries. The showDetails() function, however, I made up. So, I had to define it elsewhere in the HTML document, and it looks like:

var showDetails = function(dataJSON){
  data = JSON.parse(unescape(dataJSON));
  document.getElementById("details").innerHTML =
    data._about;
};

This example actually gives the exact same output as the code in the previous post, but with one major difference. We now can extend the function as much as we like, but the code to output the list of found compounds does not have to get more complex than it already is.

Practically, what matters here is how to ask the question and how to understand the answer.

We are supported in these practicalities with JavaScript libraries, in particular the ops.js library and general JSON functionality provided by most browsers (unless the student decided to use a different programming language, in which there are different libraries). Personally, I have only very limited JavaScript experience, and this mostly goes back to the good old Userscript and Greasemonkey days (wow! the paper is actually the 4th highest scoring BMC Bioinformatics article!). But because my JavaScript knowledge is limited and rusty, I spent a good part of today, to get a basic example running. Very basic, and barely exceeding the communication details. That is, this is the output in the browser:

So, what does the question look like? The question is actually hardcoded in the HTML source, but the page does take two parameters: the app_key and app_id that come with your Open PHACTS account.

The ops.js library helps us, and wraps the Open PHACTS LDA methods in JavaScript methods. Thus, rather can crafting special HTTP calls, we use two JavaScript calls:

var searcher = new Openphacts.ConceptWikiSearch(
  "https://beta.openphacts.org",
  params["app_id"], params["app_key"]
);
searcher.byTag(
  'Aspirin', '20', '4', '07a84994-e464-4bbf-812a-a4b96fa3d197',
  callback
);

The first statement creates an LDA method object, while the second makes an actual question. I have not defined the callback variable, which actually is a JavaScript function that looks like:

var callback = function(success, status, response){
  var result = searcher.parseResponse(response);
  document.getElementById("output").innerHTML =
    "Results: " + JSON.stringify(result);
};

When the LDA web service returns data, this method gets called, providing asynchronous functionality to keep the web page responsive. But when called, it first parses the returned data, and then puts the JSON output as text in the HTML. The output that is given in the earlier screenshot.

So, hurdle taken. From here on it’s easier. Regular looping over the results, creating some HTML output, etc. The full source code if this example is available as Gist.

In fact, I hooked in some HTML onClick() functionality so that when you click one of the compound names, you get further details (under Compound Details), though that only outputs the ConceptWiki URI at this moment. A simple for-loop does the heavy work:

html = "<table>";
for (var i=0; i<response.length; i++) {
  html += "<tr>";
  html += "<td>";
  dataJSON = JSON.stringify(response[i]);
  //   dataJSON.replace(/"/g, "'");
  html += "Name: <span>" + response[i].prefLabel + "</span>";
  html += "</td>";
  html += "</tr>";
}
html += "</table>";
document.getElementById("table").innerHTML = html;

So, we’re set to teach the students all the basics of programming: loops, variables, functions, etc.

You then approve the account via your email account and you are set. The account is needed to get an API key. Using this key, Open PHACTS developers can contact you if your scripts go berserk So, you are kindly invited to make crazy hypotheses and hack the hell out of the platform. That’s what I hope my students will do.

To try your new key, go to the documentation page, and open, for example, the SMILES to URL method:

Here you can see what parameters this LDA method has. We focus now on the app_id and app_key fields. Each account comes by default with a, um, default app_id and default app_key. Just click on the field and select them:

Select the defaults and enter a SMILES (try: CC(=O)NC1=CC=C(C=C1)O)). You can select the format you like (I like Turtle) and you get Linked Data back on this compound.

Now, go explore the LDA methods.

• have the ability to recognize various classes of chemical entities in pharmacology and to understand the basic physical and chemical interactions.
• be familiar with technologies for web services in the life sciences.
• obtain experience in using such web services with a programming language.
• be able to select web services for a particular pharmacological question.
• have sufficient background for further, more advanced, bioinformatics data analyses.

So, this course will be a mix of things. I will likely start with a lecture or too about scientific programming, such as the importance of reproducibility, licensing, documentation, and (unit) testing. To achieve these learning goals we have set a problem. The description is:

In the life sciences the interactions between chemical entities is of key interest. Not only do these play an important role in the regulation of gene expression, and therefore all cellular processes, they are also one of the primary approaches in drug discovery. Pharmacology is the science studies the action of drugs, and for many common drugs, this is studying the interaction of small organic molecules and protein targets.

And with the increasing information in the life sciences, automation becomes increasingly important. Big data and small data alike, provide challenges to integrate data from different experiments. The Open PHACTS platform provides web services to support pharmacological research and in this course you will learn how to use such web services from programming languages, allowing you to link data from such knowledge bases to other platforms, such as those for data analysis.

So, it becomes pretty clear what the students will be doing. They only have six days, so it won’t be much. It’s just to learn them the basic skills. The students are in their 3rd year at the university, and because of the nature of the programme they follow, a mixed background in biology, mathematics, chemistry, and physics. So, I have a good hope they will surprise me in what they will get done.

Pharmacology is the basic topic: drug-protein interaction, but the students are free to select a research question. In fact, I will not care that much what they like to study, as long as they do it properly. They will start with Open PHACTS’ Linked Data API, but here too, they are free to complement data from the OPS cache with additional information. I hope they do.

Now, regarding the technology they will use. The default will be JavaScript, and in the next week I will hack up demo code showing the integration of ops.js and d3.js. Let’s see how hard it will be; it’s new to me too. But, if the students already are familiar with another programming language and prefer to use that, I won’t stop them.

(For the Dutch readers, would #mscpils be a good tag?)

• McPrinciple #1: access to data
• McPrinciple #2: be clear in what you mean

Peter’s post #solo10: Green Chain Reaction; where to store the data? DSR? IR? BioTorrent, OKF or ??? gives me enough basis to write up a third principle:

Molecular Chemometrics Principles #3: We make scientific progress if we build on past achievements.

Sounds logical, right? Practically, the way we share our cheminformatics knowledge makes this standing on shoulders pretty difficult. But there is one particular aspect I would like to ask your attention for: you can contribute by making clear what shoulders you would like to stand on. That is, where do you prefer to put your effort, and what message would you like to give to your user community.

In the aforelinked post, Peter asks where he should upload his data, and he suggest BioTorrent (see my review BitTorrents for Science), DSpace, and CKAN. Now, his Green Chain Reaction is picked up (see these few blog posts), and the resulting data should be distributed as much as possible. The exact location does not really matter…

But…

By picking where you upload, you make a statement to your community: “Look guys, we are distributing our data via Foo, because we believe those guys are doing good work! Perhaps you can support them too.”.

This principle does not only apply to data, it applies to things too. For example, when iChemLabs and RSC ChemSpider Announce Partnership they do not just improve the user experience of ChemSpider (which I certainly won’t object against), but they also imply “Look dudes, your product is just not good enough and we do not want to help you improve it either”. Of course, ChemSpider has every right, and for them to succeed it is crucial to make decisions like this. Fortunately, ChemDoodle is GPL.

Every project with a user base has the opportunity to support shoulders, if they only visibly stand on them. By merely discussion the Green Chain Reaction, I show to support this social web experiment. You can too. Use these powers wisely. May the McPrinciples be with you.

Fortunately, Neil posted three very useful GreaseMonkey script to clean stuff (and since I use those to link science databases and resources anyway, see Christmas Presents and DOI:10.1186/1471-2105-8-487): FriendFeed Service Icons, Cleaner FriendFeed, and Remove avatars from Friendfeed beta. The last may require the script target websites to no longer point to the beta server, but the real thing. However, by the time you read this, the script may already be updated.

After installing these, my FriendFeed page looks better again:

*) There was some confusion on the two beta Bioclipse2 releases so far. Some people expected a release without any bugs left. That release is what we planned to call a Release Candidate. We agree that the first two betas at least turned out to be more alpha than we actually hoped, and we thank everyone who has given these releases a go. Those who tried several development releases of Bioclipse2 saw a lot of ongoing development, and we are fixing any bug reported on these releases. So, do not hesitate in reporting bugs!

Earlier in this series:

• Bioclipse2 Scripting #1: from SMILES to a UFF optimized structure in Jmol
• Next generation asynchronous webservices #2
• Scripting JChemPaint
• Bioclipse for CDK Developers #1