Is that a big issue? Hell, yes. Where do you think the FAIR ideas came from? And why FAIR in ten years has not brought about the change it was hoping for?

For me, my fascination for curation started as a student, around 1995, with the Dictionary on Organic Chemistry. At that time, my interest came from wanting to learn about chemistry and biology. During my M.Sc. and PhD, it was obvious how essential it was to derivating correct scientific conclusions from your experiment. Data, knowledge, and software alike, imo. And because curation is expensive, not having to repeat it, I prefer to do it as Open Science.

Curation

Of course, curation has been part of doing science, but to a large extens is separate step from doing science. It is done by database developers, librarians, and chemo- and bioinformaticians. For example, Chemical Abstracts Service (CAS) started over 100 years ago and started indexing chemical structures in 1965. The curation is an ongoing process, also for old records.

Biocuration is getting more and more attention:

The recognition and rewarding by having the International Society for Biocuration (ISB, Scholia page) should not be underestimated (doi:10.1038/455047A). Their Annual International Biocuration Conferences have been running since 2005. And with their awards, they give the biocuration work recognition and, literally, rewarding:

• Biocuration Career Award (2016-2021)
• Excellence in Biocuration Early Career Award (2022-)
• Excellence in Biocuration Advanced Career Award (2022-)
• Exceptional Contribution to Biocuration Award (2017-)

My curation Curriculum Vitae

I don’t have a good curation CV. For a large extend because the curation has been part of a study. The curation itself does not get recognized, and only the journal article does. With datasets slowly getting more recognition, so does data curation, but data curation is not really part of how we do FAIR at this moment, and via this route not getting the attention it gets.

But since I have been updating my CV anyway, I dug up some curation I am proud of:

• the Dictionary on Organic Chemistry, which no longer exists, but it started my Open Science chemistry research
• the Blue Obelisk Data Repositry (BODR), which has been part of various GNU/Linux distributions (see also doi:10.1021/ci050400b). A new version is long overdue
• I contributed hundreds of NMR spectra with uncommon nuclei to NMRShiftDb
• Wikidata, see this preprint, but also many small projects, like adding CXSMILES for polymers, and main subject annotation in Scholia
• WikiPathways (see these blog posts), where I started curating metabolites in 2012, set up a computer-assistent curation platform using SPARQL, and were an early curator of SARS-CoV-2 biological processes
• citation intent annotation with the Citation Typing Ontology, see this Scholia overview
• nanosafety ontology and data: the eNanoMapper Ontology (ENMO), NanoWiki, JRC nanomaterial index and the ERM indentifier database
• made RDF for supplementary information (e.g. this NanoE-Tox spreadsheet, full databases, like ChEMBL and NMRShiftDb
• organized an online ChemCuration event (inspired by the ISB annual meetings!)

I am also curation my blog, which was originally in blogger.com but being ported to Markdown with extra annotation. That includes updating URLs and annotation of blog posts with chemicals, grants, and intention-typed citations.

Long tail

Of course, I have my Wikipedia edits, and contributed to projects like Bioregistry.io, FAIRsharing, regularly submit missed mentions to Altmetric.com, etc. There is a long tail in curation. And there is a lot of curation hidden in my literature list.

And that long tail matters to me. I want every researcher to pick up the challenge to curate their own research output. Put your experimental data in databases, add important provenance, get the details rights. This is essential to reduce the cost of doing research, and that is more important than ever.

BTW, I must note that our bioinformatics team colleagues too have done a tremendous amount of biocuration, in WikiPathways (Denise, Freddie, Susan), in nanosafety (Jeaphianne, Ammar), and in toxicology (Marvin), just to name a few. Often together with B.Sc. and M.Sc. students (which can work really well).

Award nomination

And I hope this makes it clear why I am delighted to was nominated last week for an ISB Excellence in Biocuration Advanced Career Award. The list of past awardees is impressive, as are the other nominations: Laurel Cooper, Oregon State University/USA, Steven Marygold, University of Cambridge/UK, Saurabh Raghuvanshi, University of Delhi/India, and Kimberly Van Auken, California Institute of Technology/USA.

It’s an honor to be listed along these other nominees and being nominated is a great recognition! With a thank you to the person who proposed my nomination.

• When publishing data make an explicit and robust statement of your wishes.
• Use a recognized waiver or license that is appropriate for data.
• If you want your data to be effectively used and added to by others it should be open as defined by the Open Knowledge/Data Definition – in particular non-commercial and other restrictive clauses should not be used.
• Explicit dedication of data underlying published science into the public domain via PDDL or CCZero is strongly recommended and ensures compliance with both the Science Commons Protocol for Implementing Open Access Data and the Open Knowledge/Data Definition.

I think these are very workable next steps in Open Date, perhaps even worthy end goals. I endorse them .

Principle 1: an explicit and robust statement
This is in my opinion the most important principle. Too often you find a database with really useful data, but without any clue about what you are allowed to do with this data. Of course, I can contact the authors, get their permission, etc. They probably like it that way, and I can even understand that. However, it does not scale, and it is slow. Even worse is the situation when the original composer gets missing in action. Both are equally valid, but explicit statements just make things easier.

Principle 2: use a waiver or license appropriate for data
This principle is debatable. Very much like the BSD-vs-GPL flamewars, some like copylefting, others do not. There is an important difference though. Software has the concept of interfaces, allowing to more easily share incompatible licenses cleanly separated by these interfaces. This, for example, allows you to run proprietary software on a Linux kernel. However, data sets do not have such a concept. There is not such thing as an interface between two numbers.

This makes the concept of mixing data sets different: because there is no such interface, any mixing can only happen between compatible licenses. This is one reason behind the choice of very liberal licenses like CC0. This license, or waiver really, allows you to do anything, and most certainly, mix data sets.

And that makes things a lot easier. But then again, while these are nobel goals, I rather see people use a copylefting licenses than no license at all.

Principle 3: non-commercial and other restrictive clauses should not be used
I think again making things easier is the goal. The non-commercial clause is interesting, and actually likely an important one. Consider course material, a course book. Those are commercial. Some even argued that many universities themselves are actually commercial entities.

Principle 4: the public domain via PDDL or CCZero is strongly recommended
I second these choices over a mere claim claim that the data is public domain. The PD concept has many meanings and not the same in every jurisdiction. In particular, differences between USA and EU law. Waiving these right, which is just the same as claiming public domain, works in any jurisdiction, again, making things a lot easier.

Open Data, Open Source, Open Standards are not goals
The underlying pattern of my comments must be clear: the principles make life easier. This is all what Open Source and Open Standards (whatever those are).

The three pillars of the ODOSOS mantra is not goals, but merely the means of making life easier.

The Panton Principles certainly make life easier in Open Data, and initiative like the Linking Open Drug Data in which I participate will greatly benefit from people adopting them.

The Principles do not solve all problems. There is still a lot of ‘Open Data’ licensed with unrecommended licenses. For example, the NMRShiftDB uses a GNU FDL license, and data from supplementary material of Open Access journal articles is like Creative Commons.

Another related initiative should certainly not go unnoticed either: Is it Open Data? is a service where you can try to resolve what the license is for one of those databases which is not quite Panton Principles compatible yet.

OK, one last thing. The Dutch government is bursting, and I want to listen to the music. With permission, I have been hacking the Panton Principles endorsement page, and injected some extra span elements, to make it easier to machine process (again, to make things easier), so you can use the following one-liner to calculate the number of people endorsing the principles:

$ wget -O endorsed.html http://pantonprinciples.org/endorsed.html ; xpath -q -e "//span[@class='signature']/span[@class='Country']/text()" endorsed.html | sort | uniq -c

The current count is hitting 44 now, and has not quite reached the 500 I had hoped for yet:

1 Australia
1 Canada
1 Catalonia
2 Espana
2 France
6 Germany
1 Greece
1 Italy
1 Netherlands
1 New Zealand
1 Norway
1 Poland
1 Slovenia
1 Sweden
1 Switzerland
1 The Netherlands
9 UK
1 U.K.
1 United Kingdom
1 United States of America
9 USA

Anyone knows how we can convert this into some nice world map graphics with a few lines of code?

Now, I am looking for a bar in Uppsala to write up some ideas about what specifications are :)

import net.sf.json.groovy.JsonSlurper;

input = new File("substances.json")
json = new JsonSlurper().parse(input);

println "@prefix dc: <http://purl.org/dc/elements/1.1/>";
println "@prefix cp: <http://rdf.openmolecules.net/chempedia/onto#>";
json.each { it ->
  println "<" + it.uri + "> dc:identifier \"" + it.gsid + "\";";
  println " <http://www.w3.org/2002/07/owl#sameAs> <http://rdf.openmolecules.net/?" + it.inchi + ">;";
  println "  <http://www.iupac.org/inchi> \"" + it.inchi + "\".";
  if (it.namings.size() > 0) {
    for (int i = 0; i<it.namings.size(); i++) {
      naming = it.namings.get(i);
      namingURI = it.uri + "/naming" + i;
      println "<" + it.uri + "> cp:hasNaming " +
        "<" + namingURI + ">.";
      println "<" + namingURI + "> a cp:Naming;";
      println "  cp:hasName \"" + naming.name + "\";";
      println "  cp:hasStatus \"" + naming.status + "\";";
      println "  cp:hasScore \"" + naming.score + "\".";
    }
  }
}

After uploading it into Virtuoso (now using DB.DBA.TTLP instead of DB.DBA.RDF_LOAD_RDFXML_MT ), we can now have our regular SPARQL fun with the data from ChemPedia. For example, list the 10 names with the most votes:

prefix dc: <http://purl.org/dc/elements/1.1/>
prefix cp: <http://rdf.openmolecules.net/chempedia/onto#>

select distinct ?name ?score where {
  ?s a cp:Naming ;
     cp:hasName ?name ;
     cp:hasScore ?score .
} ORDER BY DESC(?score) LIMIT 10

• NMRShiftDB RDF #2: Some statistics
• NMRShiftDB RDF #1: Spectra by InChI
• NMRShiftDB enters rdf.openmolecules.net #2: SPARQL end point with Virtuoso

Peter Ansell has shortly after that copied the data into Bio2RDF, but I had not blogged about that yet. So, here goes. If you have not looked at Bio2RDF yet, this is a good time to do that. The structure of the exposed triples is not perfect, and I just realized I made a beginners mistake, to use a domain name in a namespace I have not control over (bad me). The Virtuoso6 faceted browser allows you to navigate the data in Bio2RDF by molecule (e.g. molecule 234):

And by spectrum too (e.g. spectrum 4735):

Spectra per spectrum type

One of the statistics given in the aforementioned page is the number of spectra per nuclei. This can be recalculated with the following SPARQL:

The results for the 1.3.3 release are:

I am a bit surprised by the count for the silicon NMR spectra, as I would have thought I alone had entered more than just five.

Molecules with the most spectra

It turns out that the molecules have in the 1.3.3 NMRShiftDB release at most 7 spectra, as I can calculate with:

That is going to change, as the paper I am digitizing now (doi:10.1021/jo971176v) has carbon and hydrogen NMR spectra for 7 solvents for each compound :) It should be possible to summarize the number of molecules for each number of spectra per molecule, but did not manage to get this SPARQL to work out well.

BTW, did you know you can find reprint PDFs of a paper (if any; this one happens to have a PDF copy) with Google using the title in quotes and filetype:pdf? Try this query. The top hit was molecule 10016314 (RDF), which has 4 ¹³C spectra, one ¹⁵N and two proton NMR spectra.

Molecules with the most different nuclei

In the first query, we already save saw in the first SPARQL, there are 11 different nuclei in the database, though carbon and hydrogen are by far the most abundant spectra. I like diversity, so one statistic I find interesting, is the molecules which have spectra with the most different nuclei. This is done with the query:

It shows that molecule 10023801 (RDF) has 5 different NMR types: ¹³C spectra, one ¹⁵N, ²⁹Si spectra, one ¹⁷O, and ¹H spectra. Unfortunately, the compound also has chlorines, so it disqualifies as molecule for which NMR spectra are available for all its elements.

There are precompiled binaries for Ubuntu and Debian, but Michel encouraged me to use version 6 when he visited us . And so I compiled and install 6.0.0.TP1 on the public server, while I do have the binary debs for 5.0.12 on my laptop. With some basic Apache magic, I hooked up the SPARQL end point of the server to the web:

<Proxy /nmrshiftdb/sparql>
  RewriteEngine On
  Allow from all
  ProxyPass        http://localhost:8890/sparql
  ProxyPassReverse http://localhost:8890/sparql
</Proxy>

Nice thing about this is, that I can set up multiple servers, allowing me to keep incompatibly licensed data sets apart (see Open Data: license, rights, aggregation, clean interfaces? ), which is the same approach Bio2RDF is taking.

The end point now offers about 278887 triples, but this will soon rise as I make more content from the database available in the original SQL database. The data is from the 1.3.3 release by Chris’ team, and does not include my 1000th spectrum.

Getting the data into the database was not trivial either. The documentation suggests WebDAV, and that indeed worked for me once, after using the curl approach suggested here. But upon a second upload, it did again not enter the store. The ultimate solution was to use the iSQL interface, with the following SQL

DB.DBA.RDF_LOAD_RDFXML_MT(
  file_to_string_output('/tmp/nmrshiftdb.rdf'), '',
  'http://pele.farmbio.uu.se/nmrshiftdb'
);

Scientifically, this progress is not overly interesting, although it makes it very clear that you really should not have to be happy with proprietary and non-semantic formats for anything. But, to me, this is mostly a technological success of great importance: I can now share really large sets of RDF data.

Querying this data is a simple with SPARQL, and the results are available in various formats, such as JSON, which makes it easy to integrate in third-party applications or Google Wave robots (did I hear someone say NMRShifty?). As I have blogged before, SPARQL is an excellent tool to aggregate scientific data prior to data analysis. And I will demo more interesting queries later this month.

Open licensing for data has not as much matured as for software, and international law seems to be more confusing about the issues. I guess that is because data aggregation has been around for way before the computer era. The PDDL and CC0 both try to overcome this fuzziness. But there is another issue we need to keep in mind. A lot of useful Data was aggregated and made Open before these licenses came about, and use, for example, the GNU FDL license, such as the NMRShiftDB.

Rights

Right now, there are two Open Data camps, much like the BSD-vs-GPL wars in Open Source: one that believes in waiving any rights on the Data, indicating that facts are free; others that believe that data must be protected to not be eaten by big companies and lost to the community (e.g. the WolframAlpha arragnements are suspect).

Of course, both camps are not that far apart, and both believe Open is important. Interestingly, there are some noteworthy differences with the Open Source wars. I see parallels between the two, which details an important difference: Open Source has algorithms (uncopyrightable) and implementations (copyrightable); Open Data has Data (uncopyrightable) and aggregation (copyrightable). Open Source talks mostly about the implementation, not the algorithm; it’s Open Source, not Open Algorithms after all. In cheminformatics it is even often the case that the algorithms are not even specified and that there only truly is source.

However, Open Data in title does not make distinction. Data is fairly cheap and acquisition can be automated and computerized; Aggregation, on the other hand, requires human involvement: curation and thinking about data models, etc. This is where added value is. Consider an assigned NMR spectrum or the raw data returned from the spectrometer.

It is this added value that people want to protect, not the data itself. I think.

Aggregation

One important argument that tend to show up when people argument for PDDL and CC0 is that it makes data aggregation easier. This is most certainly true: if you can do whatever you like with a blob of data, that also means aggregate with any other blob of data. However, copyleft licenses, like the GNU FDL, require the aggregation to have a compatible license too. It is the license incompatibilities that make this impossible. Or … ?

Open Source has matured to such a point that it is fairly clear what the intended behaviour is, regarding derivatives. An aggregation of software (typically refered to as a distribution) is only a derivative under certain conditions. This makes it possible to run proprietary software on top of GNU/Linux, which uses the GNU GPL but does not require software to run on top of it to be GPL too. Unless… unless, not a clear well-defined interface has been used, indicating a strong dependency. Now, surely, these things have not been confirmed to match actual law in court, but the intentions are clear.

Clean Data Interfaces?

Now, if we would translate this to Open Data, would there be the equivalent of a clean interface? Can we build a data distribution with data of various licenses? I think we can! I am not a lawyer and please consider this an invitation to discuss these matters…

Let’s start simlpe… if I put a GNU FDL image in this blog, by linking to it with a open, free, clean HTML interface (<img src=""/>), would that make my blog GNU FDL too? I don’t think so. Surely, I would need to list copyright owner, and actually would be required to put the GNU FDL in my blog too, but hope linking to the license text would suffice too. (Let’s skip fair use at this moment, and assume the use goes beyond fair use). Question: am I not using a clean interface, and would this not make the image’s license no infect my blog?

A more difficult example, consider rdf.openmolecules.net, which surely aggregated facts, including data from the NMRShiftDB and DBPedia. I am using a unique identifiers here, the NMRShiftDB compound ID, and the DBPedia URL, which surely is GNU FDL, and use this to make a <owl:sameAs> statement. Again, please do not consider fair use, which this certainly is. But, let’s say I put in some more DBPedia and NMRShiftDB data in this aggregation. The GNU FDL data on rdf.openmolecules.net would be separate RDF blocks, with proper dc:license, dc:author annotation. But the block would be part of a larger aggregation. The clean interface here is Resource Description Framework.

This second case does not only affect my rdf.openmolecules.net website, but, for example, bio2rdf.org is also in the same situation and aggregated and distribute DBPedia’s GNU FDL data (e.g. hexinanose. Does that make the whole of bio2rdf database GNU FDL. They too use RDF as clean interface.

Call for Discussion

Despite what one of the two camps like to see, the mere fact of added value when making data aggregations will keep copyleft license stay around, and instead of trying to convince everyone of the virtues of PDDL- and CC0-like licenses, we should think about to what extend it really matters.

I can do my data analysis with data sources of various licenses. I can search and retrieve data from various sources with various licenses. What obstacles are really there that disallow us to do science? Do the data interfaces we have now not provide enough technical means to address the license incompatibilities? They have in Open Source, why would that not apply to Open Data too?

And made links between the new frontend and rdf.openmolecules.net, make the Linked Open Chemistry Data (LOCD) network grow (naming following Linked Open Drug Data). In comparison with the previous depiction, I added arrows to indicate the direction of the linking. Green nodes still indicate sources with an RDF interface; therefore, the LOCD network consists really only of those green nodes:

The link with DBPedia is discussed in DBPedia enters rdf.openmolecules.net . The source code for the NMRShiftDB-RDF frontend can be found at GitHub.

And made links between the new frontend and rdf.openmolecules.net, make the Linked Open Chemistry Data (LOCD) network grow (naming following Linked Open Drug Data). In comparison with the previous depiction, I added arrows to indicate the direction of the linking. Green nodes still indicate sources with an RDF interface; therefore, the LOCD network consists really only of those green nodes:

The link with DBPedia is discussed in DBPedia enters rdf.openmolecules.net . The source code for the NMRShiftDB-RDF frontend can be found at GitHub.