• harmonize hydrogens to various states: depiction, stereo, minimal, and unsafe (useful for depictions)
• generate wedge bonds based on coordinates and stereochemistry
• more Markush / RGroup support
• atropisomers via CXSMILES
• sugar extraction

I also update the following libraries/tools to use CDK 2.12:

• NanoJava 2.16
• Bacting 1.0.10 (and the Python pyBacting will follow asap)

Research software is an integral part of scientific investigations.

This is what Struck wrote in 2018 in a contribution to the 2018 IEEE 14th International Conference on e-Science (e-Science) (doi:10.1109/eScience.2018.00016). I very much agree with this, and the notion is gaining ground in the academic community. Their paper “identifies challenges, risks and new opportunities in research software publication and discovery”.

At the same conference, Spaaks et al. presented a lightning talk about the Research Software Directory (RSD), “a content management system for research software, which promotes the visibility, reuse, and impact of research software” (doi:10.1109/eScience.2018.00013).

I wonder who spoke first at the meeting.

Anyway, I learned about RSD a while ago already and have been using it for some of our group’s research software. We don’t have a collection for our group, but you will find them under the Maastricht University organisation page.

And as sketched by Struck and implemented by Spaaks et al., RSD gives rich context to the research software. It can track the activity on the project (for GitHub, GitLab, Codeberg etc), track citations to key literature, and can have links to distributions where the software is published (like Debian, CRAN, Bioconductor, etc).

This is what it looks like for the Chemistry Development Kit:

I like initiatives like this, as they help the community work out open standards to exchange metadata, and encourage other projects by reusing their APIs.

I do not remember exactly, but I guess we must have met on the 28th and 29th? Maybe already on Wednesday. During this meeting we discussed a common data model (yes, Jmol used the CDK data model at some point) and somewhere during the meeting we wrote down a name for the project. There was the Java Development Kit, so this could be the Chemistry Development Kit. The name stuck.

A quick post like this cannot do credit to the history of the CDK, nor of everyone involved in the past or still is. You can browse some of the history of the CDK in my blog and in Chris’ blog. It has been an amazing journey and with a small grant just behind us (with Alyanne de Haan, René van der Ploeg, and Marc Teunis from Hogeschool Utrecht), and all the awesome things ongoing (new JChemPaint, various extensions, upgraded downstream tools), the CDK is alive and kicking.

A huge congrats and thanks to everyone (and every company and organization) who contributed code to the CDK with this huge milestone. There are a few people that I want to particularly thank (see the AUTHORS file for all names): Chris, who in the late nineties made a difference with open source in chemistry, Dan, for Jmol and hosting this memorable meeting at Notre Dame University, Rajarshi Guha, who operated CDK Nightly for many years, well before Travis and Google Actions, Stefan, Miguel, Gilleain, and Christian, for many years of contributions to the CDK, and John Mayfield, the current CDK release manager.

Actually, you see one pull request here that I closed. I accidentally included a circular dependency. Some core CDK functionality is hard to test with an implementation of the data model, but if that implementation depends on the module you are testing, that won’t work (not in Maven anyway). But the good bits got included in the next pull request. One of the goals was to improve the code covered by the tests.

This coverage testing has an imporant code maintenance purpose: it visualizes whih code is not checked. Sometimes code is not tested that under normal conditions should have been (a bug) and sometimes it is handling a rare situation which you want tested too, to make sure that rare case does not get covered by the common code. Thus, the percentage code covered by tests should be as high as is reasonable. The pull requests therefore aim to raise that percentage, such as for this pull request:

Indeed, over the past 12 months, the coverage did improve, perhaps not as much as we liked, with 2.32 percent point to 64.96 percent:

The CDK started routinely using unit testing somewhere in the zeroes, with home made continous integration as early as 2006, shared with the development community every night, thanks to Rajarshi Guha’s effort. We had our first code coverage results in the same year. And at some point we had sufficient coverage that it gave us the opportunity to routinely check the impact of a patch. Of course, this is exactly what many open source projects do with GitHub Actions nowadays.

Unfortunately, we also found that many of the CDK-using tools we worked on in this grant to get updated to use a (more) recent CDK version do not have such solutions in place. That left us in several cases quite in the dark. More about that soon!

Some pointers:

• we worked on the Groovy Cheminformatics with the Chemistry Development Kit book
  • move the repository to the GitHub organisation
  • improved the build system
• it was explored how the CDK can generate SMILES for glycans
• continued work on updated tools using the CDK, e.g. ChemViz2
• code clean up, e.g. on JavaDoc and the XML parsing
• a JChemPaint release, based on CDK 2.10, with a flatpak for easy install on many Linux distributions

We further had discussions about a possible change of the license (what it would involve) and OSGi support. The problem there is that Java packages can only exist in one OSGi bundle, and this is currently not the discuss. We discussed that the current modules were partially setup to clean up dependencies, and generally modularize the CDK (e.g. each module could have a separate person responsible). We now want to propose a larger core module which covers the common cheminformatics functionality. A final discussion point I want to mention is that there are serious hints that we may have the Chemistry Development Kit as JavaScript in the browser soon!

I like to thank every one who joined the workshop, particularly those that travelled to Maastricht from the UK, Germany, and Bulgaria. Also thanks to the six participants online who joined on the first day!

I opened the session with a few slides welcoming everyone at Maastricht University (and our Dept of Translational Genomics, and explaining the NWO grant. John Mayfield (NextMove) spoke about “What’s New” in the Chemistry Development Kit 2.10, e.g. explaining more about the new (much faster) AtomContainer, SMIRKS, and more.

After lunch, Jonas Schaub (Friedrich Schiller University Jena) showed various projects where the CDK is used, titled “Scaffolds, Functional Groups, Aglycones: Algorithmic Substructure Identification with CDK” (see doi:10.1186/s13321-023-00762-4, doi:10.1186/s13321-022-00656-x, and doi:10.1186/s13321-020-00467-y). Lyudvika Radeva (Ideaconsult Ltd, University of Plovdiv) showed what SYBYL Line Notation (SLN) is and how this is implemented in Ambit (see doi:10.1002/minf.202100027). Sonja Herres-Pawlis (RWTH Aachen University) updated us with “News from the InChI: making the InChI FAIR and including inorganics”, e.g. showing how they worked out how the InChI is going to handle organometalics, where the bonds and the stereochemistry as aspects that were not handled by the current InChI.

After the afternoon coffee break, Zhixu Ni (TU Dresden) showed his work on lipid maps characterization and identification. We previously met a few times at EpiLipidNET COST action meetings, and it was great to see his continued research on representation of lipids and lipid classes in hit “A Fuzzy Solution for Lipid Structures Using CXSMILES”.

Finally, Matthias Mailänder (Lablicate GmbH) gave a “Live demo of where OpenChrom uses the CDK”, and Yajie Ding (University of Groningen) told her about her glycoscience research. There, cheminformatics can also greatly help and the CDK may provide them with solutions.

This really doesn’t do justice to all the discussions, examples, use cases, etc. But it gives you an idea. We had 11 people in the room, and were joined online by an additional 6 people.

commit c5cbf9b5dd49baf582afc595c9cbafc714c5199f
Author: Egon Willighagen <egon.willighagen@gmail.com>
Date:   Fri Apr 10 12:34:42 2009 +0200

    Initial copy of the current draft; converted into separate project for easier branching
    for tunes of the book for workshops and sorts

The original version was in LaTeX and sold online via Lulu.com . Because all code examples were run (the first public edition had 72 pages with 75 code examples), like RMarkdown of Jupyter Notebooks by design, I was able to make many releases. The big advantage of this was that when API changes happened, this would be visible by code not compiling or by output changing.

At some point I open sourced the book (doi:10.6084/M9.FIGSHARE.2057790.V1) and then realized that I can convert the book to Markdown:

commit 2630699aa280200188f2ae9ef3f0698964926752
Author: Egon Willighagen <egon.willighagen@gmail.com>
Date:   Mon Dec 24 16:59:14 2018 +0100

    Create chapter3.md

This is the version available at egonw.github.io/cdkbook/ for some time now. So, now that for SMARTCyp I need to update the visualization, I went book to my book of code examples (I have a collection of more than 200 examples), but then found that the chapter on Depiction was missing. I was not looking forward to this, because I know that the code examples predate a massive improvement by John Mayfield of the rendering stack and I never got around to see if the examples from the book work well enough with that new API (one is actually updated).

That is when I realized that the Groovy Cheminformatics book actually also is a downstream project that needs updating. I have been doing this already and it’s fairly smooth so that I did not think of including it in the grant, other than updating the Migration chapter. I now had enough time to dive into this project. I need that, because the goal of the project is also to learn about all the meta science aspects of project maintenance, roles, communication, etc. Therefore also this blog post: we need a track record, to collect data.

Anyway, porting the first script went fairly easy, but I am now running into a stacktrace:

Processing  RenderSelection.groovyin
doing RenderSelection.out ...
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
/home/egonw/var/Projects/hub/cdkbook-source/code/RenderSelection.groovy: 39: unable to resolve class ExternalHighlightGenerator
 @ line 39, column 16.
   generators.add(new ExternalHighlightGenerator());
                  ^
org.codehaus.groovy.syntax.SyntaxException: unable to resolve class ExternalHighlightGenerator
 @ line 39, column 16.

That brings us to the task of how to find where that class is coming from, which happens to be something I already had to write up for up for RingSearch. Dependency galore.

The project formally started on April 1 but we had our kick-off meeting in Maastricht on April 4-5. We were joined by Javier and on the second day by Marvin, and Ozan from our BiGCaT research group in Maastricht. During this hackathon, I gave a (repeat) presentation about the history of the CDK which also included the problem that software using the CDK does not always use the most recent version.

And that, upgrading tools using the CDK with the latest CDK version, is the main topic of this grant (work package 2, WP2). The full proposal has the focus list of tools, but most of it is also listed in the issue tracker we have set up as project management tool on GitHub.

Second, we actually hacked together on two first tools, one on our focus list, but the other that was requested we have a look at too: SMARTCyp. The latest version uses RDKit (doi:10.1093/bioinformatics/btz037), but the original version uses the CDK (doi:10.1021/ml100016x).

We downloaded the source code of SMARTCyp 2.4.2, started taking notes, Javier started a Maven build environment, updated a lot of code, but we seem quite close to a version that can be tested by people that have integrated SMARTCyp in other tools. This is based on CDK 2.9 and if you ignore the 2D depiction glitch, it looks it was a nice first choice:

On a final note, we plan to record carefully our steps, in an open notebook science approach, with the intention to extract general upgrade steps. For example, we will update the Migration section of the Groovy Cheminformatics with the Chemistry Development Kit.

Note the depiction of the undefined (CIP) stereochemistry on two atoms. Thanks to Adriano and John for working that out.

More about CXSMILES in Wikidata in this Dagstuhl meeting results write up.