So, this new edition is again thicker, summing up to 152 pages now, which is 28 pages more than the 3rd edition . Indeed, the table of contents is more than half a page longer in itself, though, just barely, still fitting on four pages. In fact, I had to remove one (new) subsection title, because it would take otherwise two further pages.

The new content is again a mix of sections and chapters. While writing new chapters, I find myself realizing I need to cover more basics. Those get typically added as new sections. I did not get many feature requests, except for one email pointing me the text promised how to interpret and handle failing atom type perception, which explains one of the new sections. The full list of new content is:

• Section 2.1.4: explaining the three flavors of atomic coordinates
• Extended Section 2.2: added detail about electron counts of bonds (partly in reply to this post by Rich)
• Chapter 5 “Protein and DNA”: four pages, mostly about PDB files, and the matching CDK data structure
• Chapter 6 “IChemObjectBuilders”: four pages explaining the four alternative builders CDK 1.4.7 has
• Section 7.8: a new section with recipes on how to post-process read input, discussing MDL molfiles only now. It talks about what information is present in the file format, and what steps must be untertaken to add missing information
• Section 8.2.4 “No atom type perceived?!”
• Section 11.4: describes how to depict aromatic rings
• Section 11.5: describes how to change the background color of depictions
• Section 13.4: explains how to calculate the Van der Waals volume of molecules
• Section 18.1.3: discussing the API improvement in the iterating readers
• Appendix C: a list of all descriptors provided by the CDK
• Appendix D: a list of file formats known by the CDK, indicating which has readers and writers

On top of that, I improved other bits of the book too, such as the resolution of the depictions of molecules, as well as those of various diagrams. Also the number of scripts has seriously gone up, from 94 to 134!

Appendix C is a prelude to a chapter I am already writing, but did not get finished yet: a chapter about descriptor calculation. But since I just started a new post-doc position, it may take another six months for that chapter to make it into print.

The paperbak is available from Lulu.com, an on-demand publisher, as well as this ebook version.

I am starting to get the hang of this publishing soon, publishing often thing, and just uploaded edition 1.4.1-0 of the Groovy Cheminformatics book. The cover is the same (with one typo fix), and the content is 20 pages thicker. True, six of those pages are isotope masses of all natural isotopes. That leaves 14 pages with this new content:

• Section 2.7 on line notations with 2.7.1 about reading and writing SMILES
• Section 6.3 about Sybyl (mol2) atom types
• Section 7.4 on atom numbering with 7.4.1 on Morgan atom numbers, and 7.4.2 on InChI atom numbers
• Chapter 9 on molecule depiction with the new rendering code, with
  • Section 9.1 on drawing molecules,
  • Section 9.2 on rendering parameters, and
  • Section 9.3 on the generator API and how to add custom content
• Section 11.4 on calculating aromaticity
• Appendix A.2 listing all Sybyl atom types
• Appendix B listing all naturally occurring isotopes

Features requests most welcome.

Some project are never finished. Neither is this one, but it is never too late to change how things work, so, taking advantage of publishing-on-demand, here I introduce the release-soon, release-often equivalent of cheminformatics books, my Groovy Cheminformatics with the Chemistry Development Kit book:

With a serious discount for just being the first edition (1.3.8-0), but still counting at 72 pages with 75 code examples, this edition marks a personal milestone (and probably not much more than that). There remains much to do, but I promised a release by tomorrow, so here it is. Next releases will contain more code examples, more functionality descriptions, and more literature reviewing where such code is used in science. The plan is to make new editions with each new CDK release, as well as new editions when I added a new chapter, section, or just paragraph. But, there will not be a Nightly build service anytime soon.

The current table of content is as follows:

Now, the book content is not open content. However, it contains nothing that is not available in other means. It’s just the compilation that makes this book interesting, as well as that I put effort in ensuring the code examples remain working. For that, I ask a minor financial contribution.

But, it will need to understand things like UK-92,480. This is made possible with the Oscar4 refactorings we are currently working on. You only need to register a dedicated dictionary . Oscar4 has a default dictionary which corresponds to the dictionary used by Oscar3, and a dictionary based on ChEBI (an old version) (see this folder in the source code repository).

Adding a new dictionary is very straightforward: you just implement the IChemNameDict interface. This is, for example, what the OPSIN dictionary looks like:

public class OpsinDictionary
implements IChemNameDict, IInChIProvider {

  private URI uri;

  public OpsinDictionary() throws URISyntaxException {
    this.uri = new URI(
      "http://wwmm.cam.ac.uk/oscar/dictionariy/opsin/"
    );
  }

  // the URI is somewhat like a namespace
  public URI getURI() {
    return uri;
  }

  // there are no stop words defined in this
  // dictionary
  public boolean hasStopWord(String queryWord) {
    return false;
  }

  // see hasStopWord()
  public Set getStopWords() {
    return Collections.emptySet();
  }

  // it has the name in the dictionary if the name
  // can be converted into an InChI
  public boolean hasName(String queryName) {
    return getInChI(queryName).size() != 0;
  }

  // this dictionary can return InChIs for names
  // so, it implements the IInChIProvider interface
  public Set getInChI(String queryName) {
    try {
      NameToStructure nameToStructure =
        NameToStructure.getInstance();
      OpsinResult result = nameToStructure
        .parseChemicalName(
          queryName, false
        );
      if (result.getStatus()
          == OPSIN_RESULT_STATUS.SUCCESS) {
        Set inchis = new HashSet();
        String inchi = NameToInchi
          .convertResultToInChI(
            result, false
          );
        inchis.add(inchi);
        return inchis;
      }
    } catch (NameToStructureException e) {
      e.printStackTrace();   
    }
    return Collections.emptySet();
  }

  public String getInChIforShortestSMILES(
    String queryName)
  {
    Set inchis = getInChI(queryName);
    if (inchis.size() == 0) return null;
    return inchis.iterator().next();
  }

  // since names are converted on the fly, we do
  // not enumerate them
  public Set getNames(String inchi) {
    return Collections.emptySet();
  }
  public Set getNames() {
    return Collections.emptySet();
  }
  public Set getOrphanNames() {
    return Collections.emptySet();
  }
  public Set getChemRecords() {
    return Collections.emptySet();
  }
  public boolean hasOntologyIdentifier(
    String identifier)
  {
    // this ontology does not use ontology
    // identifiers
    return false;
  }
}

Now, you can implement the interface in various ways. You can even have the implementation hook into a SQL database with JDBC, or use something else fancy. The dictionary will be used at various steps of the Oscar4 text analysis workflow.

Mind you, the refactoring is not over yet, and the details may change here and there.

Your comments are most welcome!

So, what are the API requirements set out? These include (but are not limited to):

• have reasonable defaults
• being able to add custom dictionaries
• easily change the chemical entity recogniser
• plugin text normalization (see Peter’s post on UNICODE )

This week I worked on the dictionary refactoring, and talked with Lezan about the ChemicalTagger and trying to get this based on the newer Oscar code (I think we’ll be able to finish that today). So, I cleaned up some code I did in the first week, and introduced a Oscar class providing a Java API to the Oscar functionality.

So, to get started with Oscar in your application, you only need to do:

Oscar oscar = new Oscar(
  this.getClass().getClassLoader()
);
oscar.loadDefaultDictionaries();
Map<NamedEntity,String> structures =
  oscar.getNamedEntities(
    "Ingredients: acetic acid, water."
  );

The ClassLoader is needed because the Oscar class will not generally know how to load custom classes.

You can add additional dictionaries, by implementing the IChemNameDict interface and one or more of IInChIProvider, ISMILESProvider, and ICMLProvider. For example, adding the OPSIN dictionary would extend the above code to:

Oscar oscar = new Oscar(
  this.getClass().getClassLoader()
);
oscar.loadDefaultDictionaries();
oscar.getChemNameDict().register(
  new OpsinDictionary()
);
Map<NamedEntity,String> structures =
  oscar.getNamedEntities(
    "Ingredients: acetic acid, water."
  );

And, I think the oscar.getChemNameDict() method will be renamed to something like oscar.getDictionaryRegistry() really soon.

• Bioclipse Manager for MyExperiment.org
• Bioclipse, RDF and defeasible reasoning (see also Samuel’s blog)
• Bioclipse and SPARQL end points #2: MyExperiment
• Bioclipse and SPARQL end points
• Solubility Data in Bioclipse #2: handling RDF
• Solubility Data in Bioclipse #3: Finding ChEBI IDs
• Solubility Data in Bioclipse #4: Finding ChEBI IDs (Again, but better)
• /me is having Bioclipse/XMPP/RDF fun

Or check this screenshot from a Posterous post about a MyExperiment workflow :

One thing I have not blogged about yet (I think), is that the Bioclipse RDF manager also understands RDFa now. Well, sort of… it relies on a webservice, but this is what the script looks like:

model = rdf.createStore()
rdf.importRDFa(model, "http://egonw.github.com/")
rdf.saveRDFN3(model, "/Virtual/egonw.n3")

With support of SPARQL end points, and reading RDF from web resources directly (RDF/XML, N3, RDFa), Bioclipse is ready for the chemical semantic web.

I haven’t actually gotten around to fiddling with Java, but ran Strigi against it to extract RDF, while having the Strigi-Chemistry plugins installed. This is part of the RDF that came out:

<example-doc.docx>
  <http://freedesktop.org/standards/xesam/1.0/core#title>
    "acetic acid",
    "(8R,9S,10R,13S,14S,17S)- 17-hydroxy-10,13-dimethyl- 1,2,6,7,8,9,11,12,14,15,16,17-dodecahydrocyclopenta[a] phenanthren-3-one",
    "testosterone";
  <http://freedesktop.org/standards/xesam/1.0/core#version>
    "2",
    "2";
  <http://rdf.openmolecules.net/0.9#atomCount>
    "8",
    "49";
  <http://rdf.openmolecules.net/0.9#bondCount>
    "7",
    "52";
  <http://rdf.openmolecules.net/0.9#molecularFormula>
    "C2H4O2",
    "C19H28O2";

I believe there is quite some room for improvement, but it’s a start :) Thanx to Joe for posting the public domain test file, so that other projects can start play with the exiting new technology. I should note, however, that I am not running a Microsoft OS nor MS-Word, and the saved documents source are the only way I have access to the CML right now.

Instead, PMD is given nice overviews of what it believes to be wrong with the CDK, and also provides a decent XML format which allows extraction of information, which is used by, for example, SuperNightly as showed yesterday in PMD 2.4.5 installed in the CDK 1.2.x branch.

I have been pondering about it for a long time now, but writing a JavaDoc checking library is hardly core cheminformatics research; at least, you would not get funding for it, despite everyone always complaining about good documentation. Alas.

Last week, I was reviewing some more code, and again saw the very common error of the missing period at the end of the first sentence in JavaDoc. This one is sort of important for proper JavaDoc documentation generation, but the complexity of the current DocCheck reporting, people are not familiar enough with it. Being tired of having to repeat myself, I decided to address the problen, but creating better Nightly error reporting for the CDK JavaDoc.

So, I started OpenJavaDocCheck, or ojdcheck. As mentioned, I have made quite promising progress, and the current version provides the ability to write custom tests (which I plan to use for validating content of CDK taglet content), and create XML as well as XHTML which can be saved to any file. To give you a glimps of where things are going, here’s a screenshot of the current XHTML output:

The current list of tests is really small, and consists of a single test:

• test if each class and method has JavaDoc

These are my current results:

Non-free packages installed on egonw-laptop

fglrx-modaliases          Identifiers supported by the ATI graphics driver
linux-generic             Complete Generic Linux kernel
linux-restricted-modules- Non-free Linux 2.6.28 modules helper script
linux-restricted-modules- Restricted Linux modules for generic kernels
skype                     Skype - Take a deep breath
sun-java5-bin             Sun Java(TM) Runtime Environment (JRE) 5.0 (architectu
sun-java5-demo            Sun Java(TM) Development Kit (JDK) 5.0 demos and examp
sun-java5-jdk             Sun Java(TM) Development Kit (JDK) 5.0
sun-java5-jre             Sun Java(TM) Runtime Environment (JRE) 5.0 (architectu

            Contrib packages installed on egonw-laptop

flashplugin-installer     Adobe Flash Player plugin installer
flashplugin-nonfree       Adobe Flash Player plugin installer (transitional pack
msttcorefonts             transitional dummy package
ttf-mscorefonts-installer Installer for Microsoft TrueType core fonts

  9 non-free packages, 0.4% of 2050 installed packages.
  4 contrib packages, 0.2% of 2050 installed packages.

So, I am now writing a patch which replaces the use of int in IBond.getStereo(). But instead of going for Integer, the patch is actually going to use a enumeration.

Now, getting the the Art of Programming… while writing patches in the CDK, you run into those lovely bits of code, where intention is mixed with implementation details. They should not, and often do not need to, but they typically do. This is actually one reason why we now have a more strict peer-review installed. Below are two nice examples where intention is mixed with implementation detail.

Example 1

int stereo = container.getBond(chiralNeighbours.get(i), atom).getStereo();
if (stereo == 0) {
  // do something
}

This code is bad because we have no clue of what this code is supposed to do. When should the if-clause kick in? Be reminded that the int = 0 has the confounded meanings of no stereochemistry and perhaps has stereochemistry, but no one ever bothered telling me. So, which of the two situations does the if clause apply to. So, my patch can only assume that both were applicable (following the actual implementation), though I don’t think that makes sense on an algorithmic level. Had the author used CDKConstants.STEREO_BOND_NONE (which is the implementation for int = 0 for no stereochemistry, then I had known what the implementation was doing. Instead, the author chose to reuse implementation details: a hardcoded 0.

Example 2

There is another instance of this problem. Look at this lovely piece of code:

IBond bond = molecule.getBond(atomA, unplacedAtom);
if (Math.abs(bond.getStereo()) < 2
    && Math.abs(bond.getStereo()) != 0) {
}

This example also uses hardcoded value, instead of the matching constants. Remember that int = 0 had the meaning of no stereochemistry, so I assume this code is determining is stereochemistry is defined for the bond, making nice use that those situations at some point were coded as non-zero values. Moreover, it is only interested in a few stereochemistry definitions, and from the implementation I learn (and that actually makes sense at this location) that it is only interested in those stereochemistry for which the first bond atom is the stereochemical center. This again is leaking implementation details, instead of using semantically meaningful constants.