With this letter we take a principled stand against the proliferation of so-called ‘AI’ technologies in universities. As an educational institution, we cannot condone the uncritical use of AI by students, faculty, or leadership. We also call for reconsidering any direct financial relationships between Dutch universities and AI companies.The unfettered introduction of AI technology leads to contravention of the spirit of the EU Al act. It undermines our basic pedagogical values and the principles of scientific integrity. It prevents us from maintaining our standards of independence and transparency. And most concerning, AI use has been shown to hinder learning and deskill critical thought.

These few lines contain for me more than 25 years of research and I know the complexities. Before I can co-sign this letter, I need to understand the details. There is no definition of ‘AI’ here and it mentiones the EU AI Act (I guess, the letter actually writes “Al” (with an l of letter) act, I notice now after I read the content in another font), but I have not read the EU AI Act yet (it is 144 pages of legal text).

The legal context of the Open Letter

Let me first say, I am not a lawyer (IANAL). I am not versed in the specific legal definitions of tightly defined and controlled words.

Reading the EU AI Act, I read a reassuring opening statement (repeated later with more context, links to other laws, etc):

to promote the uptake of human centric and trustworthy artificial intelligence (AI) while ensuring a high level of protection of health, safety, fundamental rights as enshrined in the Charter of Fundamental Rights of the European Union (the ‘Charter’), including democracy, the rule of law and environmental protection, to protect against the harmful effects of AI systems in the Union

We clearly see how these things are currently routinely violated.

This Regulation does not apply to AI systems or AI models, including their output, specifically developed and put into service for the sole purpose of scientific research and development.

In Dutch this is officially translated to “wetenschappelijk onderzoek”, so scientific research seems to be legally including humanites, etc, and not limited to natural sciences [citation needed].

The EU AI Act also outlines a definition of “AI”, leaning towards machine learning, but the border between deterministic, rule-based algorithms and machine-learned patters for predictions remains a bit vague to me. But I can live with it.

The Open Letter’s contravention of the spirit of the EU Al act gets context here too. It has to be the spirit, because the law does not apply to academia. Good, clarified. The Letter continues with:

It undermines our basic pedagogical values and the principles of scientific integrity. It prevents us from maintaining our standards of independence and transparency. And most concerning, AI use has been shown to hinder learning and deskill critical thought.

Yes, that clearly links to the EU AI Act’s protection of rights. Maybe on purpose and maybe there are legal reasons to not explicitly list them, are the international human rights, which includes rigths to benefit from science, but I think this is still in the spirit of the EU AI Act. And if AI fetters our ability to learn (yes, there is scientific evidence for that [citation needed]), then it violates the EU AI Act (IANAL).

What the Open Letter expects

The next part of the Open Letter calls to what the signers expect from our universities. I will will reflect on each of them.

Resist the introduction of AI in our own software systems, from Microsoft to OpenAI to Apple. It is not in our interests to let our processes be corrupted and give away our data to be used to train models that are not only useless to us, but also harmful.

The intrinsic problem and why I think it is fair to call out these companies, is, as the letter explains, there is an clear conflict of interest. The goal of companies is to make profit (and in a Western world, as much as possible), and not any of the human or scientific needs. In this respect, companies like Elsevier could just as well have mentioned too (see e.g. this post by Prof. Van Rooij, actually 2nd signature on the letter).

Ban AI use in the classroom for student assignments, in the same way we ban essay mills and other forms of plagiarism. Students must be protected from de-skilling and allowed space and time to perform their assignments themselves.

About a year ago, I was pleasently surprised by the depth of discussion at Maastricht University on how and when to use AI, and by default not. This one is really complicated and it matters when and how the AI is used. After all, and the spirit of the EU AI Act expects us to use AI in research (to trigger innovation). So, I cannot agree with the literal statement, but I fully agree with the spirit. Particularly combined with the clear “Stop the Uncritical Adoption of AI Technologies in Academia” of the title of the Open Letter.

I read this line like this, AI in the classroom must have a purpose that aligns with the EU AI Act. That means, use for writing assays, reports, it must not be used. I am old enough that remember the academic discussions (at Radboud University) about writing and the clear hesitance among scholars about the use of written assignments: “I want to test their scientific knowledge and reasoning skills, not their ability to write narratives”. And LLMs, like ChatGPT but also the European, more open variants, they write narratives, so the written report and assay is no longer a valid way to assess a student’s scientific learning progress.

So, alternatively, we should very carefully and scientifically evaluate which forms of assessment we perform, and banning AI in the classroom may just be distracting from a more fundamental problem. Anyways… if you continue using writing assignments to test progress in learning, you must ban use of AI in that process. You must be testing the student, not some piece of software (as a teaching institute).

Cease normalising the AI hype and the lies which are prevalent in the technology industry’s framing of these technologies. The technologies do not have the advertised capacities and their adoption puts students and academics at risk of violating ethical, legal, scholarly, and scientific standards of reliability, sustainability, and safety.

Sounds like a no brainer. But I too find my own university uncritically promoting AI. Maybe the tested it well, and just forgot to share that. But hey, scientifical quality goes all ways.

Fortify our academic freedom as university staff to enforce these principles and standards in our classrooms and our research as well as on the computer systems we are obliged to use as part of our work. We as academics have the right to our own spaces.

Again, a no brainer. But important to add. It must be said as it is intrisic part of Recognition & Rewards. If you cannot guarantee academic freedom, there there is something seriously wrong with your R&R.

Sustain critical thinking on AI and promote critical engagement with technology on a firm academic footing. Scholarly discussion must be free from the conflicts of interest caused by industry funding, and reasoned resistance must always be an option.

Yeah, this is something that is underestimated. Part of our academic teaching is this critical thinking. It returns in academic reading (did you already read “What Little Red Riding Hood Can Teach Us about Reading Science”, doi:10.1515/9783110782844-010, by Monica Gonzalez-Marquez et al.?), scientific programming, data analysis, and our teaching has been lacking here. Not just for new AI forms, but also for the old algorihmts. I have seen this, and scientific literature is riddled with mistakes, just because our peer reviewers are not sufficiently skilled. This will take effort. I know, it was a major part of my PhD thesis.

Of course, this is exactly why I have been so active in Open Science. Without Open Science, we cannot work in the spirit of the EU AI Act. It’s nothing new. It’s just that the big money has found in AI a way to profit at the expense of humans.

So, go read that Open Letter and sign too!

Now, the first principle was about the need for access to data (McPrinciple #1). This principle goes without saying, one would think, but is not widely accepted yet. This is why Open Data promotion is still needed. For example, data in papers still is not freely redistributable, as Peter points out once again.

Anyway, this post is not about McPrinciple #1, but about the second principle.

Molecular Chemometrics Principles #2: In order to reproduce cheminformatics studies you need to be able to understand the input data.

Readers of my blog will surely recognize this theme. Clearly this theme explains my past fetish for the Chemical Markup Language, and my more recent work on the Resource Description Framework.

And it is so easy to jump to conclusions. Easy to make mistakes. And this is not just at the received side; the sending person may have accidentally made a mistake, or left something accidentally unclear, causing incorrect assumptions, and therefore errors in the cheminformatics computation. Now, if the data was semantically (clearly) annotated, and the meaning was clear, it was also trivial to see when a mistake had sneaked in. Think of it as a check bit.

“Well, isn’t this a bit exaggerated,” you might say. Perhaps, perhaps not. An simple, recent example. We all know SMILES, right? And we all know that lower case element symbols indicate aromaticity, right? That is, c1ccccc1 is aromatic, right? So, what’s the problem then?

Now, consider the SMILES string c1ccc1. Lower case carbon element symbols, so aromatic, right? Oh, wait…

Therefore, be clear in what you mean. It saves us from a lot of trouble.

Further reading:

• The Molecular Chemometrics Principles #1: access to data
• Molecular Chemometrics, 2006 (doi:10.1080/10408340600969601)

Indeed, I am not particularly one of those bloggers who spends trees after trees, in great detail explaining what is going on. I do make a lot of use of hyperlinking; much more than the average blogger. I actually assume that readers follow links, to read about the perspective of a blog post. But we all know that scientists do not read the cited papers in a paper they are reading, so who am I to assume blog readers would start doing that with blogs :)

Well, since principles seems popular , it might be a good start of my grand scheme that is behind this blog: the Molecular Chemometrics Principles. Hence, this first post about the why. The why is simply to provide a reference frame to what I am blogging about. In the next few posts on these McPrinciples (is that a catchy name, or what?) that will appear over the next two weeks, I will outline the code of chem-bla-ics. And, moreover, from now on, I will tag all my posts with the reaons why I make that post. I am sure that will not be too helpful for the occasional reader, but for anyone who is serious about chem-bla-ics, this will be a genuine gold mine of data for pattern recognition and data mining otherwise.

So, here goes.

Molecular Chemometrics Principles #1: In order to reproduce cheminformatics studies you need access to the input data.

The reason for this is that statistical modeling very much depends on the data on which modeling was done, patterns were recognized, etc. Therefore, without the input data, it is practically impossible to accurately reproduce results. Fortunately, the acceptance of the importance of access to data (e.g. as Open Data) is slowly getting momentum in science.

Further reading: Molecular Chemometrics, 2006 (doi:10.1080/10408340600969601)

The paper reads quite nicely, and the results are clear: by combining spectral types, the RMSEP goes down. Good! Lower prediction errors; that’s what we all want. So, a M.Sc. student of mine set off, but after about half a year, he was still unable to make statistically good models. He used bootstrapping to ‘prove’ it was not his fault: there was not enough data for the method to learn the underlying patters. Hence my above lesson. My student went on with larger data sets, and laid out the foundation of what later became the paper on using NMR spectra in QSPR modeling (doi:10.1021/ci050282s). Now you understand why QSAR is missing.

So, if those results are so clear, then why does it not work? As said, the data set was too small for pattern recognition methods to see what was going on. The RMSEP numbers just came out nicely; however, if we had only made the below plot, if would have warned. But I failed to do that at the time. Lesson learned: do not just look at the data, but also look at the model. And look really means looking with your eyes at graphical representations of that model. The plot:

The numbers in this plot are hidden in tables in the paper. The RMSEP values earlier mentioned are calculated from those. From the plot, you can see that the test data consisted of 5 compounds; the training set contained 37 compounds; all are congenerics, and do not span a high diversity. Now, the plot shows five models: black is COMFA; orange is based on experimental IR spectra; red, green, and blue are models where two types of representations are combined. From the RMSEP values it can be seen that combining representation improves the RMSEP values. That’s what you want, and sort of makes sense.

Now, I did not make this plot until I started writing up the paper, and tried to figure out why the QSAR data set did not work. My eyes opened wide when I saw the orange dots! Anti-correlation! WTF?!?! I mean, we are looking at a plot visualizing the predicted versus the experimental activity… Actually, the others are not really convincing either, are they? Looking at the predictions for compounds with experimental values around 1.0-1.5 (if you really want to know the unit, read the article), the pattern is pretty much anti-correlated too. Thinking about it, it seems the RMSEP is mainly reflecting the error of the left most compound, the one with a experimental activity of about 0.3.

Clearly, the orange model is hopeless, but the others are not really better. Now, the paper actually makes statements comparing the various combinations of representations, but, in retrospect and looking at this plot, I wonder if the green model is really different from the blue or red models.

Since then, I always make these kind of plots, just to see what my model is like. Since then, I distrust papers that only show RMSEP, Q², or other quality statistics. Now, the tricky part is, you need those statistics if you want to automate model selection; the variance on those model quality statistics is actually so high (see also my other post today), that you must carefully validate that model selection too, visually of course.

I have been long thinking what to do with these observations. I did not dare publish them in my thesis; I did not dare write a letter to the editor. Perhaps I should. But even writing up this blog makes me feel uncomfortable. Besides the fact that I might be wrong, I also do not like to point out mistakes (IMHO); particularly, when those are published in a respectable journal. I was fooled by the statistics too (and was already well trained), so I cannot comment on the authors overlooking the issue. Or the reviewers! Or the community at large. Also, I do not know what the fate of this paper should be. The idea is quite interesting, even though the published results do not support it. Not shown here, but the bootstrapping results show that the apparent slight improvement is merely a numerical artifact, just happening by chance, based on luckily selecting the test compounds; the data is just insufficient in size to draw any conclusion.

• The CDK/Metabolomics/Chemometrics Unconference results
• When to stop including QSAR model variables…

Given the prominent place QSAR has in my thesis , this is somewhat surprising. Anyway, here is some more QSAR modeling talk.

Gilleain implemented the signature descriptors developed by Faulon et al. (see doi:10.1021/ci020345w; I mentioned the paper in 2006), and the CDK patch is currently being reviewed. With some transformations, the atomic signatures for a molecule can be transformed into a fixed-length numerical representation: [70:1, 54:1, 23:1, 22:1, 9:9, 45:2]. This string means that atomic signature 70 occurs once in this molecule and signature 9 occurs nine times. At this moment, I am not yet concerned about the actual signature, but just checking how well these signature can be used in QSPR modeling.

Rajarshi’s fingerprint code provides a good template to parse this into a X matrix in R:

For my test case, I have used the boiling point data I used in my thesis paper On the Use of 1H and 13C 1D NMR Spectra as QSPR Descriptors (see doi:10.1021/ci050282s). Some of this data is actually available from ChemSpider, but I do not think I ever uploaded the boiling point data. This constitutes a data set with 277 molecules, and my paper provides some reference model quality statistics; that way, I have something to compare against. Moreover, I can use my previous scripts to do the PLS modeling (there are many tutorials online, but you can always buy an expensive book like the one shown on the right, if you really have to), (10-fold) cross-validation (CV), and 5 repeats of random sampling.

I strongly suggest people interested in statistical modeling to read this interesting post from Noel: whatever test set sampling method you use, you must do some repeats to learn about the sensitivity of your modeling approach to changes in the data set. Depending on the actual sampling approach, you might see different sizes of variance, but until you measure it, you will not know. For my application, these are the numbers:

# source("pls.R")
Read 277 items
   rank=66     LV=42     R2=0.987     Q2=0.921  RMSEP=31.37
   rank=66     LV=42     R2=0.983     Q2=0.924  RMSEP=12.405
   rank=65     LV=42     R2=0.985     Q2=0.949  RMSEP=38.503
   rank=63     LV=42     R2=0.983     Q2=0.948  RMSEP=36.981
   rank=65     LV=42     R2=0.986     Q2=0.923  RMSEP=21.49
   rank=64     LV=42     R2=0.983     Q2=0.91  RMSEP=17.759
   rank=64     LV=42     R2=0.983     Q2=0.921  RMSEP=17.062
   rank=66     LV=42     R2=0.986     Q2=0.94  RMSEP=40.311
   rank=66     LV=42     R2=0.982     Q2=0.927  RMSEP=13
   rank=68     LV=42     R2=0.986     Q2=0.929  RMSEP=16.23

I know 42 is the answer to the universe, but 42 latent variables (LVs)?!? Well, it’s just a start. A more accurate number of LVs seems to be around 15, but my script had to make the transition from the old pls.pcr package to the newer pls package. And I have yet to discover how I can get the new package to return me the lowest number of LVs for which the CV statistic is no longer significantly different from the best (see my paper how that works). Actually, I have set the maximum LVs to consider to 1/5th of the number of objects (which is about the accepted ratio in the QSAR community); otherwise, it would have happily increased.

However, the five repeats nicely show the variance in the quality statistics, R², Q², and root mean square error of prediction (RMSEP). From the numbers, a model with Q² = 0.94 is not better than one with Q² = 0.93 (and I have seen the variance quite some larger). Bottom line: just measure that variability, and put it in the publication, will you??

Anyway, what we all have been waiting for: the prediction results visualized (in black the CV predictions; in red the test set predictions):

Well, there is still much work to do, and you can expect the result to get better. Part of statistical modeling is to find the source of variance, and I have yet to explore a few of them. For example, what are the effects of:

• creating signature from the hydrogen-depleted graph
• effect of tautomerism (see this special issue)
• effect of the height of the signature

And there are so many other things I like to do. But this will do for now.

More blogging when I have more frequent internet access again…

We used the R statistics software together with Rajarshi’s rcdk package (an R wrapper around the CDK library) and Ron’s (my PhD supervisor) PLS package (see this paper), to predict retention indices for a number of metabolites.

We ended up with this R script:

library("rJava")
library("rcdk")
library("pls")
mols = load.molecules("data_cdk.sdf")
selection = get.desc.names()
selection = selection[-which(selection=="org.openscience.cdk.qsar.descriptors.molecular.AminoAcidCountDescriptor")]
x = eval.desc(mols, selection, verbose=TRUE)
x2 = x[,apply(x, 2, function(a) {all(!is.na(a))})]
y = read.table("data_cdk_RI")
input = data.frame(x2, y)
pls.model = plsr(V1 ~ ., 50, data=input, validation="CV")
summary(pls.model)
plot(RMSEP(pls.model))
plot(pls.model, ncomp=20)
abline(0,1, col="red")
plot(pls.model, "loadings", comps=1:2)
savehistory("finalHistory.R")

The AminoAcidCountDescriptor threw us a NullPointerException and there were a few NAs in the resulting matrix. The CV results were not so good as Velitchka’s best models, but still a good start:

No variable selection; 200 objects, 190 variables.

Questions:

• Can we do this in Bioclipse2 too?
• Can we improve the default CDK descriptor parameters to maximize the column count?
• Rajarshi, what would be involved to write some wrapper code for atomic descriptors for rcdk?

With this blog item, I would to thank everyone who left a message, sent email, etc with good luck messages. Very much appreciated! I’d also like to thank my supervisors, promotores Lutgarde Buydens and Peter Murray-Rust (he mentions the event here), and Ron Wehrens for their confidence in me and their guidance on the path towards the post-doc life. I also thank all those who attended my defense; I had a brilliant day, and actually enjoyed talking to those who took place in my promotion committee and who asked me the not-really-nasty-questions about my work.

CDK-Chemometrics in Metabolomics Unconference

For today, I organized a small, informal unconference, oriented around the CDK, chemometrics and metabolomics. I’m certain we will be online much of the day, as we typically do. The meeting will start around 10:00 CEST, but we’ll attend a seminar by Marjana Novič at 11:00 CEST. If you happen to be in Nijmegen, just drop in on the Analytical Chemistry department. Otherwise, join the #cdk chat channel in the irc.freenode.net network.

What we’ll do?? Hey, it’s an unconference; we have no idea yet :)

Table of Contents

1. Introduction
2. Molecular Chemometrics (doi:10.1080/10408340600969601)
3. 1D NMR in QSPR(doi:10.1021/ci050282s)
4. Comparing Crystals (doi:10.1107/S0108768104028344)
5. Supervised SOMs (doi:10.1021/cg060872y)
6. Chemical Metadata in RSS (doi:10.1021/ci034244p)
7. Interoperability (doi:10.1021/ci050400b, the Blue Obelisk paper)
8. Discussion and Outlook

Chapters 2, 3, 4, and 5 are first author papers, while for chapters 6 and 7 I am just co-author.

Summary

Chemometrics and chemoinformatics play important roles in the analysis and modeling of molecular data. In particular, in understanding and prediction of properties of molecules and molecular systems. Both chemometrics and chemoinformatics apply statistics, machine learning and informatics methodologies to chemical questions, though originating from a different background. Where chemometrics had its origins in the extraction of information from chemical experiments, chemoinformatics had roots in the representation of chemical data for storage in databases. The technological advances in chemistry and biochemistry in the past decades have led, however, to a flood of data and new questions, and the data analysis and modeling have become more complex. The standing challenge in data analysis and data exchange, is how to represent the molecular features relevant to the problem at hand. This representation of molecular information is the topic of this thesis.

Chapter 1 introduces the field of data analysis and modeling of molecular data and describes the aforementioned importance of representation of relevant features. It discusses different approaches to molecular representation, such as line notations, chemical graphs, and quantum chemical models. Each of these have limitations when used in data analysis and modeling. Numerical representations are then introduced, which allow the application of statistical and mathematical modeling approaches. These numerical representations are commonly derived from chemical graph and quantum chemical representations. CoMFA and the classification of enzyme reactions are examples were the choice of molecular representation as well as the analysis method are important.

The term molecular chemometrics is coined in Chapter 2 for the field that applies statistical modeling methods to molecular structure. It reviews the advances made in this field in recent years. New numerical descriptors for molecules are discussed, as well as approaches to represent molecules in more complex systems like crystal structures and reactions. Molecular descriptors are used in similarity and diversity analysis. The applications of new methods for structure-activity and structure-property modeling, and dimension reduction are described. An overview of recent approaches in model validation show new insights and approaches to estimate the performance of classification and regression models. The last section of this chapter lists new databases and introduces new methods that improve the extracting of chemical data from database and repositories. Semantic markup languages improve the exchange of data, and new methods have been introduced to extract molecular properties from text documents.

Chapter 3 studies the in literature proposed use of 1D ¹³C and ¹H NMR spectra as molecular descriptor. These spectra are known to describe features relevant to physical properties like solubility and boiling point. The NMR representation is studied for the predictive powers of its PLS models for three structure-property data sets. The results indicate that proton NMR is not suitable for building QSPR models in combination with PLS. Carbon NMR-based models, however, do give reasonable QSPR models, and the regression vectors for the carbon NMR data, correlate with spectral regions relevant to molecular fragments. Nevertheless, the predictive power of the carbon NMR-based spectra is still less than models based on common molecular descriptors. It is concluded that NMR spectra should not be considered first choice when making predictive models in general, and that proton NMR should probably not be used at all.

A computational method to calculate similarities between crystal structures based on a new representation is introduced in Chapter 4. While a reference method is perfectly able to identify structures with high similarity, it fails to recognize the different similarities between two similar structures and two completely different structures. This makes it very difficult for clustering algorithm to organize small clusters of identical and highly similar structures into larger clusters. The new representation of crystal structures introduced in this chapter shows a much smoother transition in similarity values when crystal structures go from identical, via similar, and finally to dissimilar structures. Clustering a set of simulated polymorphic structures of estrone, and classification of a set of experimental cephalosporin structures reproduce expected clustering and classification.

Chapter 5 uses supervised self-organizing maps to cluster crystal structures represented by their powder diffraction pattern and one or more properties. The topological structure of the resulting maps not only depends on the similarity of the diffraction data, but also on the properties of interest, such as cell volume, space group, and lattice energy. This approach is used to analyze and visualize large sets of crystal structures, and the results show that these supervised maps not only give a better mapping, they can also be used to predict crystal properties based on the diffraction patterns, and for subset selection in polymorph prediction. The two applications in crystallography show that suitable representations and similarity measures that allow data analysis and modeling of molecular crystal data are now available. Both approaches are flexible enough to open up a new field of research; especially combinations with other classification schemes for crystal structures, such as those based on hydrogen bonding patterns, come to mind.

Chapter 6 introduces and discusses a method that allows information rich distribution of molecular data between machines, such as measuring devices and computers. Existing approaches often imply not or badly documented semantics which may lead to information loss. CMLRSS is proposed and combines two existing web standards: Rich Site Summaries (RSS), also known as RDF Site Summaries, and the Chemical Markup Language (CML). Here, RSS is used as transport layer, while CML is used to contain the chemical information. CML supports a wide range of chemical data, including molecular (crystal) structures, reaction schemes, and experimental data such as NMR spectra. It is shown that this semantic representation allows automated dissemination of chemical data, and is increasingly used to exchange data between web resources.

Chapter 7 describes a communal effort to realize interoperability in chemical informatics, which is called the Blue Obelisk movement. This movement currently consists of more than ten smaller and larger, open source and open data projects all related to chemoinformatics and chemistry in general. To increase the reproducibility of molecular representations, this chapter introduces a collaborative dictionary of chemoinformatics algorithms, and a public repository of chemical data of general interest, including data for chemical elements and isotopes, (boiling points, colors, electron affinities, masses, covalent radii, etc.), definitions of atom types, and more. The availability of a standard set of atomic properties, open source algorithms and open data (for example via CMLRSS feeds), it is much easier to reproduce and validate published results in molecular chemometrics. Results from Chapter 3 show that such ability is no luxury.

The last chapter summarizes the efforts in this thesis and how they address the challenges in molecular chemometrics. This thesis shows the strong interaction between representation and the methods used for data analysis: molecular representation need to capture relevant information and be compatible with the statistical methods used to analyze the data. The chapters review molecular representations and put focus on model validation using statistics, visualization methods, and standardization approaches.