![\"\"](\"/assets/images/signaturePrediction.png\")
I had to dig deep to find posts on QSAR modeling. There are quite a few on QSAR in Bioclipse,\nbut that focuses on the descriptor calculation. In a quick scan, I could only spot two modeling posts:
\n\nGiven the prominent place QSAR has in my thesis ,\nthis is somewhat surprising. Anyway, here is some more QSAR modeling talk.
\n\nGilleain implemented the signature descriptors developed by Faulon et al.\n(see doi:10.1021/ci020345w; I mentioned the paper in 2006),\nand the CDK patch is currently being reviewed.\nWith some transformations, the atomic signatures for a molecule can be transformed into a fixed-length numerical representation:\n[70:1, 54:1, 23:1, 22:1, 9:9, 45:2]. This string means that atomic signature 70 occurs once in this molecule and signature 9 occurs\nnine times. At this moment, I am not yet concerned about the actual signature, but just checking how well these signature can be used\nin QSPR modeling.
Rajarshi’s fingerprint code provides a good\ntemplate to parse this into a X matrix in R:
\n\nFor my test case, I have used the boiling point data I used in my thesis paper On the Use of 1H and 13C 1D NMR Spectra as QSPR Descriptors\n(see doi:10.1021/ci050282s). Some of this data is actually available from ChemSpider,\nbut I do not think I ever uploaded the boiling point data. This constitutes a data set with 277 molecules, and my paper provides\nsome reference model quality statistics; that way, I have something to compare against. Moreover, I can use my previous scripts\nto do the PLS modeling (there are many tutorials online, but you\ncan always buy an expensive book like the one shown on the right, if you really have to), (10-fold) cross-validation (CV), and\n5 repeats of random sampling.
\n\nI strongly suggest people interested in statistical modeling to read this\ninteresting post from Noel: whatever test\nset sampling method you use, you must do some repeats to learn about the sensitivity of your modeling approach to changes\nin the data set. Depending on the actual sampling approach, you might see different sizes of variance, but until you measure it,\nyou will not know. For my application, these are the numbers:
\n\n# source(\"pls.R\")\nRead 277 items\n rank=66 LV=42 R2=0.987 Q2=0.921 RMSEP=31.37\n rank=66 LV=42 R2=0.983 Q2=0.924 RMSEP=12.405\n rank=65 LV=42 R2=0.985 Q2=0.949 RMSEP=38.503\n rank=63 LV=42 R2=0.983 Q2=0.948 RMSEP=36.981\n rank=65 LV=42 R2=0.986 Q2=0.923 RMSEP=21.49\n rank=64 LV=42 R2=0.983 Q2=0.91 RMSEP=17.759\n rank=64 LV=42 R2=0.983 Q2=0.921 RMSEP=17.062\n rank=66 LV=42 R2=0.986 Q2=0.94 RMSEP=40.311\n rank=66 LV=42 R2=0.982 Q2=0.927 RMSEP=13\n rank=68 LV=42 R2=0.986 Q2=0.929 RMSEP=16.23\nI know 42 is the answer to the universe, but 42 latent variables (LVs)?!? Well, it’s just a start. A more accurate number of LVs\nseems to be around 15, but my script had to make the transition from the old pls.pcr package to the newer pls package. And I have\nyet to discover how I can get the new package to return me the lowest number of LVs for which the CV statistic is no longer\nsignificantly different from the best (see my paper how that works). Actually, I have set the maximum LVs to consider to 1/5th of\nthe number of objects (which is about the accepted ratio in the QSAR community); otherwise, it would have happily increased.
\n\nHowever, the five repeats nicely show the variance in the quality statistics, R², Q², and root mean square error of prediction\n(RMSEP). From the numbers, a model with Q² = 0.94 is not better than one with Q² = 0.93 (and I have seen the variance quite some\nlarger). Bottom line: just measure that variability, and put it in the publication, will you??
\n\nAnyway, what we all have been waiting for: the prediction results visualized (in black the CV predictions; in red the test set\npredictions):
\n\n
Well, there is still much work to do, and you can expect the result to get better. Part of statistical modeling is to find\nthe source of variance, and I have yet to explore a few of them. For example, what are the effects of:
\n\nAnd there are so many other things I like to do. But this will do for now.
\n\n