![](\"/assets/images/jenaSlow.png\")
\n\n\n
Doing searches in RDF stores is commonly done with SPARQL queries. I have been using this with the semantic web translation of WikiPathways\nby Andra to find common content issues, though sometimes combined with some additional Java code.\nFor example, find PubMed identifiers that are not numbers.
\n\nBased on Ryan’s work on interactions, a more complex curation query I\nrecently wrote in reply to issues that Alex ran into with converting pathways to\nBioPax, is to find interactions that convert a gene to another gene. Such occurred in WikiPathways\nbecause graphically you do not see the difference. I originally had this query:
\n\nSELECT (str(?organismName) as ?organism) ?page\n ?gene1 ?gene2 ?interaction\nWHERE {\n ?gene1 a wp:GeneProduct .\n ?gene2 a wp:GeneProduct .\n ?interaction wp:source ?gene1 ;\n wp:target ?gene2 ;\n a wp:Conversion ;\n dcterms:isPartOf ?pathway .\n ?pathway foaf:page ?page ;\n wp:organismName ?organismName .\n} ORDER BY ASC(?organism)\nThis query properly found all gene-gene conversions to be fixed. However, it was also horribly slow with my\nJUnit/Apache Jena set up. The queries runs very efficiently on the Virtuoso-based SPARQL end point.\nI had been trying to speed it up in the past, but without much success. Instead, I ended up batching the\ntesting on our Jenkins instance. But this got a bit silly, with at some point subsets of less than 100 pathways.
\n\nSo, I turned to twitter, and quite soon got\nthree useful\nleads. The first two suggestions did not help, but helped me rule out the problem.\nOf course, there is literature about optimizing, like this recent paper by Antonis (doi:10.1016/j.websem.2014.11.003),\nbut I haven’t been able to convert this knowledge into practical steps either. After ruling out these options (though I kept the\nsameTerm() suggestion), and realized it had to be the first two triples with the variables ?gene1 and ?gene2. So,\nI tried using FILTER there too,\nresulting with this query:
WHERE {\n ?interaction wp:source ?gene1 ;\n wp:target ?gene2 ;\n a wp:Conversion ;\n dcterms:isPartOf ?pathway .\n ?pathway foaf:page ?page ;\n wp:organismName ?organismName .\n FILTER (!sameTerm(?gene1, ?gene2))\n FILTER (?gene1 a wp:GeneProduct)\n FILTER (?gene2 a wp:GeneProduct)\n} ORDER BY ASC(?organism)\nThat did it! The time to run a query halved. Not so surprising, in retrospect, but it all depends on the SPARQL engine:\nwhich parts does it run first. Apparently, Jena’s SPARQL engine starts at the top. This seems to be confirmed by\nthe third comment I got. However, I always understood\nengine can also start at the bottom.
\n\nBut that’s not all. This speed up made me wonder something else. The problem clearly seems to engine approach to run\nparts of the query. So, what if I remove further choices in what to run first? That leads me to\na second observation. It helps significantly if you\nreduce the number of subgraphs it should later “merge”. Instead, if possible, use\nproperty paths. That again, about halved the runtime of the query.\nI ended up with the below query, which, obviously, no longer give me access to the pathway resources, but I can live\nwith that:
\n\nWHERE {\n ?interaction wp:source ?gene1 ;\n wp:target ?gene2 ;\n a wp:Conversion ;\n dcterms:isPartOf/foaf:page ?pathway ;\n dcterms:isPartOf/wp:organismName ?organismName .\n FILTER (!sameTerm(?gene1, ?gene2))\n FILTER EXISTS {?gene1 a wp:GeneProduct}\n FILTER EXISTS {?gene2 a wp:GeneProduct}\n} ORDER BY ASC(?organism)\nI’m hoping these two observations may help other with using Apache Jena with unit and integrated testing of RDF generation too.
\n\n