![\"\"](\"/assets/images/wp_cited_in.png\")
I have been wanting to blog about this since this summer, but with everything going on, I never really got around to it.\nWhat is this Cited In feature of WikiPathways and where does that information come from?\nIf you have not noticed this yet, this is what it looks like for WP4846:
\n\n
Recently, I was close to writing up the context, because it is related to a new feature of the profile pages, where you\nnow can look up citations to pathways that you first authored (see\nthis post).\nAnd it also relates to the data I have been collecting around citation intention annotations:\narticles that cite one of the WikiPathways papers and mention a specific pathway, could be considered cito:usesDataFrom\n(see doi:10.1186/s13321-023-00683-2).
\n\nA third angle to citations to specific WikiPathways is the following. WikiPathways is used a lot in data analyses and\nputting experimental data in biological context. How researchers do this varies a lot, in multiple ways. But just\nthinking about this factually, research output cite specific biological pathways. And there are some interesting\nphenomena there. Back in 2015 at the Metabolomics Society meeting in San Francisco (apparently, I only\nblogged about the meeting only once?),\nwhen I visited the 500+ posters looking for interesting biological pathways, there were a lot of studies\non different species, different diseases, different toxicities. The biological response had one thing in common:\nit always was the TCA cycle that was key (see doi:10.1096/FJ.11-203091 for\na 2012 comparison of TCA models).
\n\nThus, with so many articles mentioned specific pathways and deriving biological knowledge from this, what is\nreasonable to expect? Do we expect co-citation effects? That is, if two articles found the same set of pathways\nof interest to their data, is the data showing a similar biological response? Do we expect a similar thing\nlike the above TCA cycle in metabolomics, something similar to the notion of frequent hitters (see\ndoi:10.1021/jm010934d)?
\n\nOf course, to test this hypothesis we need data and the Cited In feature comes in. At the time of\nwriting of this blog post, we can see on this page\nthat 878 pathways have been cited a total of 2715 times. We are getting somewhere. This blog\npost will not analyze this data, which is one reason why I had not blogged about it. But from the\nabove you can understand that I want to :)
\n\nThis Cited In feature was introduced along with the new website (see doi:10.1093/nar/gkad960),\nwhere we change how GPML files are stored and how web pages are created from that.\nBecause we are no longer confined to the MediaWiki platform (which has served the project for very long,\nvery effectively), it is easier to integrate information from other sources. For example,\nfrom literature databases. This feature was developed by Alex Pico\nat the Gladstone Institutes (see this 2022 commit),\nwhere he uses the NCBI eUtils API to access\nPubMed Central.\nThe data is then collected into this YAML file\nwhich then gets used to generate webpage content (like the section in the above screenshot\nand the page mentioning the current statistics).
\n\nAs just explained, originally the data was only coming from NCBI.\nHowever, because I found many articles citting specific pathways that were not picked up by this\napproach, and I wanted more data, so I started searching Europe PMC the European\npartner of PubMed Central. However, I am not automating this. I want to see the data, the articles, and\nhow people cite the pathways. I need to see that so that I can better understand how people are\nusing the data/knowledge from WikiPathways. I cannot keep up with checking why people are citing\nmy own research, but I once was.\nI learn(-ed) a lot from that.
\n\nI normally use a search that requires the word “WikiPathways” to be\nmentioned in the article (in most, but\nnot all of them; citing literature you extend sounds like a core scholarly value, but is factually\nnot systematically complied with), and then manually searching for “WP”. With close to 1000\nPubMed Central articles mentioning WikiPathways in 2025 and that these are mostly full texts,\nI can see if the cite specific pathways. A good number of article mentions the WikiPathways\nidentifier, e.g. the aforementioned WP4846. If the article only mentions a pathway title,\nI cannot confidently identify which pathway is cited, so I exclude that.
I originally started out manually editing the YAML file where the citations are collected,\nbut by now use a script similar to Alex’ R script.\nThis makes it far easier to scale up, as I just have to populate a three column TSV file,\nwhich is used by my R script to update the YAML file. This manual approach ensures that\nI am not looking at text mining results, but see the citation of the WikiPathways identifier\nwith my own eyes. That’s just how I like it.
\n\nThe full history of the YAML file content can be found on this GitHub page\nand this git blame\ntells you if the information came from PubMed Central via the API, or was added by me:
\n\n![\"\"](\"/assets/images/wp_cited_in_git_blame.png\")
This is Open Science in action: added transparency and making it easier for anyone to verify,\nso that no one needs to be stuck in (dis)trust.
\n\nOf course, as we know from the CiTO ontology and real-world data, there are so\nmany different reasons why journal articles are cited (just an example),\nthe data in the YAML file and on the WikiPathways website in the Cited In feature\ndoes not have direct meaning. Just like a high citation count for an article or\neven a journal impact factor cannot be directly interpreted (despite so many researchers\njust blindly doing just that).
\n\nWell, while I did not do any analysis yet, and do not even know yet how much citations we need to\nreach some level of statistical significance, there are some observations I can mention:
\n\nAnd about biological interpretation, our group has long published that some genes with\ndifferential data mapping to a pathway does not imply that that pathway is really affected.\nGene-set enrichment and over-representation analysis are a starting point; not a conclusion.\nI wish more people were more aware of the work in our (now)\nTranslational Genomics research group.\nLike that of Martina Kutmon (now as\nMaCSBio2),\nwhom I have had the pleasure of collaborating with for quite some years now (and long time\narchtect of WikiPathways).
\n\nThere is so much more I want to write up about WikiPathways, but I leave it to this\nfor now.
\n\n