• provide a Java API for identifier mapping
• provide ID mappings (two flavors: with and without semantic meaning)
• provide services (R package, OpenAPI webservice)
• track the history of identifiers

The last one is more recent and two aspects are under development here: secondary identifiers and dead identifiers. More about that in some future post. About the first and the third I am also not going to tell much in this post. Just follow the above links.

I do want to say something in this post about the actually identifier mapping databases, in particular those we distribute as Apache Derby files, the storage format used by the Java libraries. These are the files you download if you want mapping databases for PathVisio (doi:10.1371/journal.pcbi.1004085). BridgeDb has mapping files for various things and some example databases the data it maps between:

1. genes and proteins: Ensembl, UniProt, NCBI Gene
2. metabolites; HMDB, ChEBI, LIPID MAPS, Wikidata, CAS
3. publications: DOI, PubMed
4. macromolecular complexes: Complex Portal, Wikidata

The BridgeDb API is agnostic to the things it can map identifiers for.

Downloading mapping files: BridgeDb has an BioSchemas-powered web page with an overview of the latest released mapping files. It looks like this:

This webpage is the result from the cyber attack in late 2019, disrupting a good bit of the infrastructure. This is why we renewed the website, including the download page. The new page actually is hosted on GitHub as a Markdown file, but this is where things get interesting. The Markdown file is actually autogenerated from a JSON file with all the info. Everything, including the BioSchemas annotation is created from that. Basically, JSON gets converted into Markdown (with a custom script), which gets converted into HTML by a GitHub Action/Pages. So, when someone releases a new mapping file on Zenodo or Figshare, they only have to send me a pull request with updated JSON file.

Now, previously, downloading all released mapping files, for example for the BridgeDb webservice, was a bit complicated. The information was a HTML file generated by the webserver for a folder. No metadata. Nuno wrote code to extract the relevant info and download all the files. However, since the information is now available in a public JSON file, it is a lot easier. The following code uses wget and jq, two tools readily available on the popular operating systems. Have fun!

!/bin/bash

wget -nc https://bridgedb.github.io/data/gene.json
wget -nc https://bridgedb.github.io/data/corona.json
wget -nc https://bridgedb.github.io/data/other.json

jq -r '.mappingFiles | .[] | "\(.file)=\(.downloadURL)"' gene.json > files.txt
jq -r '.mappingFiles | .[] | "\(.file)=\(.downloadURL)"' corona.json >> files.txt
jq -r '.mappingFiles | .[] | "\(.file)=\(.downloadURL)"' other.json >> files.txt

for FILE in $(cat files.txt)
do
  readarray -d = -t splitFILE<<< "$FILE"
  echo ${splitFILE[0]}
  wget -nc -O ${splitFILE[0]} ${splitFILE[1]}
done

Actually, while writing this blog post, I notice the code can be further simplified.

var jsonData = JSON.parse(jsonString);

Now, we previously covered maps. Maps have keys and values: the keys unlock a particular value. For example, take this JavaScript:

var map = { "key": "value", "key2": "value2" };

We define here a key-value object, and we can access the two values with the two keys:

map["key2"]; // == value2

These examples are JavaScript source code. Not a string. The content of the map variable is a data structure. But when we communicate with a web service, we need a (string) serialization of the data model, because we cannot send around memory pointers (which a variable is) because they are only valid on a single machine.

This is where the JSON format comes in. We can convert the content of the above map variable into a string representation with this code:

var mapStringified = JSON.stringify(map);

which gives us the following output:

{"key":"value","key2":"value2"}

This string looks an awful lot like the JavaScript code we wrote earlier.

And, likewise we can convert the JSON string back into a JavaScript data model again, with:

var mapAgain = JSON.parse(mapStringified);

Now, I did warn you earlier that values can be lists and maps itself again, so consider this JSON example from Wikipedia:

{
    "id": 1,
    "name": "Foo",
    "price": 123,
    "tags": [ "Bar", "Eek" ],
    "stock": {
        "warehouse": 300,
        "retail": 20
    }
}

Here we see that the value behind the stock key is another map, and the value behind the tags key is a list. This creates a quite flexible serialization format, which is happily used by Open PHACTS. (And for the semantic web readers, yes, we can make JSON more semantic. The Open PHACTS LDA supports a “rdfjson” format.)