A random public domain picture with 10 in it.

1. Goodman A, Pepe A, Blocker AW, Borgman CL, Cranmer K, Crosas M, et al. Ten Simple Rules for the Care and Feeding of Scientific Data. Bourne PE, editor. PLoS Computational Biology. 2014 Apr 24;10(4):e1003542.
2. List M, Ebert P, Albrecht F. Ten Simple Rules for Developing Usable Software in Computational Biology. Markel S, editor. PLOS Computational Biology. 2017 Jan 5;13(1):e1005265.
3. Perez-Riverol Y, Gatto L, Wang R, Sachsenberg T, Uszkoreit J, Leprevost F da V, et al. Ten Simple Rules for Taking Advantage of Git and GitHub. Markel S, editor. PLOS Computational Biology. 2016 Jul 14;12(7):e1004947.
4. Prlić A, Procter JB. Ten Simple Rules for the Open Development of Scientific Software. PLoS Computational Biology. 2012 Dec 6;8(12):e1002802.
5. Sandve GK, Nekrutenko A, Taylor J, Hovig E. Ten Simple Rules for Reproducible Computational Research. Bourne PE, editor. PLoS Computational Biology. 2013 Oct 24;9(10):e1003285.

Regarding licensing, I can highly recommend reading this book:

1. Rosen L. Open Source Licensing [Internet]. 2004. Available from: https://www.rosenlaw.com/oslbook.htm

Regarding Git, I recommend these two resources:

1. Wiegley J. Git From the Bottom Up [Internet]. 2017. Available from: https://jwiegley.github.io/git-from-the-bottom-up/
2. Task 1: How to set up a repository on GitHub [Internet]. 2018. Available from: https://github.com/OpenScienceMOOC/Module-5-Open-Research-Software-and-Open-Source/blob/master/content_development/Task_1.md

ACE inhibitors

The first team (Mischa-Alexander and Hamza) focused on the ACE inhibitors (type:”drug class”) and the WP554 from WikiPathways. The use a tree structure to list inhibitors along with their activity:

The source code for this project is available under a MIT license.

Diabetes

The second team (Catherine and Moritz) looked at compounds hitting diabetes mellitus targets. They take advantage from the new disease API methods and first ask for all targets for the disease, and then query for all compounds. Mind you, the compounds are not filtered by activity, so it mostly shows interactions that real targets.

This product too is available with the MIT license.

Tuberculosis

The third project (Nadia en Loic) also goes from disease to targets and they looked at tuberculosis.

Asynchronous calls

If you know the ops.js, d3.js, and JavaScript a bit, you know that these projects are not trivial. The remote web service calls are made in an asynchronous manner: each call comes with a callback function that gets called when the server returns an answer, at some future point in time. Therefore, if you want to visualization, for example, compounds with activities against targets for a particular disease, you need two web service calls, with the second made in the callback function of the first call. Now, try to globally collect the data from that with JavaScript and HTML, and make sure to call the visualization call when all information is collected! But even without that, the students need to convert the returned web service answer into a format that d3.js can handle. In short: quite a challenge for six practical days!

While there are full debugging tools, achieving the task of finding where the bug is can often be reached with simpler means:

1. take notice of error messages
2. add debug statements in your code

Error messages

Keeping track of error messages is first starting point. This skill is almost an art: it requires having seen enough for them to understand how to interpret them. I guess error messages are the worst developed aspects of programming language, and I do not frequently see programming language tutorial that discuss error messages. The field can certainly improve here.

However, at least error messages in general give an indication where the problem occurs. Often by a line number, though this number is not always accurate. Underlying causes of that are the problem that if there is a problem in the code, it is not always clear what the problem is. For example, if there is a closing (or opening) bracket missing somewhere, how can the compiler decide what the author of the code meant? Web browsers like Firefox/Iceweasel and Chrome (Ctrl-Shift-C) have a console that displays compiler errors and warnings:

Another issue is that error messages can be cryptic and misleading. For example, the above error message “TypeError: searcher.bytag is not a function example1.html:73” is confusing for a starting programmer. Surely, the source code calls searcher.bytag() which definately is a function. So, why does the compiler say it is not?? The bug here, of course, is that the function called in the source code is not found: it should be byTag().

But this bug at least can be detected during interpretation and executing of the code. That is, it is clear to the compiler that it doesn’t know how to handle the code. Another common problem is the situation where the code looks fine (to the compiler), but the data it handles makes the code break down. For example, an variable doesn’t have the expected value, leading to errors (e.g. null pointer-style). Therefore, understanding the variable values at a particular point in your code can be of great use.

Console output

A simple way to inspect the content of a variable is to use this console visible in the above screenshot. Many programming languages have their custom call to send output there. Java has the System.out.println() and JavaScript has console.log():

Thus, if you have some complex bit of code with multiple for-loops, if-else statements, etc, this can be used to see if some part of your code that you expect to be called really is:

console.log("He, I'm here!");

This can be very useful when using asynchronous web service calls! Similarly, see what the value of some variable is:

var label = jsonResponse.items[i].prefLabel;
console.log("label: " + label);

Also, because JavaScript is not a strongly typed programming I frequently find myself inspecting the data type of a variable:

var label = jsonResponse.items[i].prefLabel;

console.log("typeof label: " + typeof(label));

Conclusion

These tools are very useful to find the location of a bug. And this matters. Yesterday, I was trying to use the histogram code in example6.html to visualize a set of values with negative numbers (zeta potentials of nanomaterials, to be precise) and I was debugging the issue, trying to find where my code when wrong. I used the above approaches, and the array of values looked in order, but different from the original example. But still the histogram was not showing up. Well, after hours, and having asked someone else to look at the code too, and having ruled out many alternatives, she pointed out that the problem was not in the JavaScript part of the code, but in the HTML: I was mixing up how default JavaScript and the d3.js library add SVG content to the HTML data model. That is, I was using <div id="chart">, which works with document.getElementById("chart").innerHTML, but needed to use <div class="chart"> with the d3.select(".chart").innerHTML code I was using later.

OK, that bug was on my account. However, it still was not working: I did see a histogram, but it didn’t look good. Again debugging, and after again much too long, I found out that this was a bug in the d3.js code that makes it impossible to use their histogram example code for negative values. Again, once I knew where the bug was, I could Google and quickly found the solution for it on StackOverflow.

So, the workflow of debugging at a top level, looks like:

1. find where the problem is
2. try to solve the problem

Happy debugging!

The Open PHACTS API support various formats and this JSON is the default format used by the ops.js library. However, the amount of information returned by the Open PHACTS cache is complex, and generally includes more than you want to use in the next step. Therefore, it is needed to extract data from the JSON document, which was not covered in the post #10 or #11.

Let’s start with the example JSON given in that post, and let’s consider this is the value of a variable with the name jsonData:

{
    "id": 1,
    "name": "Foo",
    "price": 123,
    "tags": [ "Bar", "Eek" ],
    "stock": {
        "warehouse": 300,
        "retail": 20
    }
}

We can see that this JSON value starts with a map-like structure. We can also see that there is a list embedded, and another map. I guess that one of the reasons why JSON has taken such a flight is how well it integrates with the JavaScript language: selecting content can be done in terms of core language features, different from, for example, XPath statements needed for XML or SPARQL for RDF content. This is because the notation just follows core data types of JavaScript and data is stored as native data types and objects.

For example, to get the price value from the above JSON code, we use:

var price = jsonData.price;

Or, if we want to get the first value in the Bar-Eek list, we use:

var tag = jsonData.tags[0];

Or, if we want to inspect the warehouse stock:

var inStock = jsonData.stock.warehouse;

Now, the JSON returned by the Open PHACTS API has a lot more information. This is why the online, interactive documentation is so helpful: it shows the JSON. In fact, given that JSON is so much used, there are many tools online that help you, such as jsoneditoronline.org (yes, it will show error messages if the syntax is wrong):

BTW, I also recommend installing a JSON viewer extension for Chrome or for Firefox. Once you have installed this plugin, you can not just read the JSON on Open PHACTS’ interactive documentation page, but also open the Request URL into a separate browser window. Just copy/paste the URL from this output:

And with a JSON viewing extension, opening this https://beta.openphacts.org/1.3/pathways/… URL in your browser window will look something like:

And because these extensions typically use syntax highlighting, it is easier to understand how to access information from within your JavaScript code. For example, if we want the number of pathways in which the compound testosterone (the link is the ConceptWiki URL in the above example) is found, we can use this code:

var pathwayCount = jsonData.result.primaryTopic.pathway_count;

Open PHACTS has come up with 20 questions (doi:10.1016/j.drudis.2013.05.008; Open Access):

1. Give me all oxidoreductase inhibitors active <100 nM in human and mouse
2. Given compound X, what is its predicted secondary pharmacology? What are the on- and off-target safety concerns for a compound? What is the evidence and how reliable is that evidence (journal impact factor, KOL) for findings associated with a compound?
3. Given a target, find me all actives against that target. Find/predict polypharmacology of actives. Determine ADMET profile of actives
4. For a given interaction profile – give me similar compounds
5. The current Factor Xa lead series is characterized by substructure X. Retrieve all bioactivity data in serine protease assays for molecules that contain substructure X
6. A project is considering protein kinase C alpha (PRKCA) as a target. What are all the compounds known to modulate the target directly? What are the compounds that could modulate the target directly? I.e. return all compounds active in assays where the resolution is at least at the level of the target family (i.e. PKC) from structured assay databases and the literature
7. Give me all active compounds on a given target with the relevant assay data
8. Identify all known protein–protein interaction inhibitors
9. For a given compound, give me the interaction profile with targets
10. For a given compound, summarize all ‘similar compounds’ and their activities
11. Retrieve all experimental and clinical data for a given list of compounds defined by their chemical structure (with options to match stereochemistry or not)
12. For my given compound, which targets have been patented in the context of Alzheimer’s disease?
13. Which ligands have been described for a particular target associated with transthyretin-related amyloidosis, what is their affinity for that target and how far are they advanced into preclinical/clinical phases, with links to publications/patents describing these interactions?
14. Target druggability: compounds directed against target X have been tested in which indications? Which new targets have appeared recently in the patent literature for a disease? Has the target been screened against in AZ before? What information on in vitro or in vivo screens has already been performed on a compound?
15. Which chemical series have been shown to be active against target X? Which new targets have been associated with disease Y? Which companies are working on target X or disease Y?
16. Which compounds are known to be activators of targets that relate to Parkinson’s disease or Alzheimer’s disease
17. For my specific target, which active compounds have been reported in the literature? What is also known about upstream and downstream targets?
18. Compounds that agonize targets in pathway X assayed in only functional assays with a potency <1 μM
19. Give me the compound(s) that hit most specifically the multiple targets in a given pathway (disease)
20. For a given disease/indication, give me all targets in the pathway and all active compounds hitting them

Students in the Programming in the Life Sciences course will this year pick one of these questions as a starting point in the project. The goal is to develop a HTML+JavaScript solution that will answer the question the selected. There is freedom to tweak the question to personal interests, of course. By selecting a simpler pharmacological question that last year, more time and effort can be put into visualization and interpretation of the found data.

The first app is by Tim and Taís, and look up activities from the Open PHACTS platform and filters it for activities related to a set of five anti-oxidants (see also their FigShare):

The next app is by Janneke and Lukas and uses the Open PHACTS API to report on single protein targets for the compound the user enters (see also their SlideShare):

More apps will follow soon.

HTML has many dialects, and HTML5 is the upcoming next version. The features have become so extensive that we will not have capture half of them; instead, we will stick to the bare minimum needed. But even at an minimum, writing a web page with HTML code is basically writing source code. The compiled version is the view of the webpage your web browser shows you. One important difference is that HTML is much more like a data model representation than it is like computational instructions. That is, rather than saying things like put("String", xCoord, yCoord), we define what is to be shown in in what order with general instructions. Well, in pure HTML that is. Cascading Style Sheets (CSS) is quite outside the scope of this course.

A minimal HTML page looks like:

<html>
  <head>
  </head>
  <body>
  Hello world!
  </body>
</html>

When we think about this structure, we notice that it is not unlike the key-value maps we covered earlier. For example, compare it to this JSON:

{
  "html": {
    "head":{},
    "body":{value:"Hello world!"}
  }
}

Even if we introduce HTML attributes:

<html>
  <head>
  </head>
  <body>
  <h1><a name="hello">Hello world!</a></h1>
  </body>
</html>

The JSON equivalent would be:

{
  "html": {
    "head":{},
    "body":{
      "h1":{
        "a":{
          attributes:{"name":"hello"},
          value:"Hello world!"
        }
      }
    }
  }
}

So, while these are quite different languages than programming languages, we can clearly see they have been made up by the same (computer science) people. But in my opinion, this is an advantage: because we only need to learn the underlying patterns and can then much more easily switch between different language.

Now, returning to the HTML example, we introduce a bit of terminology. Let’s start with the last example:

<h1><a name="hello">Hello world!</a></h1>

This HTML code example shows the <h1> element which has one child element <a>. This child element has an attribute @name. Elements can contain string content, such as the <a> element has, and one or more child elements (and any combination of that). Attributes can only have string content. The HTML specification defines in detail which elements can be child elements of other elements. For example, the <head> element can only be a child element of <html>. Similarly, each HTML element can only have specific attributes, though some attributes can be attached to any element.

There is plenty of documentation on the web, but there are also tools that can help us write HTML. For example, the http://validator.w3.org/. This website detects errors in your HTML code, and is quite helpful if you are new to editing HTML, as well as useful if you have a lot of HTML experience.

HTML elements you may find useful include the following:

• <h1>, <h2>, …, <h5>: these are header and can be used to make sections
• <p>: indicates a paragraph
• <div id="someID">: indicates a section of text. The content of any element with an id attribute can be replaced by any appropriate HTML content with JavaScript
• <a href="http://...">some link</a>: this is used to make hyperlinks, href means hyperlink reference
• <a name="mark1">some text</a>: this is used to create bookmarks. with <a href="#mark1">jump to section Mark 1</a>
• <script>: used to include JavaScript code in your HTML page
• <head>: this HTML blob contains metadata, a list of libraries to be loaded, but also JavaScript which is executed before the HTML <body> is processed
• <body>: this contains the HTML that is depicted in your browser window

Keep the HTML simple; the programming is more important.

Exercise: below is part of the HTML/JavaScript source code behind this app. Please indicate which lines are HTML source code, and what is JavaScript.

<!DOCTYPE HTML PUBLIC
  "-//W3C//DTD HTML 4.01 Transitional//EN"
  "http://www.w3.org/TR/html4/loose.dtd">
<html>
<!--

Copyright (c) 2013  Egon Willighagen <egon.willighagen@maastrichtuniversity.nl>

 Permission is hereby granted, free of charge, to any person
 obtaining a copy of this software and associated documentation
 files (the "Software"), to deal in the Software without
 restriction, including without limitation the rights to use,
 copy, modify, merge, publish, distribute, sublicense, and/or sell
 copies of the Software, and to permit persons to whom the
 Software is furnished to do so, subject to the following
 conditions:

 The above copyright notice and this permission notice shall be
 included in all copies or substantial portions of the Software.

 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
 EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
 OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
 NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
 HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
 WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
 FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
 OTHER DEALINGS IN THE SOFTWARE.

-->
<head>
  <title>OpenPHACTS Jasmine Spec Runner</title>
  <script src="lib/jquery-1.9.1.min.js"></script>
  <script type="text/javascript" src="lib/purl.js"></script>

  <!-- include source files here... -->
  <script type="text/javascript" src="src/OPS.js"></script>
  <script type="text/javascript" src="src/ConceptWikiSearch.js"></script>

  <!-- setup -->
  <script type="text/javascript">
  // get the app_key and app_id from the webpage call -->
var prmstr = window.location.search.substr(1);
var prmarr = prmstr.split ("&");
var params = {};
for ( var i = 0; i < prmarr.length; i++) {
    var tmparr = prmarr[i].split("=");
    params[tmparr[0]] = tmparr[1];
}
  </script>
</head>

<body>
  <h3>Output</h3>
  <h3>Search Results</h3>
  <p><div id="table"></div></p>
  <h3>Compound Details</h3>
  <p><div id="details"></div></p>
  <h3>JSON reply</h3>
  <p><div id="json">Nothing yet</div></p>
  <script type="text/javascript">
var searcher = new Openphacts.ConceptWikiSearch(
  "https://beta.openphacts.org",
  params["app_id"], params["app_key"]
);
var callback = function(success, status, response){
  document.getElementById("json").innerHTML = JSON.stringify(response);
  html = "<table>";
  for (var i=0; i<response.length; i++) {
    html += "<tr>";
    html += "<td>";
    html += "Name: <span>" +
      response[i].prefLabel +
      "</span>";
    html += "</td>";
    html += "</tr>";
  }
  html += "</table>";
  document.getElementById("table").innerHTML = html;
};
searcher.byTag(
  'Aspirin', '5', '4',
  '07a84994-e464-4bbf-812a-a4b96fa3d197',
  callback
);
  </script>
</body>
</html>

var jsonData = JSON.parse(jsonString);

Now, we previously covered maps. Maps have keys and values: the keys unlock a particular value. For example, take this JavaScript:

var map = { "key": "value", "key2": "value2" };

We define here a key-value object, and we can access the two values with the two keys:

map["key2"]; // == value2

These examples are JavaScript source code. Not a string. The content of the map variable is a data structure. But when we communicate with a web service, we need a (string) serialization of the data model, because we cannot send around memory pointers (which a variable is) because they are only valid on a single machine.

This is where the JSON format comes in. We can convert the content of the above map variable into a string representation with this code:

var mapStringified = JSON.stringify(map);

which gives us the following output:

{"key":"value","key2":"value2"}

This string looks an awful lot like the JavaScript code we wrote earlier.

And, likewise we can convert the JSON string back into a JavaScript data model again, with:

var mapAgain = JSON.parse(mapStringified);

Now, I did warn you earlier that values can be lists and maps itself again, so consider this JSON example from Wikipedia:

{
    "id": 1,
    "name": "Foo",
    "price": 123,
    "tags": [ "Bar", "Eek" ],
    "stock": {
        "warehouse": 300,
        "retail": 20
    }
}

Here we see that the value behind the stock key is another map, and the value behind the tags key is a list. This creates a quite flexible serialization format, which is happily used by Open PHACTS. (And for the semantic web readers, yes, we can make JSON more semantic. The Open PHACTS LDA supports a “rdfjson” format.)

Application Programming Interfaces

APIs define how programs can be used by other programs. An API defines how methods are called and what feedback you can expect. It basically is the combination of documentation and the program itself. But, unlike any piece of software, an API is aimed at users, rather than use in the same program. The API is how you communicate between programs.

Now, in this course we will see two key types of APIs. The first are the APIs provided by the libraries that we use. For example, we already indicated that we will be using at least the following two libraries, ops.js and d3.js. These libraries are a collection of functional bits (e.g. classes and methods). For example, ops.js defines an API which wraps closely the Open PHACTS Linked Data API (LDA) itself. The API requires as to do a few things: 1. create a wrapper for the LDA; 2. define a call back function; 3. invoke the actual

call.var searcher = new Openphacts.ConceptWikiSearch(
  "https://beta.openphacts.org", appID, appKey
);  
var callback=function(success, status, response){  
    searcher.parseResponse(response);
};  
searcher.findCompounds('Aspirin', '20', '4', callback);

Web Services

Web services are a special kind of APIs: they expose an API over the web. That imposes some features of these APIs: first, they are based on a web transport layer, commonly HTTP, but XMPP is possible too. HTTP is used by your web browser too. Secondly, the web server needs a common communication language to serialize the method call. Here, two key standards are used, XML and JSON. We will cover these in more detail later. For now, it suffices to think of these as envelopes in which are message is sent. Now, another aspect standardized is how to call the web services. For that, SOAP and REST are the most important standards for the life sciences (though I still think Wagener’s XMPP approach is still worthwhile checking out!). SOAP and REST use XML and JSON are underlying serialization format.

So, web services are theoretically complex. For this course, most of it is hidden by the client library that will take care of the HTTP and SOAP/REST layers. The students who wish to use Java instead of JavaScript, will face the problem that you first need to find a Java client library for the LDA. There is this library, but that needs exploring for use with the latest Open PHACTS LDA. Higher stakes, higher rewards.

Take home message

Practically, you do not need to know much of the technologies behind web services, just like you do not need to know machine instructions CPUs follow to run your program. But, it is important to have seen these terms. You will run into them, and need enough context to know where and how to find answers to the questions that you will have.

There is one exception: JavaScript Object Notation, JSON. That is the format in which the data is returned by the service, and you will have to handle that. JSON will be the topic of the next post.