{"id":65,"date":"2015-03-17T16:19:31","date_gmt":"2015-03-17T23:19:31","guid":{"rendered":"https:\/\/www.enlis.com\/blog\/?p=65"},"modified":"2015-03-17T16:25:45","modified_gmt":"2015-03-17T23:25:45","slug":"the-best-variant-prediction-method-that-no-one-is-using","status":"publish","type":"post","link":"https:\/\/www.enlis.com\/blog\/2015\/03\/17\/the-best-variant-prediction-method-that-no-one-is-using\/","title":{"rendered":"The Best Variant Prediction Method That No One Is Using"},"content":{"rendered":"

A test of the latest functional prediction algorithms:<\/em><\/span><\/p>\n

CADD, DANN, FATHMM<\/em><\/span><\/p>\n

 <\/p>\n

\n

Spoiler alert!\u00c2\u00a0 Summary for those short on time:<\/span><\/p>\n

Our conclusion:\u00c2\u00a0 Despite being released <\/span> to little fanfare <\/span>last October – we found that DANN offered the best sensitivity and specificity (highest true positives and lowest false positives).\u00c2\u00a0 DANN had less ‘noise’ compared to both CADD and FATHMM.\u00c2\u00a0 Our latest software includes DANN predictions and you can test it out on your own data!<\/span><\/span><\/p>\n

\n

 <\/p>\n

How do you decide which variations are important in a genome?<\/span><\/p>\n

When we lack good information on a variation, can we predict whether it is functional and important?\u00c2\u00a0 Why do we even want to predict?\u00c2\u00a0 Let’s get to the bottom of this.<\/span><\/p>\n

 <\/p>\n

The Problem<\/strong>:\u00c2\u00a0<\/span> <\/span><\/p>\n

The human genome is vast.\u00c2\u00a0 Billions of bases.\u00c2\u00a0 Really, really quite big.\u00c2\u00a0 And we don’t know much about it.<\/span><\/p>\n

\"stuffWeKnowAbout\"
\n

\nThe latest sequencing technology has made nearly all of those bases available to us.\u00c2\u00a0 In many cases, when we see a variation, we don’t have any information on its functional impact.\u00c2\u00a0 It could have no impact whatsoever (benign), or it could be the cause of a disease that we are studying (pathogenic).\u00c2\u00a0 If we could predict which ones were more likely to be functional, we can focus validation efforts on those variations first – potentially saving time and money.<\/span><\/p>\n

To this aim, researchers have developed algorithms to predict if a variation is functional, based on a number of different criteria.<\/span><\/p>\n

 <\/p>\n

 <\/p>\n

The algorithms:<\/strong><\/span><\/p>\n

For many years, the most commonly used prediction algorithms were SIFT and POLYPHEN2.\u00c2\u00a0 These are both limited to variations that change the amino acid sequence of a protein.\u00c2\u00a0 However, increasingly researchers want to look outside of protein-coding regions.\u00c2\u00a0 In addition, given all the new genome-wide data that has been released over the last few years, like ENCODE and new allele frequency data, the world was ripe for a new strategy.<\/span><\/p>\n

\u00c2\u00a0<\/span><\/p>\n

The CADD<\/a> algorithm was published in February 2014, and it appeared to be a significant improvement over existing methods.<\/span><\/p>\n

\"CADD

CADD Supplementary Data Figure 12 – A curve furthest to the top left shows improved sensitivity and specificity<\/p><\/div>\n

 <\/p>\n

CADD used machine learning to develop its algorithm.\u00c2\u00a0 In a nutshell, CADD developers gave the computer a set of training data so that it could have a list of functional vs. non-functional variations.\u00c2\u00a0 Then, they fed the machine 63 different annotations of that data, and let the computer figure out how to use those annotations to rank functional vs. non-functional variations.\u00c2\u00a0 Once they had an algorithm set, they ran it on every possible single nucleotide variation in the reference genome and gave each variation a score – higher the score, the higher the prediction that it is functional.<\/span><\/p>\n

\u00c2\u00a0<\/span><\/p>\n

More recently, two new algorithms were published – both of which claim to be improvements on CADD:<\/span><\/p>\n

\u00c2\u00a0<\/span><\/p>\n

DANN<\/a> – (October 2014)\u00c2\u00a0 DANN uses the exact same training and annotation data as CADD, but uses a different ‘nonlinear’ machine learning approach.<\/span><\/p>\n

FATHMM<\/a> – (January 2015)\u00c2\u00a0 FATHMM uses a similar machine learning approach to CADD, but uses a different set of training and annotation data.<\/span><\/p>\n

\u00c2\u00a0<\/span><\/p>\n

We decided to take a look at all three and compare their results.\u00c2\u00a0 Which one will prove to be the best?<\/span><\/p>\n

 <\/p>\n

 <\/p>\n

The test data set:<\/strong><\/span><\/p>\n

We need to compare a list of known functional (pathogenic) variations against a list of known non-functional (benign) variations.\u00c2\u00a0 Then, we can score all the variations with each algorithm and see how the lists compare.<\/span><\/p>\n

We chose a difficult data set, but one that we think accurately portrays the task that is faced by many researchers today.\u00c2\u00a0 The test data is based on ClinVar.<\/span><\/p>\n

ClinVar<\/a> has fast become an important resource for the analysis of sequencing data.\u00c2\u00a0 It stores information on pathogenic variations and how they are connected to human health.\u00c2\u00a0 In fact, all three of the papers describing the contenders used ClinVar to demonstrate their effectiveness.<\/span><\/p>\n

So what will we do differently?\u00c2\u00a0 The papers used pathogenic variations in ClinVar, and compared them to (assumed benign) variations that have a global allele frequency of >5%.\u00c2\u00a0 But ClinVar also records some variations that are known to be benign, and we think this is a more interesting set of non-functional variation for these reasons:<\/span><\/p>\n

\u00c2\u00a0<\/span><\/p>\n