Concept recognition and its application for protein function prediction Christopher Funk, Ph.D Candidate University of Colorado School of Medicine 3/18/2015 Ph.D Committee: Dr. Larry Hunter Dr. Kevin Cohen Dr. Karin Verspoor Dr. Asa Ben-Hur Dr. Joan Hooper 0
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Concept recognition and its application for protein function
predictionChristopher Funk, Ph.D Candidate
University of Colorado School of Medicine
3/18/2015
Ph.D Committee:
Dr. Larry Hunter
Dr. Kevin Cohen
Dr. Karin Verspoor
Dr. Asa Ben-Hur
Dr. Joan Hooper 0
Growth in PubMed
0
200000
400000
600000
800000
1000000
12000001
91
4
19
18
19
22
19
26
19
30
19
34
19
38
19
42
19
46
19
50
19
54
19
58
19
62
19
66
19
70
19
74
19
78
19
82
19
86
19
90
19
94
19
98
20
02
20
06
20
10
20
14
Pu
blic
atio
ns
pe
r ye
ar
Year
1
Biomedical Knowledge Lifecycle
MedlinePMC
GenBankPfam
GEO
FlyBase
UniProt/SwissProt
Experimental Data
Literature
BiomedicalDatabases
2
Biomedical Knowledge Lifecycle
MedlinePMC
GenBankPfam
GEO
FlyBase
UniProt/SwissProt
Experimental Data
Literature
BiomedicalDatabases
3
Biomedical Knowledge Lifecycle
MedlinePMC
GenBankPfam
GEO
FlyBase
UniProt/SwissProt
Experimental Data
Literature
BiomedicalDatabases
4
Biomedical Knowledge Lifecycle
MedlinePMC
GenBankPfam
GEO
FlyBase
UniProt/SwissProt
Experimental Data
Literature
BiomedicalDatabases
5
Manual Curation is a Bottleneck
MedlinePMC
GenBankPfam
GEO
FlyBase
UniProt/SwissProt
Experimental Data
Literature Baumgartner et al. 2007
BiomedicalDatabases
6
My dissertation
MedlinePMC
GenBankPfam
GEO
FlyBase
UniProt/SwissProt
Experimental Data
Literature
BiomedicalDatabases
7
Natural Language
Processing Pipeline
My dissertation
MedlinePMC
GenBankPfam
GEO
FlyBase
UniProt/SwissProt
Experimental Data
Literature
BiomedicalDatabases
Natural Language
Processing Pipeline
8
My dissertation
MedlinePMC
GenBankPfam
GEO
FlyBase
UniProt/SwissProt
Experimental Data
Literature
BiomedicalDatabases
Text-mined data
Data
Pre
dic
tio
ns
/ H
ypo
thes
is
Machine Learning
9
My dissertation
MedlinePMC
GenBankPfam
GEO
FlyBase
UniProt/SwissProt
Experimental Data
Literature
BiomedicalDatabases
Text-mined dataData
Pre
dic
tio
ns
/ H
ypo
thes
is
Machine Learning
10
My dissertation
MedlinePMC
GenBankPfam
GEO
FlyBase
UniProt/SwissProt
Experimental Data
Literature
BiomedicalDatabases
Text-mined dataData
Pre
dic
tio
ns
/ H
ypo
thes
is
Machine Learning
Validation
11
Biomedical ontologies
• Great enabling technology of bioinformatics
• Contain concepts linked with hierarchical relationships
• Over 400 different ontologies in NCBO BioPortal
12
Concept/Term
Gene Ontology
• Represents standardized way to refer to functions
• Input: Any text and OBO file.• Output: List of concepts from ontology contained
within the text in multiple output files (xml, a1, inline)
28
Recognition of Gene
Ontology terms is
poor
Funk et al. 2014 29
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Performance of GO on CRAFT
Precision
Recall
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
f=0.1
f=0.2
f=0.3
f=0.4
f=0.5
f=0.6
f=0.7
f=0.8
f=0.9
Systems
MetaMap
Concept Mapper
NCBO Annotator
Ontologies
CC
MF
BP
30
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Performance of GO on CRAFT
Precision
Recall
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
f=0.1
f=0.2
f=0.3
f=0.4
f=0.5
f=0.6
f=0.7
f=0.8
f=0.9
Systems
MetaMap
Concept Mapper
NCBO Annotator
Ontologies
CC
MF
BP
Neji (Campos et al 2013)
Whatizit (Rebholz-Shuhmann et al 2008)
Case sensitivity and information gain (Groza et al
accepted)
Concept variation within textGO:0006900 – membrane budding
Variation in PMID: 12925238
• Lipid rafts play a key role in membrane budding…
• …involvement of annexin A7 in budding of vesicles…
• …Ca2+-mediated vesiculationprocess was not impared.
• Red blood cells which lack the ability to vesiculate casuse…
• Having excluded a direct role in vesicle formation…
31
Gene Ontology vs. natural languageGO:0006900 – membrane budding
[Term]
id: GO:0006900
name: membrane budding
…
def: "The evagination of a membrane, resulting in formation of a vesicle.”
…
synonym: "membrane evagination”
synonym: "nonselective vesicle assembly”
synonym: "vesicle biosynthesis”
synonym: "vesicle formation”
…
Variation in PMID: 12925238
• Lipid rafts play a key role in membrane budding…
• …involvement of annexin A7 in budding of vesicles…
• …Ca2+-mediated vesiculationprocess was not impared.
• Red blood cells which lack the ability to vesiculate casuse…
• Having excluded a direct role in vesicle formation…
32
Gene Ontology vs. natural languageGO:0006900 – membrane budding
[Term]
id: GO:0006900
name: membrane budding
…
def: "The evagination of a membrane, resulting in formation of a vesicle.”
…
synonym: "membrane evagination”
synonym: "nonselective vesicle assembly”
synonym: "vesicle biosynthesis”
synonym: "vesicle formation”
…
Variation in PMID: 12925238
• Lipid rafts play a key role in membrane budding…
• …involvement of annexin A7 in budding of vesicles…
• …Ca2+-mediated vesiculationprocess was not impared.
• Red blood cells which lack the ability to vesiculate casuse…
• Having excluded a direct role in vesicle formation…
33
Related work
• TermGenie allows for on-the-fly creation of concepts and has modules for automatic synonym generation for a few classes of concepts (Dietze et al 2014)
• Hamon et al 2008 automatically generate synonym sets from Gene Ontology concepts
Different types of regulationRegulation of biological processNegative regulation of BPPositive regulation of BP
T-cell differentiationT-cell proliferationT-cell activationRegulation of cell differentiationRegulation of cell proliferationPositive regulation of cell differentiationPositive regulation of cell proliferationRegulation of T-cell differentiationRegulation of T-cell differentiationPositive regulation of T-cell differentiationPositive regulation of T-cell proliferation…
Different types of regulationRegulation of biological processNegative regulation of BPPositive regulation of BP
T-cell differentiationT-cell proliferationT-cell activationRegulation of cell differentiationRegulation of cell proliferationPositive regulation of cell differentiationPositive regulation of cell proliferationRegulation of T-cell differentiationRegulation of T-cell differentiationPositive regulation of T-cell differentiationPositive regulation of T-cell proliferation…
Different types of regulationRegulation of biological processNegative regulation of BPPositive regulation of BP
T-cell differentiationT-cell proliferationT-cell activationRegulation of cell differentiationRegulation of cell proliferationPositive regulation of cell differentiationPositive regulation of cell proliferationRegulation of T-cell differentiationRegulation of T-cell differentiationPositive regulation of T-cell differentiationPositive regulation of T-cell proliferation
GO concepts are built compositionally
39
T-cell differentiationT-cell proliferationT-cell activationRegulation of cell differentiationRegulation of cell proliferationPositive regulation of cell differentiationPositive regulation of cell proliferationRegulation of T-cell differentiationRegulation of T-cell differentiationPositive regulation of T-cell differentiationPositive regulation of T-cell proliferation…
“positive regulation of”, Biological Process concept
40
Syntactic and derivational rules
• External ontological mappings (Cell type ontology for now)• Input from biologist and ontologist• Derivations from WordNet and Lexical Variant Generator• Manually analyzing of CRAFT annotations
– GO:0050729 positive regulation of inflammatory response • proinflammatory• pro-inflammatory
– GO:0045597 positive regulation of cell differentiation• differentiation-promoting
– GO:0043065 positive regulation of apoptosis • up-regulation of apoptosis• pro-apoptotic
– GO:0007131 meiotic recombination• recombination in meiosis
– GO:0040020 regulation of meiosis• meiotic regulatory
41
Example application of rules
42
Example application of rules
43
Example application of rules
44
Example application of rules
45
Generated synonyms:cyclic AMP biosynthesis (current synonym)adenosine 3’,5’-cyclophosphate biosynthesis (current synonym)formation of cAMPcAMP productiongeneration of cAMP…
Example application of rules
46
Generated synonyms:activation of cyclic AMP biosynthesisadenosine 3’,5’-cyclophosphate biosynthesis enhancementformation of cAMP activationstimulation of cAMP productionStimulation of generation of cAMP…
Synonyms appear in the literature
• A drug-like antagonist inhibits thyrotropinreceptor-mediated stimulation of cAMP production in Graves' orbital fibroblasts.
– PMCID: 3407388
• These data suggest that ethanol treatment increases in vitro hCG production in human placental trophoblasts by enhancing cAMP production.
– PMID: 9413929
47
18 rules generated 291k synonyms for 16k GO concepts (66%).
Increase in F-measure for all GO of 0.14.
Overall performance 0.64.
48
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Precision
Recall
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
f=0.1
f=0.2
f=0.3
f=0.4
f=0.5
f=0.6
f=0.7
f=0.8
f=0.9
●
Ontologies
CC
MF
BP
GO
Rules improve performance on the CRAFT corpus
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Precision
Recall
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
f=0.1
f=0.2
f=0.3
f=0.4
f=0.5
f=0.6
f=0.7
f=0.8
f=0.9
●
Ontologies
CC
MF
BP
GO
49
Neji (Campos et al 2013)
Whatizit (Rebholz-Shuhmann et al 2008)
Case sensitivity and information gain (Groza et al
accepted)
Compositional rules show higher performance than any reported numbers.
1 million full text corpus – rules produced 42% more annotations and
• Experimentally determining function is time consuming and expensive
• Task: Given a protein, what are the functions it performs?
• Function is everything that happens to or through a protein (Rost et al. 2003)
• Specify function by the Gene Ontology (GO)
57
Commonly used methods/features
• Transfer of function based on homology (Bork et al 1998, Rost et al 2003, Xin et al 2013)
• Amino acid sequence (Jensen et al 2003, Martin et
al 2004, Clark et al 2011)
• 3D structure (Pal et al 2005, Laskowski et al 2005)
• Co-localization (Walker et al 1999, Klomp et al 2012)
• Protein interaction networks (Deng et al
2003, Nabieva et al 2005)
• Microarray experiments (Huttenhower et al
2006, Sokolov et al 2013)
• Combinations of all above (Costello et al
2009, Sokolov et al 2010)
58
Literature based function prediction
• Which literature features?• How to combine them?
• BioCreative IV (2014) - <protein,document> -> <protein,document,GOterm>
– K-nearest neighbor based on similar abstracts was best performing (F-measure 0.13) (Gobeill et al)
• Exploit document similarity to establish relationships between genes (Shakay et al 2000, Raychaudhuri et al 2002, Chaussabel et al 2002, Gobeill et al 2014)
• Protein-protein co-occurrence supplements PPI (Gabow et al 2008)
• Critical Assessment of Functional Annotation (2011)– Wong and Shatkay 2014 train and test a classifier and characterize
proteins using key-terms from related abstracts. Only predict concepts at 2nd level of GO.
– Bjorne et al 2011 utilize biomedical events on their ability to predict 385 GO concepts with F-measure of 0.09.
59
Literature based function prediction
• Which literature features?• How to combine them?
• Exploit document similarity to establish relationships between genes (Shakay et al 2000, Raychaudhuri et al 2002, Chaussabel et al 2002, Gobeill et al 2014)
• BioCreative IV (2014) - <protein,document> -> <protein,document,GOterm>
– K-nearest neighbor based on similar abstracts was best performing (F-measure 0.13) (Gobeill et al 2014)
• Protein-protein co-occurrence supplements PPI (Gabow et al 2008)
• Critical Assessment of Functional Annotation (2011)– Wong and Shatkay 2014 train and test a classifier and characterize
proteins using key-terms from related abstracts. Only predict concepts at 2nd level of GO.
– Bjorne et al 2011 utilize biomedical events on their ability to predict 385 GO concepts with F-measure of 0.09.
60
Literature based function prediction
• Co-mentions – co-occurring entities within a specified span of text.
– Protein-Protein
– GO-GO
– Protein-GO
• Sentence
• Non-sentence
• Bag of words (BoW)
– All words in sentence where protein is mentioned
61
Literature based function prediction
• Co-mentions – co-occurring entities within a specified span of text.
– Protein-Protein
– GO-GO
– Protein-GO
• Intra-sentence
• Inter-sentence
• Bag of words (BoW)
– All words in sentence where protein is mentioned
Protein GO term co-mentions:intra_comen(P50281, GO:0008237), intra_comen(P50281, GO:0006508), intra_comen(P50281, GO:0009056), intra_comen(P50281, GO:0031012),inter_comen(P50281, GO:0010467), inter_comen(P50281, GO:0005623)
Protein GO term co-mentions:inter_comen(P50281, GO:0008237), inter_comen(P50281, GO:0006508), inter_comen(P50281, GO:0009056), inter_comen(P50281, GO:0031012),inter_comen(P50281, GO:0010467), inter_comen(P50281, GO:0005623)
Bag of Words:P40281, known=2, membrane=1, protein=1, proteolytic=1, enzyme=1, …Protein GO term co-mentions (intra-sentence):P40281, GO:0008237=1, GO:0006508=1, GO:0009056=1, GO:0031012=1,…Protein GO term co-mentions (inter-sentence):P40281, GO:0010467=2, GO:0005623=2,…
68
Evaluate literature features within GOstruct framework (Sokolov et al 2010)
• Multi-view hierarchical support vector machine (SVM) framework designed to predict entire Gene Ontology at once
• Combine many different types of features for prediction
• One of top performing systems in CAFA 2011
(Adapted fromSokolov et al 2013)
69
Extraction & Analysis pipeline
70
Only using literature is useful for function prediction
71
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
MF BP CC
Mac
ro-a
vera
ged
F-m
eas
ure
Gene Ontology Branch
Baseline (co-mentions as predictions) Co-mentions BoW Co-mentions + BoW
Literature features approach performance of commonly used biological features
(Sokolov et al 2013, Kahanda et al unpublished)
720
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
MF BP CC
Mac
ro-a
vera
ged
F-m
eas
ure
Trans/Localization
Homology
Network
Literature
All Combined
Top predicted GO concepts using synonym rules have higher information content
– “Genes related to carbohydrate metabolisminclude PPP1R3C, B3GNT1, and GCNT1…” -PMID:23646466
• CERS2 – ceramide biosynthetic process (Q96G23 - GO:0046513)
– “…CersS2, which uses C22-CoA for ceramidesynthesis…” -PMID:22144673
74
Future directions
• Explore interaction of dictionary and machine learning based methods for concept recognition
• Extend and refine GO synonym generation rules
• Use already created gold standard of functionally annotated co-mentions to reduce the high false positive rate (70%)
• Provide “noisy” large collection of extracted co-mentions to biologists to explore interactively
75
Contributions
• Performed a comprehensive evaluation of prominent general concept recognition systems for eight biomedical ontologies against a gold standard corpus.
• Created more variable set of Gene Ontology synonyms utilizing concept compositionality and used them to improve recognition of GO concepts within the literature.
• Showed the utility of literature mined features, including mined concepts, for automated protein function prediction and validation.
76
Publications
First author• Christopher Funk, K Bretonnel Cohen, Lawrence Hunter, Karin Verspoor “Simple Gene ontology synonym
generation rules lead to increase in biomedical concept recognition” (alsmost submitted 2015)
• Christopher Funk, Indika Kahanda, Asa Ben-Hur, and Karin Verspoor (2014) “Evaluating a variety of text-mined features for automatic protein function prediction” Journal of Biomedical Semantics (accepted).
• Christopher Funk, Indika Kahanda, Asa Ben-Hur, and Karin Verspoor (2014) “Evaluating a variety of text-mined features for automatic protein function prediction” BioOntologies Special Interest Group ISMB 2014.
• Christopher Funk, Lawrence E. Hunter, and K. Bretonnel Cohen “Combining heterogeneous data for prediction of disease related and pharmacogenes” Pacific Symposium of Biocomputing 2014.
• Christopher Funk, William Baumgartner Jr., Benjamin Garcia, Christophe Roeder, Michael Bada, K. Bretonnel Cohen, Lawrence E. Hunter, and Karin Verspoor “Large-scale biomedical concept recognition: An evaluation of current automatic annotators and their parameters” BMC Bioinformatics 2014.
Co-author• Artem Sokolov, Christopher Funk, Kiley Graim, Karin Verspoor, Asa Ben-Hur “Combining Heterogeneous
Data Sources for Protein Function Prediction” BMC Bioinformatics 2013.
• Radivojac, Predrag and Clark, Wyatt T and Oron, Tal Ronnen and Schnoes, Alexandra M and Wittkop, Tobias and Sokolov, Artem and Graim, Kiley and Funk, Christopher and Verspoor, Karin and Ben-Hur, Asaand others “A large-scale evaluation of computational protein function prediction” Nature Methods 2013.
• Karin Verspoor, K. Bretonnel Cohen, Arrick Lanfranchi, Colin Warner, Helen L. Johnson, Christophe Roeder, Jinho D. Choi, Christopher Funk, Yuriy Malenkiy, Miriam Eckert, Nianwen Xue, William A. Baumgartner Jr., Michael Bada, Martha Palmer, and Lawrence E. Hunter “A corpus of full-text journal articles is a robust evaluation tool for revealing differences in performance of biomedical natural language processing tools” BMC Bioinformatics 2012.
• K. Bretonnel Cohen, Karin Verspoor, Michael Bada, Christopher Funk, and Lawrence E. Hunter (accepted) “The Colorado Richly Annotated Full Text (CRAFT) corpus: Multi-model annotation in the biomedical domain.” In Nancy Ide and James Pustejovsky, editors, Handbook of Linguistic Annotation.
77
Acknowledgements
• All paper co-authors– William Baumgartner Jr.– Benjamin Garcia– Christophe Roeder– Michael Bada– Kevin Cohen– Lawrence E. Hunter– Karin Verspoor
• CPBS program– David Knox– Mike Hinterberg– Mike Bada– Meg Pirrung– Charlotte Siska– Natalya Panteleyva– Negacy Hailu
• Committee– Larry Hunter– Karin Verspoor– Kevin Cohen– Joan Hooper– Asa Ben-Hur
References• Dogan, Rezarta and Lu, Zhiyong “An Inference Method for Disease Name Normalization” 2012• Blaschke, Christian et al “Evaluation of BioCreative assessment of task 2” 2005• Morgan, Alexander et al “Overview of BioCreative II gene normalization” 2008• Mao, Yuqing et al “Overview of the gene ontology task at BioCreative IV” 2014• Hirschman, Lynette et al “Overview of BioCreative task 1B: normalized gene list” 2005• Kang et al “Using rule-based natural language processing to improve disease normalization in biomedical text” 2013• Jimeno, Antonio et al “Assessment of disease named entity recognition on a corpus of annotated sentences” 2008• Shah, Nigam et al “Comparison of concept recognizers for building the Open Biomedical Annotator” 2009• Mungall et al “Obol: integrating language an meaning in bio-ontologies” 2004• Rost et al “Automatic prediction of protein function” 2003• Shore et al “Fibrodysplasia ossificans progressiva: a human genetic disorder of extraskeletal bone formation, or—how does one tissue become
another? “ 2012• Goichi et al “Cartilage dierentiation regulating gene” https://www.google.com/patents/WO2003087375A1?cl=en 2003• Van der Borght et al “Reduced neurogenesis in the rat hippocampus following high fructose consumption” 2011• Goncalves et al “The cox-2 inhibitors, meloxicam and nimesulide, suppress neurogenesis in the adult mouse brain” 2010• Bork et al “Predicting function: from genes to genomes and back” 1999• Xin et al “Computational methods for identification of functional residues within protein structures” 2003• Jensen et al “Prediction of human protein function from post-translational modification and localization features” 2003• Martin et al “Gotcha: a new method for prediction of protein function assessed by the annotation of seven genomes” 2004• Clark et al “Analysis of protein function and its prediction from amino acid sequence” 2011• Pal et al “Inference of protein function from protein structure” 2005• Lakowski et al “Protein function prediction using local 3D templates” 2005• Walker et al “Prediction of gene function by genome-scale expression analysis: prostate cancer associated genes” 1999• Klomp et al “Genome-wide matching of genes to cellular roles using guilt-by-association models derived from single sample analysis” 2012• Deng et al “Prediction of protein function from protein/protein interaction data: a probablistic approach” 2003• Nabieva et al “Whole-proteome prediction of protein function via graph-theorietic of interatcion maps” 2005• Huttenhower et al “A scalable method for integration and functional analysis of multiple microarray datasets” 2006• Sokolov et al ”Hierarichical classification of gene ontology terms using the Gostruct method” 2010• Sokolov et al “Combining heterogeneous data sources for accurate functional annotation of proteins” 2013• Costello et al “Gene networks in Drosophilia melenagaster: integrating experimental data to prediction protein function” 2009• Shatkay et al “Texts-as-data: using text-based features for proteins representation and for computational prediction of their characteristics” 2014• Bjorne et al “A machine learning model and evaluation of text mining protein function prediction” 2011• Shatkay et al “Finding themes in Medline documents: probabalistic similarity search” 2000• Chaussabel et al “Mining microarray expression data by literature profiling” 2002• Raychaudhuri et al “Associating Gene Ontology codes with genes using a maximum entropy analysis of biomedical literature” 2002• Gabow et al “Improving protein function prediction methods with integrationof literature data” 2008 79