Office of Portfolio Analysis Artificial intelligence/machine learning at the National Institutes of Health (AI/ML at the NIH) Predicting translational progress in biomedical research George Santangelo, Ph.D. Ian Hutchins, Ph.D. Office of Portfolio Analysis, NIH
26
Embed
Artificial intelligence/machine learning at the National ... · ̶Hutchins BI et al. PLoS Biology 2016 14:e1002541 ̶Hutchins BI et al. PLoS Biology 2017 15:e2003552 ̶Santangelo
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Office of Portfolio Analysis
Artificial intelligence/machine learningat the National Institutes of Health
(AI/ML at the NIH)
Predicting translational progress in biomedical research
George Santangelo, Ph.D.Ian Hutchins, Ph.D.
Office of Portfolio Analysis, NIH
Mission of the NIH Office of Portfolio Analysis (OPA)
Consult &Collaborate
Develop New Analytical Methods
BuildTools
Data Cleaning & Analysis
Supportdata-driven
decision-making
Disseminate Best PracticesClassroom Training (Custom Classes Available)
Online TrainingWeb Resources (Case Studies, FAQs)
Office HoursSymposia
OPA website: https://dpcpsi.nih.gov/opa/index
Support data-driven decision-making
• Enable NIH research administrators and decision-makers to evaluate and prioritize current and emerging areas of research that will advance scientific knowledge and improve human health
•Help ensure that the NIH research portfolio ― is balanced― is free of unnecessary duplication― takes advantage of collaborative,
cross-cutting research― stimulates the emergence of
*a publication that describes a clinical trial or clinical guideline
Office of Portfolio Analysis
Non-uniform probability of being cited by a clinical article
Low
Mid
High
Office of Portfolio Analysis
Training a machine learning system to predict future translation
Office of Portfolio Analysis
Training a machine learning system to predict future translation
*
*
*Random Forests outperformed Support Vector Machines, Neural Networks, logistic regression, et al.
Office of Portfolio Analysis
Validation of machine learning predictions of future translation
Accurate predictions can be made two years after publication
Office of Portfolio Analysis
Validation of machine learning predictions of future translation
Machine learning outperforms experts rating the clinical impact of publications
Office of Portfolio Analysis
Validation of machine learning predictions of future translation
Increase in APT score is responding to new information in the citing networks
Approximate Potential to Translate (APT) scores are stable but can change over time
Office of Portfolio Analysis
“Genetic” analysis of machine learning predictions of future translation
Office of Portfolio Analysis
Summary
• The Office of Portfolio Analysis at NIH develops tools and new methods, including AI/ML approaches, to improve data-driven decision-making
• Predicting translational progress in biomedical research has the potential to accelerate scientific advances that improve human health
• We built and validated an ML system that accurately determines, within two years post-publication, the likelihood any/all paper(s) will be cited by a clinical article (a paper that describes a trial or guideline)
• Making accurate predictions requires both the features of the paper in question and the features of the papers that cite it
• Analyzing citation patterns indicates that knowledge flows through different domains before percolating into the clinical arena
Office of Portfolio Analysis
Temporal dynamics of translation
Office of Portfolio Analysis
Using fractional in place of binary counting of MeSH terms
Office of Portfolio Analysis
Temporal dynamics of translationLow, mid, or high fraction of Human MeSH terms