Dr. Eliot Siegel: Watson and Deep QA Software in Pursuit of Personalized Medicine

1

CBIIT Speaker Series Watson and Deep Q/A Software In Pursuit of Personalized Medicine

Eliot Siegel, M.D., FACR, FSIIM Professor and Vice Chair University of Maryland Department of Diagnostic

Radiology Chief Imaging VA Maryland Healthcare System

2

caBIG Mission

• Widespread, sustainable availability of critical standards-based, interoperable academic/commercial biomedical capabilities

• Large and diverse cancer research data sets sustainably available for analysis, integration, and mining • Rather than 2% or 3% of patients’ data captured in clinical trials, capture

all patient data for decision and treatment support and data driven research

3

Clinical Scenario

4

Clinical Challenge VINCI – VA Informatics and Computing Infrastructure

5

Year of Artificial Intelligence in Medicine

• 2011 will likely be remembered as the year of the re-emergence of artificial intelligence in medicine with Watson and of course, Siri, arguably the best feature of the new iPhone 4S

• 2011 may well be the year that AI finally gets real traction in the medical informatics community and in medicine in general including the lay population

• Biggest contribution of Dr. Watson software in addition to Deep Q/A may be excitement to overcome inertia of the past

6

IBM and Jeopardy: A New Era?

• The Jeopardy match between the two best human players of all time and the IBM Deep Q/A software, “Watson” captured the spotlight and stimulated the imagination of the entire world

• The subsequent announcement of IBM’s involvement in the creation of “Dr. Watson” has created an incredible interest in the healthcare community about the potential breakthrough technology as well as the potential pitfalls of the use of “artificial intelligence” in medicine.

7

Dr. Watson Overview and History

• Initially had opportunity to visit IBM team about a year and a half ago • Engaged Jeopardy team and discussed the potential for medical

applications as next steps after Jeopardy Challenge • Began initial research with IBM approximately one year ago • Current grant with IBM for initial exploratory work with physician

helping team to understand the medical domain and challenges • Worked together on deeper understanding of the medical domain

using multiple resources

8

Introduction

• Deep Q/A is unique and exciting because it represents a fundamentally new approach that creates tools to rapidly mine a dynamic and non-predefined database

• Represents a potential fundamental change in opportunities for Artificial Intelligence applications in medicine

• But in some ways Watson is a “special needs” student • How does one train a system that is so remarkable at Jeopardy!

questions and apply to medicine?

9

• Watson can process 500 gigabytes, the equivalent of a million books, per second

• Hardware cost has been estimated at about $3 million • 80 TeraFLOPs , 49th in the Top 50 Supercomputers list • Content was stored in Watson's RAM for the game because data

stored on hard drives too slow to process

10

Deep Q/A

• Massively parallel, component based pipeline architecture • Uses extensible set of structured and unstructured content sources • Uses broad range of pluggable search and scoring components

11

Deep Q/A

• These allow integration of many different analytic techniques • Input from scorers is weighed and combined using machine learning

to generate a set of ranked candidate answers and associated confidence values

• Each answer is linked to its supporting evidence

12

Deep Q/A

• Does not map question to database of answers • Represents software architecture for analyzing natural language

content in both questions and knowledge sources • Discovers and evaluates potential answers and gathers and scores

evidence for those answers using unstructured sources such as natural language documents and structured sources such as relational and knowledge databases

13

Hardware

• Cluster of ninety IBM Power 750 servers (plus additional I/O, network and cluster controller nodes in 10 racks) with a total of 2880 POWER7 processor cores and 16 Terabytes of RAM

• Each Power 750 server uses a 3.5 GHz POWER7 eight core processor, with four threads per core

• The POWER7 processor's massively parallel processing capability is an ideal match for Watson's IBM DeepQA software which is embarrassingly parallel (that is a workload that is easily split up into multiple parallel tasks)

14

Software

• Watson's software was written in both Java and C++ and uses Apache Had0op framework for distributed computing

• Apache UIMA (Unstructured Information Management Architecture) framework

• IBM’s DeepQA software and SUSE Linux Enterprise Server 11 operating system

• “More than 100 different techniques are used to analyze natural language, identify sources, find and generate hypotheses, find and score evidence, and merge and rank hypotheses.”

15

High Level View of DeepQA Architecture

16

Deep QA Process

• Analyzes input question and generates many possible candidate answers through broad search of volumes of content

• Hypothesis is formed based on considerate of each candidate answer in context of original question and topic • For each of these, DeepQA spawns independent thread attempting to

prove it • Searches content sources for evidence supporting or refuting each

hypothesis • Applies hundreds of algorithms for each evidence hypothesis pair that

dissects and analyzes along different dimensions of evidence

17

Types of Dimensions of Evidence

• Type classification • Time • Geography • Popularity • Passage support • Source reliability • Semantic relatedness

18

Dimensions of Evidence for Jeopardy!

19

Scoring Features

• These features/scores are then combined based on their learned potential for predicting the right answer resulting in a ranked list of candidate answers, each with a confidence score indicating degree to which the answer is believed to be correct, along with links back to the evidence

20

Deep QA for Differential Diagnosis

21

Advantages of Dr. Watson Approach

• Represents new architecture for evaluating unstructured content • Different from traditional expert systems using forward reasoning

(data to conclusions) or backward reasoning • Unlike systems such as Stanford’s Mycin that used If-Then

statements: • If

• The stain of the organism is grampos and the morphology of the organism is coccus and the growth conformation of the organism is chains

• Then • There is suggestive evidence that the identity of the organism is

streptococcus

22

Advantages of Watson Approach

• If then approach is costly and difficult to develop and maintain • Traditional expert systems are brittle because underlying reasoning

engine requires perfect match between input data and existing rule forms

• Not all rule forms can be known in advance for all forms that the input data may take

23

Advantages of Watson Approach

• Watson uses NLP and variety of search techniques to generate likely candidate answers in hypothesis generation (analogous to forward chaining”)

• Uses evidence collection and scoring (analogous to “backward chaining”)

• These make DeepQA more flexible, maintainable, and scalable as well as cost efficient in terms of staying current with vast amounts of new information

24

Clinical Setting

• Deep QA can develop diagnostic support tool using the context of an input case (information about patient’s medical condition)

• Generates ranked list of differential diagnoses with associated confidences

• The dimensions of evidence include • Symptoms • Findings • Patient history • Family history • Demographics • Current medications • Many others

25

Is There A Need for Artificial Intelligence In Medicine? Do Physicians Need Assistance?

26

Motivation for Artificial Intelligence Software in Medicine

• Schiff • Diagnostic errors far outnumber other medical errors by 2-4X

• Elstein • Diagnostic error rate of about 15% in line with autopsy studies

• Singh and Graber • Diagnostic errors are single largest contributor to ambulatory

malpractice claims (40% in some studies) and cost about $300,000 per claim

• Graber • Literature review of causes of diagnostic error suggest 65% system

related (e.g. communication) and 75% had cognitive related factors

27

Cognitive Errors Graber et al Diagnostic Error in Internal Medicine, Arch Intern Med 2005; 165:1493-1499

• Cognitive errors primary due to “faulty synthesis or flawed processing of the available information”

• Predominant cause of cognitive error was premature closure (satisfaction of search in diagnostic imaging) • Failure to continue considering reasonable alternatives after an initial

diagnosis was reached

28

Cognitive Errors

• Other contributors to cognitive errors • Faulty context generation – lack of awareness of aspects of patient info

relevant to diagnosis • Misjudging salience of a finding • Faulty detection or perception • Failed use of heuristics – assuming single rather than multifactorial

cause of patient symptoms

29

Cognitive Errors

• Graber suggested augmenting “a clinician’s inherent metacognitive skills by using expert systems”

• Suggested that clinicians continue to miss diagnostic information and “one likely contributing factor is the overwhelming volume of alerts, reminders, and other diagnostic information in the Electronic Health Record”

30

Previous Attempts at Artificial Intelligence in Medicine

• Mycin- Stanford • Doctoral dissertation of Edward Shortliffe designed to identify bacterial

etiology in patients with sepsis and meningitis and to recommend antibiotics

• Had simple inference engine and knowledge base of 600 rules • Proposed acceptable therapy in 69% of cases which was better than

most ID experts • Never actually used in practice largely due to lack of access and time

for physician entry >30 minutes • Caduceus – similar inference engine to Mycin and based on Harry

Pope from U of Pittsburgh’s interviews with Dr. Jack Myers with database of up to 1,000 diseases

31

Previous Attempts at Artificial Intelligence in Medicine

• Internist I and II – Covered 70-80% of possible diagnoses in internal medicine, also based on Jack Myers’ expertise

• Worked best on only single disease • Long training and unwieldy interface took 30 to 90 minutes to interact

with system • Was succeeded by “Quick Medical Reference” which was

discontinued ten years ago and evolved into more of a reference system than diagnostic system

• Each differential diagnosis includes linkes to origin evidence to provide meaningful use of EMR’s and supports adoption of evidence based medicine/practice

32

Medical Diagnostic Systems

• Dxplain used structured knowledge similar to Internetist I, but added hierarchical lexicon of findings

• Iliad system developed in ‘90s added probabilistic reasoning • Each disease had associated a priori probability of disease in population

for which it was assigned

33

Diagnostic Systems using Unstructured Knowledge

• ISABEL uses information retrieval software developed by “Autonomy” • First CONSULT allows search of medical books, journals, and

guidelines by chief complaints and age group • PEPID DDX is diagnosis generator

34

Diagnosis Systems Using Clinical Rules

• Acute cardiac ischemia time insensitive predictive instrument uses ECG features and clinical information to predict probability of ischemia and is incorporated into heart monitor/defibrillator

• CaseWalker system uses four item questionnaire to diagnose major depressive disorder

• PKC advisor provides guidance on 98 patient problems such as abdominal pain and vomiting

35

Reasons Current Diagnostic Systems Aren’t Widely Used

• They aren’t integrated into day to day operations and workflow of health organizations and patient information is scattered in outpatient clinic visits and hospital visits and their primary provider and specialists

• Entry of patient data is difficult – requires too much manual entry of information

• They aren’t focused enough on recommendations for next steps for follow up

• Unable to interact with practitioner for missing information to increase confidence and more definitive diagnosis

• Have difficulty staying up to date

36

Watson in the News This Week As Oncology Librarian

• March 22, 2012 -- Memorial Sloan-Kettering Cancer Center (MSKCC) and IBM plan to collaborate on the development of a powerful tool built on IBM's Watson artificial intelligence platform that will provide medical professionals with improved access to current and comprehensive cancer data and practices, MSKCC said.

• The initiative will combine the computational and language-processing ability of IBM Watson with MSKCC's clinical knowledge, existing molecular and genomic data, and repository of cancer case histories in order to create an outcome and evidence-based decision-support system, according to MSKCC

37

Watson in the News as Research Librarian

• Development work has begun for the first applications, which include lung, breast, and prostate cancers. The goal is to begin testing the tool with a small group of oncologists in late 2012, with wider distribution planned for late 2013, MSKCC said.

• The computer will assist doctors in making diagnoses and treatment decisions by mining current information and alerting doctors to new developments and research,

38

• "Sloan-Kettering and IBM are already developing the first applications using Watson related to lung, breast, and prostate cancers, and aim to begin piloting the solutions to some oncologists in late 2012, with wider distribution planned for late 2013.”

39

Watson as Diagnostician

40

41

Google Search

42

43

44

Google Search: Uveitis Cause

45

46

Google Search: Uveitis, Arthritis, Circular Rash, Headache

47

48

49

50

User Information Design for Decision Support

51

My Involvement in Helping to Train Dr. Watson

• Initial research and grant to help educate Watson in medical domain • Could Watson software for Jeopardy! be successfully ported into the

medical domain? • Began discussing challenge associated with NEJM Clinico-Pathological

Conference • Talked about books and journals and other sources that could augment

the general knowledge built into the Jeopardy! playing software

52

After Jeopardy! Match: Initial Reactions/Expectations

• E-mails and interviews from all over the world: • Most were incredibly impressed with potential for medicine and

opportunities for the future • Some however:

• SKYNET and end of world as we know it • Pre-medical students speculating that it really doesn’t make sense to attend

medical school any more • Physicians writing blogs predicting that they would be replaced by the

computer within a short period of time

53

Taking Watson to Medical School

• Want 3 components similar to medical students education • Book knowledge • Sim Human Model • Experiential learning from actual EMR

54

Book Learning

• Textbook, journal, and Internet resource knowledge. Quiz materials • Like medical student this alone not enough don’t want to make

hypochondriac

55

Advancing Deep Q/A’s Medical Knowledge

• Continue to develop medical knowledge database • Harrison’s • Merck • Current Medical Diagnosis and Treatment • American College of Physicians Medicine • Stein’s Internal Medicine • medical Knowledge Self Assessment Program • NLM’s Clinical Question Repository

56

Advancing Deep Q/A’s Medical Knowledge

• Use New England Journal of Medicine 130 CPC cases and quiz material • Additional CPC cases at U of Maryland

• Begin developing interactive capability to develop hypotheses and refine them depending on the answer to those questions

• Develop a tool that allows for physician feedback to the system for various hypotheses so community can interact and teach Watson

57

SIM Human

• SIM Human model of physiology – work done at the University of Maryland School of Medicine and UMBC by Dr. Bruce Jarrell and colleagues

• Want to have understanding from model of physiology • Work has been done to create simulations of disease processes and

then observe how it affects other physiology in the body

58

Clinical/Hospital “Experience”

• Consumption of electronic medical record which is largely just paper represented digitally, cannot search for “rash” for example

• Access to records at U of Maryland and VA but also larger repositories from the VA in de-identified manner

59

Electronic Medical Record Challenges and Limitations

• Epic system at the University of Maryland • VA’s VISTA System • University of Maryland EPIC system

• EMR • Electronic version of paper records • Review large number of discharge summaries • Review progress notes and structured and unstructured additional

information from EMR

60

IBM and VA Team Review of EMR

• Patient EMR such as VA’s highly publicized and praised VISTA revealed numerous challenges

61

Despite the fact that virtually 100% of patient information is available in the electronic EMR with records going back more than 15 years

• Not possible to search for a term within or among patient records such as “rash”

• Majority of data is unstructured and in free text format • Much of the text in progress notes and other types of notes is highly

redundant since interns and residents and attending physicians typically cut and paste information from lab and radiology and other studies and other notes

• Information is entered with abbreviations that are not consistent and misspellings

62

Patient Problem List

• Patient problem list has no “sheriff” and each physician is free to add “problems” but very few delete them for “problems” that are temporary

• The problem lists themselves often have contradictory information

63

Medical Domain Adaptation

• 5000 questions from American College of Physicians Doctor’s Dilemma competition

• E.g. • The syndrome characterized by joint pain, abdominal pain, palpable

purpura, and a nephritic sediment • Henoch-Schonlein Purpura

• Familial adenomatous polyposis is caused by mutations of this gene: APC gene

• Syndrome characterized by narrowing of the extrahepatic bile duct from mechanical compression by a gallstone impacted in the cystic duct: Mirizzi’s Syndrome

64

3 Areas of Adaptation for Deep QA

• Content • Organizing domain content for hypothesis and evidence generation

such as textbooks, dictionaries, clinical guidelines, research articles • Tradeoff between reliability and recency

• Training • Adding data in the form of sample training questions and correct

answers from the target domain so system can learn appropriate weights for its components when estimating answer confidence

• Functional • Adding new question analysis, candidate generation, and hypothesis

evidencing analytics specialized for the domain

65

Content Adaptation

• Text content is converted into XML format used as input for indexing • Text analyzed for medical concepts and semantic types using Unified

Medical Language System terminology to provide for structured query based lookup

• “Corpus expansion technique” used by DeepQA searches web for similar passages given description of symptoms for example and generates pseudo documents from web search results

66

Medical Content Sources for Watson Include:

• ACP (American College of Physicians) Medicine

• Merck Manual of Diagnosis and Therapy • PIER (collection of guidelines and evidence

summaries) • MKSAP (Medical Knowledge Self Assessment

Program study guide from ACP) • Journals and Textbooks

67

Discovering the Untapped, Disconnected Gold Mines of Clinical and Research Data

• Despite all of the advances in computer technology we are arguably still at the paper stage of research as far as ability to discover and combine important data • Research data including those associated with major

medical journals and clinical trials are typically created for a single purpose and beyond a one or two manuscripts, remain largely locked up or inaccessible

• Even when the data are made accessible, they are typically associated with limited access through a proprietary Internet portal or even by requesting data on a hard drive

• Often requires submission of a research plan and data and then a considerable wait for permission to use the data which is often not granted

68

ADNI

• Alzheimer’s Disease Neuroimaging Initiative • Excellent example of patient data and associated images with great

sharing model • However requires access through their own portal and requires

permission from ADNI Data Sharing and Publications Committee

69

CTEP (NIH Cancer Therapy Evaluation Program) Pediatric Brain Tumor Consortium One of the Better Sources of Data

• As an NCI funded Consortium, the Pediatric Brain Tumor Consortium (PBTC) is required to make research data available to other investigators for use in research projects

• An investigator who wishes to use individual patient data from one or more of the Consortium's completed and published studies must submit in writing: • Description of the research project • Specific data requested • List of investigators involved with the project • Affiliated research institutions • Copy of the requesting investigator's CV must also be provided.

• The submitted research proposal and CV shall be distributed to the PBTC Steering Committee for review

• Once approved, the responsible investigator will be required to complete a Material and Data Transfer Agreement as part of the conditions for data release

• Requests for data will only be considered once the primary study analyses have been published

70

Institutional Database General Practice Research Database

71

Institutional Database: VA’s Corporate Data Warehouse Vinci

72

Disease Specific Databases Alzheimer’s, Parkinson’s, Schizophrenia

73

Cornucopia of Sources of Data for Dr. Watson

• University hospital databases • Large medical system e.g. Kaiser Permanente data warehouse • Insurance databases such as WellPoint • State level databases

74

Discovering and Consuming Databases

• At best, freely sharable databases are accessed using their own idiosyncratic web portal

• Currently no index of databases or their content • No standards exist to describe how databases can “advertise” their

content and availability (free or business model) and their data provenance and sources and peer review, etc.

• Would be wonderful project for AMIA or NLM to investigate the creation of an XML standard for describing the content of databases

• This will be critical to the continuing success of the Dr. Watson project in my opinion

75

Medical Guidelines

• Medical guidelines are increasingly being put into machine intelligible form although this is not an easy process

• Incorporating these into Watson software could serve multiple purposes including health surveillance, could factor into diagnostic decision making, and could be an early implementation of the Watson technology

76

Peleska et al: General Graphic Model Making Guidelines “Formal” and Machine Readable 2003 European Guidelines on Cardiovascular Disease Prevention and 2003 ESH/ESC Hypertension Guidelines

77

The Electronic Medical Record

• The transition to the 3rd year of medical school begins a new phase in education from theoretical to empirical

• Medical students are exposed for the first time to the wards and of course, importantly, to one of their major jobs for the next few years: • Maintenance and review of patient charts, nowadays the Electronic

Medical Record

78

Introducing Dr. Watson to the Electronic Medical Record

79

Watson and the EMR

• Despite the tremendous strides we have made toward an electronic medical record, we are really just at the 1.0 stage and arguably most current EMR systems really represent just a digital form of paper

• The Watson development team was really surprised when we reviewed the EMR at how primitive it was, even in 2011 • Lack of ability to search for terms within a patient’s record • Lack of ability to search across patient records • Lack of ability to perform basic statistics or have access to basic

decision support tools in EMR

80

EMR

• The diagnosis of a specific type of pneumonia, for example, can be made according to patient signs and symptoms using journal articles and textbooks

• But it can also be made more reliably by a system such as Watson by also mining the local EMR database as to what diagnoses have been made over the past few days, weeks, months, etc. locally

81

EMR

• It can then be further refined by not necessarily being constrained to tentative diagnoses that have been made but the microbiology/pathology proven causes of pneumonia

• The EMR provides empirical data about the association of these signs and symptoms with diagnoses and the means to verify what was found by lab tests etc.

82

EMR Challenges

• Challenges mining EMR • Unstructured free text with abbreviations, variable terms (e.g. MRI

terminology) • Difficulty in having Watson technology analyze large databases such as

VA’s EMR due to PHI concerns and need to stay within the firewall • Watson needs to incorporate the concept of changing signs and

symptoms in a patient over time which creates added dimension to diagnosis of a single patient presentation

• Challenge is the fragmentation of electronic medical records by multiple hospitals, clinics, outpatient settings, etc.

83

EMR Opportunities

• Watson can gain empirical knowledge of vast numbers of physicians and patients in a way that would not be possible for any single practitioner

• Watson could use EMR to perform research and discovery in healthcare such as unanticipated drug responses and interactions and factors impacting patient response to therapy

• Watson can be impetus to medical community for the development of more structured EMR in a more friendly machine readable format

84

Personal Health Records May Help Ameliorate Fragmentation of EMR’s Hospital and Clinics and Offices

• PHR’s will enable Watson to get all information in one place when patients centralize and take control of their own electronic health records

• Patients will be able to control level of access to their information

85

ADDITIONAL APPLICATIONS FOR DR. WATSON

86

Additional Applications for Dr. Watson

• Surveillance – e.g. Los Alamos Labs • Bioterrorism • Drug • Infectious Disease

87

Chart Review and Patient Problem List Sheriff

• Review for patient safety issues • Computerized patient problem list

• IBM team and I found patient problem list typically poorly maintained and updated

• Problems not deleted when they are no longer important • Contradictions in patient problem list • Patient on medications not corresponding to problems on the list

88

Personalized Medicine

• Dr. Watson software can utilize genomic and proteomic information in addition to patient signs and symptoms to provide personalized diagnostic and treatment information

• Will be able to utilize an increasing number of genomic and proteomic databases such as The Cancer Genome Atlas and The Million Veteran Program

89

Synthesis/Display of Complex Information in EMR

90

Utilizing the NCI caBIG Semantics and Technologies To Support Phenotype/Genotype Clinical Analysis for Personalized Medicine in the Diagnosis of Glioblastoma Multiforme

91

Current Dr. Watson Opportunities for Improvement

• Need to understand to listen and human speech including accents • Needs to have improved ability to understand abbreviations and

medical jargon • Needs mechanism to obtain feedback (learn) from physicians using it

• Continue to refine and improve user interface to allow feedback and refinement of algorithms

92

Interactive

• Emergency Department Scenario • Requires “real-time” decision making • Cannot use same model with all information entered

into the chart before Watson makes its assessment and recommendations

• Need better systems to capture information at point of care

• Vital signs and lab and signal monitoring • Do we need additional methods of inputting data? • Do we need to capture live conversations with

providers and patients?

93

Current Opportunities for Improvement

• Could use more personality • Female voice chosen for Siri after much research and feedback

• Needs to understand nuances of communication such as patients questions expressing emotions such as fear etc.

94

Siri: Artificial Intelligence Devices Say the Darnest Things

95

Watson Opportunity: As Unifier for Interoperability and Test Bed

• Potential for Watson to be bridge to allow connectivity and interoperability since so many islands currently being set up with health information exchanges at city and state and other levels

• Watson or Watson like technology may provide test bed for standards in medicine and may improve interoperability

96

Teaching Dr. Watson Bedside Manners

• According to a study done by the Mayo Clinic in 2006, the most important characteristics patients feel a good doctor must possess are entirely human

• According to the study, the ideal physician is confident, empathetic, humane, personal, forthright, respectful, and thorough

• Watson may have proved his cognitive superiority, but can a computer ever be taught these human attributes needed to negotiate through patient fear, anxiety, and confusion? Could such a computer ever come across as sincere?

97

Turing Test

• Introduced by Alan Turing in his 1950 paper “Computing Machinery and Intelligence”

• Opens with the words “I propose to consider the question, ‘Can machines think?”

• Asks whether a computer could fool a human being in another room into thinking it was a human being

• Modified Dr. Watson Turing Test might ask: Can a computer fool a human being into thinking it was a doctor?

98

Ultimate Challenge: Medical Imaging Scientific American June 2011 Testing for Consciousness Alternative to Turning Test Christof Koch and Giulio Tononi

99

Imaging May Be Ultimate/Future Frontier For Dr. Watson

100

Does Watson Obviate Need for Standards and Structure?

• No, in order to achieve their full potential we will need to make our medical records more structured and standardized, and rethink how we can make our clinical trial and other research databases more readily discoverable and reusable

• These changes will also accelerate interoperability and information exchange which will improve healthcare

101

Conclusions

• I am absolutely convinced that natural language processing and Artificial Intelligence applications such as IBM’s Dr. Watson will have a major impact on the practice of medicine in the very near future

• It will result in more cost effective, higher quality care and will help to decrease the disparities of care that we currently see geographically, socioeconomically, and according to subspecialty

• It will also allow us to finally achieve true personalized medicine, taking clinical signs and symptoms and history and laboratory information and diagnostic imaging and genomics and proteomics into account to personalize treatment recommendations

102

Conclusion

• Dr. Watson will evolve as an amiable, knowledgeable, fast, and reliable assistant

• If there are any pre-med students out there in the audience, please do plan to attend medical school and rest assured that Dr. Watson will require your wisdom, common sense, and humanity in order to be a continuing and evolving success

103

• The Watson Q/A technology and Jeopardy demonstration have captured the imagination of many people including those in healthcare and this may provide a critical springboard to revive many of the excellent initiatives on artificial intelligence applications in medicine

• The potential of these to revolutionize medicine is tremendous and exciting

104

DR. WATSON – A PROMISING STUDENT IN PURSUIT OF SMARTER MEDICINE

Eliot Siegel, M.D. Professor and Vice Chair University of Maryland Department of Diagnostic Radiology