BelBi2016 presentation: Hybrid methodology for information extraction from tables in biomedical literature

Hybrid methodology for information extraction from tables in biomedical literature

Nikola Milošević, Cassie Gregson, Robert Hernandez, Goran Nenadić

Contact: [email protected]

Literature growth

• MEDLINE contains more than 26 million citations• Number of citation is growing exponentially• 2100 new articles published daily in biomedicine• Professionals are no more able to cope with the state-of-the-art

Text mining

Source: https://www.jisc.ac.uk/reports/value-and-benefits-of-text-mining

Table mining• Current text mining efforts focus on main text of the article• Usually ignore tables and figures• Tables contain

• Settings of the experiment (patient characteristics, arms, dosages, etc.)• Results of the experiment• Definition of terms and quantitative scales• Examples (i.e. questionnaires)• …

• Article information are incomplete without tables (and figures)

Table complexity

One dimensional (list) table Two dimensional (matrix) table

Table complexity (2)

Multi-dimensional (super-row) table

Multi-dimensional (multi-table) table

Challenges

• Dense content• Variety of layouts• Variety of value representation formats• Misleading visualization markup• Lack of resources (labelled datasets)

Aim and objectives

• Create a multi-layered approach to mining information from tables

• to facilitate largescale semi-automated extraction • curation of data stored in tables

Table mining methodology overview

Functional processing

• Classifies cells to functional classes• Header, • super-row, • stub, • data

• Uses heuristics based on content and position

• Described in: Milosevic, N., Gregson, C., Hernandez, R.,Nenadic, G.

Disentangling structure of tables in scientific literature. In Proceedings of the 21th International Conference on Applications of Natural Language to Information Systems (NLDB 2016) (2016), Springer.

Structural processing

• Determines relationships between cells• Using cell functions and table structure classifies table into one of the structural table type:

• List• Matrix• Super-row• Multi-table

• Based on the type, set of rules resolves the relationships• Milosevic, N., Gregson, C., Hernandez, R.,Nenadic, G. Disentangling structure of tables in scientific literature. In Proceedings of the 21th International Conference on Applications of Natural Language to Information Systems (NLDB 2016) (2016), Springer.

Semantic tagging

• Semantically tags terms, phrases or words• Knowledge sources (UMLS, DBPedia, WordNet)• Used MetaMap for tagging with UMLS• Helps with pragmatic classification and information extraction

Pragmatic processing

• Determines the purpose of the table• Machine learning approach• Naïve Bayes, Bayes Nets, SVM, Decision trees, random forests• More specific classes -> better results• Evidence based on 2 trials

• Settings, findings, support tables - ~ 80% F-score• Baseline characteristics, Adverse events, Inclusion/Exclusion, Other - ~95%

F-score

Value identification and syntactic processing• Indemnifying the cell of interest:

• Looks at the navigational cells for lexical cues or for semantic types in tags

• Lexical cues in white and black lists• Syntactic processing

• Uses set of pattern to determine semantics of the value

• Extracts the selected value

Pragmatic classification results

• Pragmatic classification performs well with specific classes• 4 classes – baseline characteristics, adverse events,

inclusion/exclusion, other• Best performance - SVM

Information extraction results

• Extracted number of patiens

• New tests on extracting patient age, adverse events (using UMLS)

Patiens’ age

Adverse reactions

Lessons learned

• Table mining requires multi-layered analysis• Functional and structural analysis are crucial• Semantics of value presentation patterns• Semantic tagging helps• Machine learning helps in certain steps (i.e. pragmatic analysis)• Combination of heuristic based and machine learning based steps• Availability:

• https://github.com/nikolamilosevic86/TableAnnotator• https://github.com/nikolamilosevic86/TableInformationExtractionScripts

Future plans

• Develop easy to use methodology• Develop UI tool (wizard) for information extraction from tables• Improve the methodology• Compare heuristic based vs machine learning based IE• Examine methods for unbalanced datasets

Acknowledgements

Dr Michele Filannino

Dr Azad Dehghan

Nikola MiloševićRuth Stoney

Maksim Belousov

Dr Goran Nenadić

Robert Hernandez

Cassie Gregson

Richard Boyce

Jodi Schneider Steven DeMarco

[email protected]