Identifying Safety Signals by Data Mining the FDA Adverse Event Reporting System with Empirica Signal

Identifying Safety Signals By Data Mining the FDA Adverse Event Reporting System with

Empirica Signal

Chris Wocosky, Lead Business ConsultantLife Sciences Business Unit

facebook.com/perficient twitter.com/Perficientlinkedin.com/company/perficient

Perficient is a leading information technology consulting firm serving clients throughout

North America and Europe.

We help clients implement business-driven technology solutions that integrate business

processes, improve worker productivity, increase customer loyalty and create a more agile

enterprise to better respond to new business opportunities.

About Perficient

• Founded in 1997

• Public, NASDAQ: PRFT

• 2013 revenue ~$373 million

• Major market locations throughout North America Atlanta, Boston, Charlotte, Chicago, Cincinnati, Columbus, Dallas, Denver, Detroit, Fairfax, Houston, Indianapolis, Los Angeles, Minneapolis, New Orleans, New York City, Northern California, Philadelphia, Southern California, St. Louis, Toronto and Washington, D.C.

• Global delivery centers in China, Europe and India

• >2,100 colleagues

• Dedicated solution practices

• ~90% repeat business rate

• Alliance partnerships with major technology vendors

• Multiple vendor/industry technology and growth awards

Perficient Profile

Oracle Products• Argus Safety Suite

• Argus Safety Japan• Affiliate• Interchange• Reconciliation• Dossier• Insight

• AERS• TMS• Empirica Signal• Empirica Topics

Our Solutions Expertise

Life Sciences Business UnitSafety & Pharmacovigilance

Provides Consulting, Implementation, Integration, Upgrade, Migration and Hosting Services for the most widely used and highly sought after drug safety, pharmacovigilance, risk, and signal management systems in the world.

Introduction

Industry Experience: 17 years• Medical Device Manufacturing

– Edwards Lifesciences (formerly Edwards Laboratories) Abbott Vascular (formerly Guidant), J&J Biosense Webster

• Cardiovascular and Electrophysiology– Tissue & Mechanical Heart Valves, Coronary

Stents and Ablation Catheters• Quality/Reliability Engineering, Quality Assurance,

Regulatory Compliance and Complaint Management

Software Experience: 8 years• Software Application Sales

– ArisGlobal, Phase Forward, Oracle, BioPharm Systems (now Perficient)

• Safety & Pharmacovigilance Products– Register, inforMed, ARISg; – ClinTrace, Empirica Signal, Study and Topics– Siebel AECM, Argus Safety Suite

• Pre-Sales Consulting, Implementation Consulting, Training

Christina Wocosky

Lead Business ConsultantSafety and Pharmacovigilance

Life Sciences Business Unit o: 951-676-9501 m: 951-505-7011

chris.wocosky@perficient.com

Agenda

• Background of Empirica Signal

• Why Empirica Signal?

• Product Overview

• Q&A

Background

• Developed 2003-2005 as a (CRADA) Cooperative Research and Development Agreement between FDA and Lincoln Technologies, Inc. lead by Bill DuMouchel and Ana Szarfman.

• Objective: User friendly / web based tool for FDA

• WebVDME – Web Visual Data Mining Environment

• Conclusion: Tool assists in identifying unusual patterns – BUT – the patterns need interpretation!

• 2005 - Phase Forward acquired LT – rebranded as Empirica Signal

• 2010 - Oracle acquired Phase Forward

• In use at FDA today

Regulatory Landscape

• Increased focus on technology to save time and money

• Companies are expected to do more with tighter budgets

• Global requirements growing – still not harmonized

• An increased focus on safety related issues

Drugs Withdrawn Last 5 years*

Withdrawn Drug name Remarks Country(s)2013 Tetrazepam Serious cutaneous reactions. EU

2011 Drotrecogin alfa (Xigris)

Lack of efficacy as shown by PROWESS-SHOCK study

WorldWide

2010 Propoxyphene (Darvocet/Darvon)

Increased risk of heart attacks and stroke. WorldWide

2010 Gemtuzumab ozogamicin (Mylotargz)

No improvement in clinical benefit; risk for death.

US

2010 Rosiglitazone (Avandia)

Risk of heart attacks and death. This drug continues to be available in the U.S.

EU

2010 Sibutramine (Reductil/Meridia)

Increased risk of heart attack and stroke. Australia, Canada, China, the EU, Hong Kong, India, Mexico, New Zealand, the

Philippines, Thailand, the UK, and the US

2010 Sitaxentan Hepatotoxicity Germany

2009 Efalizumab (Raptiva) Increased risk of progressive multifocal leukoencephalopathy

Germany

*See References

Why These Drugs Were Withdrawn

• Risks to the patients, serious side effects

• Unexpected AE’s that were not detected during Phase III Clinical Trials

• Only became apparent from post-market surveillance data from the wider patient community

• Not limited to drugs that were ever approved by the FDA

• Some were approved in markets other than the US

Empirica Signal Solution Area

• Growing government and public concern about drug safety has increased the importance of pharmacovigilance

– Traditional methods such as individual case review no longer suffice due to increased data volume and complexity

– New strategies to proactively identify and expeditiously manage emerging safety risks are needed

• In this highly competitive and risk-averse environment, it is essential for regulatory authorities as well as pharmaceutical companies to establish pharmacovigilance programs

– Need to capitalize on multiple data sources using advanced tools for developing a thorough and well understood safety profile

Process Proposed by CIOMS VIII

Signal Prioritization

Signal Detection

Signal Evaluation

GVP Module IX

The signal management process shall cover all steps from detecting signals (signal detection), through their validation and confirmation, analysis, prioritization and assessment to recommending action, as well as the

tracking of the steps taken and of any recommendations made.

The GVP Module IX Process

Signal Detection

Signal Validation

Exchange of Information

Empirica Signal

Empirica Topics

Signal Analysis and Prioritization

Signal Assessment

Recommendation for Action

Why Empirica Signal/Topics?

• The same tool developed and in use by FDA, as well as

the MHRA

• Early detection of new potential signals, or changes in signal profiles over time

• Track and document every step you take from the first moment a potential signal is identified, monitoring, verification, decisions and all actions taken

• Provide regulators with a .pdf with all the necessary information to back up decisions

If you didn’t document it, it didn’t happen!

Signal Detection and Management

Signal Detection• Monitoring of “Drug Profiles”

with visualization tools

• Quick and easy drill down to the data under the charts and graphs

• Create your own data mining runs to further refine your analysis

• Compare “like” products side by side with the same charts and graphs

Signal Management• Integrate quantitative signal

detection techniques into routine safety reviews

• Assess the latest information on all marketed products and how “signals” evolve as data accumulates over time

• Store public and private comments and blanket comments for multiple drug-event combinations

• Store findings with all other data sources

Empirica Signal Benefits

• Early detection of potential safety signals for marketed drugs• Multiple disproportionality scores available• Powerful graphical visualizations to facilitate medical interpretation• Intuitive user interface including drill-down capabilities without

requiring any programming• Built-in functionality to address confounding and assist in signal

assessment such as stratification, logistic regression and extended logistic regression

• Extensive built-in signal management, tracking and documentation functionality

Empirica Signal Drug Profiles

• Drug profiles support easy and consistent drug reviews

– Presentations of statistical and other information about one or more drugs (generic, trade or ingredient)

– Users don’t need to navigate Empirica Signal; instead they can review their drug using a pre-defined set of charts/graphs

• Display can be in slide show mode or desktop review• Full power of data mining results, including drilldown, without having to

navigate Empirica Signal• This graphical, easy to use review tool is directly accessible upon login

without the need to issue any commands

– Layouts define the charts (graphs or report outputs), options and positioning to be included

– All graphs and tables can be printed directly according to the user-defined layouts

– Access to publicly available labeling information (FDA, NIH)

Comprehensive Drug Profile Layout

Empirica Signal

• Proactively identify and expeditiously manage safety risks

• Detect and quantify safety signals by using advanced data mining techniques that can be applied to any spontaneous reporting database

• Use the most advanced set of tools to develop a thorough and well understood safety profile

Sector Map (Heatmap, Treemap)

• Overall Safety Profile of a Drug• Large tiles are the MedDRA System Organ

Class (SOC)• Smaller tiles are the Preferred Terms (PT)• Colors represent the strength of the signal –

bright red indicates the AE’s occurring significantly higher than expected with the drug – bright green indicates significantly less occurrences than expected, or potential indications.

• The chart below, is the ranking in descending order by the user selected disproportionality score – the algorythm in this example is the (EBGM – Empirical Bayes Geometric Mean) – also available are EB05,Chi or PRR.

• The focus is on a particular drug, with the background being the 5M+ cases in the FDA AERS database

Visual Presentation of Safety Signals

In this map, you can clearly see the large number of AE’s in the Cardiac SOC as represented by the bright red color.

This led to the 1998 market withdrawal of Seldane

Sector Map

• The size of PT tiles is based on the public health impact of PTs, which is computed as: (number of times the PT occurs in serious-or-fatal cases) * (proportion of cases with that PT that are serious-or-fatal).

• PTs with values over a minimum threshold are included in the sector map.

• A higher Public Health Impact score corresponds to a larger tile in the sector map.

• Since size is based on the number of cases of the PT alone (not the number of cases that have the PT and a particular drug), the tile size is stable over sector maps for different drugs.

Side by Side Comparison

Zoom In on A Sector Map

Drug Profile: Desktop View

Drug Profile: Slide Show View

Drug Profiles: Slide Show View

Drug Profile: Slide Show View

Drug Profile: Slide Show View

Access to Safety Report Data

All of the blue underlined items are hyperlinks to the underlying data

The sub-menus are the same on the charts and graphs

Access to Safety Report Data

Download Safety Report Data

Analytical Graphics for Safety Review

Data Mining, Statistics and Data Visualization

Query & Reporting

Creating a Case Series with the Query Wizard

Use selection options and/or saved lists

After selecting variable values, edit the query logic to combine query elements

Flexible Reporting and Tabulation

Empirica Signal Management “Dashboard”

• Supports “strategic pharmacovigilance,” helping drug safety groups prioritize work and focus resources on areas of greatest impact

• Combines signal scores with metrics based on case counts and changes over time

• Easy-to-use table and graphs interface with workflow and annotation tools

Integrated single-table overview

Drug & event details Signal statistics for multiple time points Number of new cases in current period Trending

50% increase in EBGM from last period Current EB05 > EB95 measured previously

Indication of labeled event (“expectedness”) Indication of FDA “sentinel event” Percent of database cases with event that were fatal Percent of database cases with event that were serious Quadrant – A, B, C, D quadrant (Patrick Waller - MHRA) Additional columns are easy to add

Annotation

• Annotations at the drug, drug-event combination and case levels

• Permits both free-text and controlled vocabulary comments• Convenient way to build and share knowledge base

Empirica Signal Capabilities

• Data mining and statistics

– Disproportionality statistics including empirical Bayes estimates of relative reporting rate (EBGM) and non-Bayesian statistics such as PRR and ROR

– Logistic regression using adverse event as a response variable– Graphical and tabular presentation of results

• Access to safety report data

– Convenient drilldown to case data from graphs and tables– Download of signal scores and case data– Available for both public and private safety databases

Empirica Signal Capabilities

• Query & Reporting

– Aggregate as well as line listing reports– Library of standard reports– Drilldown, download and publishing capabilities

• Fully web-based application environment

– All processing on the server – can be provided as hosted service– Simple web browser client – operates through firewalls

Empirica Signal Management Capabilities (Optional)

• Extends Empirica Signal functionality for routine ongoing review by safety evaluators

• Combines disproportionality measures with other relevant information such as listedness, fatal outcomes, etc. to enhance the review process

• Purpose: Timely detection of signals for marketed products while minimizing the overall monitoring effort

• Capabilities include:– Easy detection of changes in signal scores– Structured on-line assessments– Prioritize, track and document signals – ‘Drill down’ to AE reports in underlying data

Empirica Signal Topics Capabilities (Optional)

• Provides workflow, auditing and tracking capabilities for potential signals

• Supports topic attachments from multiple sources, either from inside the Empirica Signal application or from files or web site references

• Provides topics workflow assignment and deadline capabilities

• Topics can encompass multiple individual Topic Actions, each with their own description, attachments, assignees and deadlines

• Topic visibility can be controlled in great detail

• At any point in time a Topic can be represented in the form of a PDF or ZIP file for communication and archiving purposes

Key Features

• Customizable, easy-to-use Drug Profile Portal• Enhanced, configurable workflow• Support for a full range of statistical techniques• Enhanced, higher performance logistic regression • Bayesian Multi-item Gamma Poisson Shrinker interaction

calculations for runs with three or more dimensions• Direct access to safety data through drilldown and case series

features• Streamlined interface• Interactive reporting with parameter driven case selection• Options to download data and case details in native Microsoft

Excel (.xls), .pdf, or rich text (.rtf) format• Robust, comprehensive online help

Key Benefits

• Identify and monitor safety trends using the same tools used by regulators

• Prioritize and execute critical pharmacovigilance activities• Employ a variety of classical and Bayesian data mining techniques• Analyze both drug/event combinations and drug/drug interactions• Filter out false positives to focus on the most important risks• Detect problems that may have been overlooked during clinical

development• Gather intelligence on competing drugs’ safety profiles• Maintain a comprehensive history of safety related topics• Track signals and manage actions in a regulatory complaint

environment

• Hosting• Training• Integration: Argus Argus Mart Empirica Signal Topics

Perficient Offerings

Empirica Signal Data Provisioning Services

Subscriptions to periodic updates of prepared data for Empirica Signal

• FDA FOI AERS data (1968 – present)Substantial data cleaning is conducted to prepare a proprietary version of FOI AERS safety data; these same cleaning techniques are also used to prepare the FDA-internal version of the AERS data for the FDA’s use with Empirica Signal:

• Integrate incremental information at database and case level

• Harmonize MedDRA coding of adverse events throughout the cumulative database across dictionary updates

• Map verbatim drug descriptions to a controlled hierarchical vocabulary of trade and generic names (WHO-DD and WHO-ATC)

• Identify and flag duplicate cases

• FDA VAERS data (1990 – present)• Offered as a monthly or quarterly subscription

• WHO-UMC Vigibase database of reports (1968 – present)• In-house corporate adverse event databases• Occasional spiking of in-house data into public sources

Questions?

References

1. Tetrazepam:http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/human/referrals/Tetrazepam_containing_medicinal_products/human_referral_prac_000015.jsp&mid=WC0b01ac05805c516f.htm

2. Xigris: https://investor.lilly.com/releasedetail.cfm?ReleaseID=617602.htm3. PropoxypheneDarvocet/Darvon):http://www.fda.gov/Safety/MedWatch/SafetyInformation/SafetyAlertsforHumanMedicalProducts/ucm234389.htm4. Gemtuzumab ozogamicin (Mylotargz):http://www.fda.gov/safety/medwatch/safetyinformation/safetyalertsforhumanmedicalproducts/ucm216458.htm5. Rosiglitazone (Avandia):

http://www.ema.europa.eu/ema/index.jsp?curl=pages/news_and_events/news/2010/09/news_detail_001122.jsp&mid=WC0b01ac058004d5c16. Sibutramine (Reductil/Meridia): http://www.fda.gov/safety/medwatch/safetyinformation/safetyalertsforhumanmedicalproducts/ucm228830.htm7. Sitaxentan: http://www.mhra.gov.uk/Safetyinformation/DrugSafetyUpdate/CON1057528. Efalizumab (Raptiva): http://www.fda.gov/Drugs/DrugSafety/PostmarketDrugSafetyInformationforPatientsandProviders/ucm133337.htm9. CIOMS VIII: http://www.cioms.ch/index.php/publications/available-publications?task=view&id=27&catid=54

Thank You!

For more information, please contact:

chris.wocosky@perficient.comLifeSciencesInfo@perficient.com (Sales)

+44 (0) 1865 910200 (U.K. Sales)+1 877 654 0033 (U.S. Sales)

www.facebook.com/perficient

www.perficient.com

www.twitter.com/perficient

http://www.oracle.com/us/products/applications/health-sciences/safety/empirica-signal/index.html

Oracle HSGBU Quantitative Safety Signal Detection and Data Mining Bibliography March 2011 Abstracts by Year.pdf