Introduction:+ …manarbm.com/userfiles/955/files/Implementing-Risk-Based-Monitori… · ImplementingRisk.BasedMonitoring:** APrimer!

Implementing Risk-Based Monitoring: A Primer

MANA RBM Copyright 2015 For more information, contact [email protected] Page 1 of 28

Introduction: This White Paper is designed to help organizations adopt a Risk-Based Approach to Clinical Trial Monitoring. Risk-Based Monitoring (RBM) requires fundamental changes in the approaches to Clinical Trial oversight. These changes involve people, processes, and technology. The key components of Risk-Based Monitoring appear in Figure 1. Each aspect must be considered to optimally implement RBM. Figure 1 Key Components to Risk Based Monitoring Implementation

Key$Components$to$Risk$Based$Monitoring$Implementa7on$

3"

People""

with"New"Skills"EDC/ESource/

eDiary/"eConsent"

ESF/eTMF"

Clinical"Data"Warehouse/"

"Repor?ng"Tools"

Opera?onal"Reports"""Metrics"Reports""Safety"Reports"

LABS"IVR/IWR"

Data/"Opera?onal""Metrics"

Opera?onal""

Metrics"

CTMS"

Technology:"

New"Data"from"Technology/"New"Technologies"

Processes:"New"Processes"



Background In August 2011, the FDA released Draft Guidance for Industry on the Oversight of Clinical Investigations—A Risk-Based Approach. It was designed to change the focus of monitoring from an onsite, SDV focused approach to one that used more comprehensive oversight of clinical trials using technology and remote methodology. Final Guidance was released in August 20131. Key points of the final guidance were as follows:

• Wide Variety of options for monitoring can be used • Recommendation for more use of centralized monitoring using technology • Focus on patient safety and protecting human subjects • Support of risk-based methods for monitoring • Focus on Data Quality.

The EMA released its draft guidance in 2011 and its final guidance, A Reflection Paper on Risk based Quality Management in Clinical Trials2, in November 2013. The EMA paper aligned with the FDA guidance but also focused on quality. Both agencies, during this time, released a second set of guidance documents addressing the use of electronic source. . The FDA’s Guidance, “Electronic Source Documentation in Clinical Investigations” was released in 2011 with final guidance released in 20133. The EMA’s draft document, “Reflection paper on expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials” was released in 2007 with final guidance released in 20104. Many reasons mandated the release of these guidance documents and its temporal release together. Options became available to better and more comprehensively use remote oversight technology today;; technology not available in 1988 when the FDA released the original guidance for clinical trial oversight.



It also became clear that 100% Source Data Verification (SDV) did not adequately support oversight of clinical trials. In 2009, Pfizer received a 483 Warning Letter from the FDA concerning subject safety when multiple research subjects received the wrong dose of study medication. The error was not identified despite multiple onsite monitoring visits with 100% SDV. The data management group finally identified the issue several months later. After site retraining, the issue reoccurred.5

Johnson &Johnson, along with its CRO, ICON, received warning letters in 2010 for errors at a site concerning administration of study drug. A site showed multiple infusions started at the same time by the same person, a physical impossibility. Reviewing data subject by subject did not identify the issue;; it could only have been identified if a review of all subjects at the site was performed.6,7

The Clinical Trials Transformation Initiative (CTTI), a public-private partnership of over 60 organizations including the FDA, played a key role in refining the current state of Monitoring. It developed three work streams. The first was on Current Monitoring Practices, the second on Key Quality Initiatives, and the third was on Building Quality into Trials.8 The FDA Guidance adopted many of CTTI’s findings. Ten Pharma and Biotechnology companies formed a nonprofit organization named TransCelerate BioPharma (TransCelerate). It focused on enhancing the process of medicine development in noncompetitive activities. Risk-Based Monitoring was one of the first areas targeted. TransClerate published a collaborative methodology position paper and tool set for Risk-Based Monitoring in 20139. Its position paper questioned the value of source data verification (SDV) as a quality oversight measure. TransCelerate evaluated 9 studies from 6 member companies regarding the rate of SDV-generated queries. Only 7.8% of all queries generated were attributed to SDV. Only 2.4% of the SDV-only queries involved critical data. 9 Sheetz et al. published an article evaluating SDV as a Quality Control Measure10 The authors used the Medidata Insights metric warehouse to identify data changes temporally related to onsite monitoring visits from 1168 Phase I-IV biopharmaceutical studies across 53 sponsors. They found only 3.7% of the data were changed for any reason and of that percentage, less than one third of the changes were related to SDV.



They further evaluated AE’s and SAE’s reporting in relation to onsite monitoring visits. They found that 7.5% of AE’s were entered within one day of a monitoring visit and 11.8% were reported within 7 days of a monitoring visit. 1.7% OF SAE’s were reported within 1 day of an onsite-monitoring visit and 3.6% were reported within 7 days. It is not clear whether the onsite monitoring resulted in these reports as other activities may have resulted in the reporting of these AE’s and SAE’s. Further, the authors noted that remote risk-based methods would likely have identified many of these findings. Figure 2, from Sheetz et al. paper, shows the breakdown of data changes.1



The SDV findings align with the data from Smith et al.11 In their evaluation of the role of SDV in an oncology trials with 533 research subjects across 75 secondary and tertiary care centers across the UK, they found SDV was expensive, not completely error-free (patients that should have been excluded were not identified), and SDV identified errors had little impact on the results and clinical conclusion. 11 A critical aspect of RBM is rapid access to the site’s clinical data. In a February 2013 blog, Stephen Young from Medidata presented industry median values from 2009-2012 Phase II and III trials (Figure 3). He showed that the median time from eCRF entry to Data Manager Query Opened was 59-89 days. This is even more extraordinary when one considers that the median time from subject visit to query close (all queries including automatically generated queries) ranged from 30-36 days.12 These data demonstrate the need to re-evaluate the process for data review, which has traditionally required onsite monitoring visits before the data manager accesses the data. Figure 3. Cycle Timelines from Medidata Insights12



These findings emphasize that direct data entry, either into the EDC or eSource systems, provides significant value in overseeing study conduct quality. Mitchel et al. 13 reported on their experience implementing direct data entry (DDE) and RBM in a clinical trial of 18 clinical sites in the U.S. and Canada studying 180 research subjects. In that trial, 92% of the data was entered within 1 day of the subject visit and 98% within 8 days. Data review was also faster with 50% of the data reviewed within 13 hours of data entry. SDV was completed at the site for approximately 20% of the data within the EDC. There were changes on 0.8% of the pages with the majority in three areas: concomitant medications, medical history, and clinical laboratory results. RBM provides tremendous opportunities to improving clinical trial development and oversight. Demonstration studies have illustrated the benefits and regulators have provided strong incentives to improve processes. This paper will describe the changes that can and should be made to adopt the principles of RBM.

People RBM (including remote oversight) skill sets differ from than those used in traditional monitoring focused primarily on SDV. Competencies for the new monitoring role were identified and published by a cross-functional team from Pharma/Biotech Sponsor Companies, CROs, Academic Institutions, and Consultants from the U.S., E.U., and Asia.14 (See Table 1).



Table 1: New Competencies for Risk Based Monitors

Wilson et al also published a comprehensive paper focused on the new capability and new role of a remote or central monitor 15. The competencies they listed aligned with those of Cerullo et al.14 MANA Consulting (MANA) developed a self-assessment tool provided to anyone interested in assessing his or her skills on a 31-question survey (https://surveymonkey.com/s/MonitorCompetencyAssessment). 118 people responded to the survey. Most of the monitors (86%) had monitored from 2-10 years. As expected, questions concerning the new competencies were answered incorrectly more frequently than questions covering existing competencies. Each user received his/her score and the competency questions not answered correctly.

New$Competencies:$Risk$Based$Monitor$•  Evaluate$clinical$data$from$an$individual$subject$to$assure$subject$is$appropriate—Source$Data$Review$

•  Perform$centralized$review$and$interpret$data$on$site$performance$in$comparison$with$other$sites$

•  Understand$Clinical$Trial$Technology$•  Develop$consistent$strategies$for$assessing$and$resolving$problems.$$This$includes$root$cause$idenHficaHon,$remediaHon,$and$evaluaHon.$

3$



Table 2 describes common challenges to RBM Adoption.

Challenges to RBM Adoption Monitors aren’t trained to conduct the more comprehensive RBM review Understanding of how study quality will be evaluated in the absence of SDV Existing Standard Operating Procedures “Silo-ization” of functional expertise and activities Multiple data sources for information used within a clinical trial RBM seen as risk to individual and organizational viability

One challenge to adopting RBM involves the way onsite monitors have been trained over the past 10-15 years. As EDC became more prevalent, SDV became the larger focus with less concentration on comprehensively reviewing site performance. New monitor training is needed to support building new, RBM skill sets. Organizations that have not replaced SDV with the more comprehensive remote review envisioned in the Regulatory Guidance documents present another challenge. Monitors are asked to decrease their SDV of data within the trial, but there is no communication (or no corresponding comprehensive remote review) about how study oversight will be conducted. Simply eliminating SDV and not replacing it with remote study oversight does not comply with the FDA and EMA’s guidance document. Standard Operating Procedures (SOPs) can also affect how RBM is implemented. Monitors and Sponsors can agree to perform limited SDV and adopt a plan for oversight but if the data management SOPs do not allow for “SDV complete” checkboxes to be empty, the entire process can be derailed.



Monitoring a trial involves many team members with different responsibilities, different skill sets, and often, different line functions. RBM activities and responsibilities require all team members work together to identify risk factors, review the data, and make decisions about study conduct. Different team members will evaluate different risk factors and sometimes, different team members will evaluate the same item in a different context. The Quality Management Plan and its components must align to identify who will review what data, the timeframe for the review, and the process for presenting findings to the larger project team. Table 3 illustrates how different functional groups use study data in distinct ways. Each needs to see the data, but will “take away” different things from accessing the data. Note all rates are normalized to allow efficient comparisons across sites. Table 3. Impact of data on different functional groups. Project

Manager OnSite Monitor

Data Manager

Medical Monitor

Remote Monitor Statistician

Site Screen Failure (SF) Rate*

Is there a global issue that requires protocol revision? Do certain I/E categories disproportionately result in SF? How will this affect budget and timelines?

Why does the site have a rate above threshold or outside normal distribution (e.g. >2SD). Is the issue education, new staff, or patient population?

Determine review requirements for screen failures.

Is there a global issue that requires protocol revision?

Review in context of enrollment and site distribution? Are there trends by region? Review outlier performance in greater detail.

Do rates align with assumed power calculation?



Project Manager

OnSite Monitor

Data Manager

Medical Monitor

Remote Monitor Statistician

Early Termination Rate** (ETR)

Will study meet timelines and budget constraints?

The rate at my site was higher than the other sites;; need to understand why. Has the Investigational Product been accounted for in the early terminations?

Have all subjects completed the appropriate assessments for early termination?

Does a high, early termination rate indicate safety concerns?

Is the ETR the same across all sites? Is it a study-wide, region, or site issue?

Do we have enough subjects to analyze the data?

SAE Rate**

Are the rates worrisome from a study conduct or completion?

Is the rate of my site in the above threshold or an outlier? If so, what is root cause and how do I address? Do I have all the documentation to support SAE’s?

Are all data available for the SAE in the clinical database? Is there alignment with SAE database? Do we need to institute standard data collection of a SAE of special interest? Have I identified all cases through medical coding?

Is there a safety signal from the SAE rates? Is it what was projected? Is there a signal when AE/SAE’s are combined?

What are the rates for the entire study and then by site? If high rates are expected, which sites have rates outside the expected (too high, too low)?

Are additional analyses needed to better evaluate the SAE’s reported?

*normalized for all subjects signing the Informed Consent **normalized for all subjects that were randomized and received Investigational Product (IP)



Processes RBM requires significant process changes. Developing a Quality Management Approach Risk Analysis is the first step in developing a Quality Management Approach and instituting RBM. This starts in protocol development, Any opportunity to eliminate risk through the protocol design should be incorporated. Once that has been accomplished, the team must develop risk mitigation plans for all other risk areas which must be detailed, as the team develops its Quality Management Plan, Data Review, Monitoring and Safety Management plans. The team must identify critical data elements (e.g., primary and secondary efficacy elements), critical processes (e.g., rater reviews), patient safety indicators (investigational product management, dosing, and administration, safety assessments), protocol compliance, and GCP compliance indicators. Once the issues have been identified, the team must define the impact, likelihood of occurrence, and how the data will reviewed and by whom. TransCelerate released a Risk Assessment and Categorization Toolkit (RACT)17. This tool facilitates a cross-functional identification and management approach to Risk Assessment and categorization. It highlights specific factors that add to the trial complexity and raises the risk of proper study completion. The cross-functional discussion of risks and mitigation strategies should be completed prior to protocol finalization. Setting thresholds provides another challenge. When setting thresholds, it is helpful to identify the usual frequency or distribution (e.g., queries, screen failures, etc.). High risk can be set as a percentage above (e.g., 20%) or below. Checking how well these thresholds identify sites that are identified by other methods of oversight may help to adjust these values. For larger studies, using Z scores, a function of the distribution of data, can be extremely helpful. Assuming normal distribution, a Z score of 2 is 2 Standard Deviations (SD) above the mean. A Z score of -2 is 2 SD below the mean, providing a rapid and easily recognizable approach to identifying outliers. Risk can be classified as program, trial, or site level. In setting “risk levels” using a medium site risk is a reasonable choice for all but sites with new investigators, investigators that are also sponsors, investigators



that are extremely busy, or investigators that have had a 483 or other regulatory action. Sites with these characteristics should be considered high risk sites with more extensive oversight. When teams proactively establish how to identify and review high-risk areas, systems can be developed to assure the data are available as needed for review, the person conducting the review is identified, and the frequency of the review is determined. Spending the time to how issues will be identified, and to identify actions required based on potential findings, assures team members work efficiently and appropriate training, if needed, is instituted. Remote Study Management Setting up trials for optimal remote management assures Sponsors and CROs that better study oversight can be accomplished with fewer onsite visits. The FDA Guidance recommends increased use of remote oversight of trial conduct. For instance, EDC can be designed to collect source data for many trials. Direct data entry can be facilitated when sites use tablets to enter the data directly when seeing subjects, assuring immediate access to subject data and rapid identification of site errors. This often requires a team effort between Clinical and Data Management to assure that primary data supporting trial conduct is incorporated in addition to the data required for analysis. Remote management of Site Regulatory Binders can be conducted online while maintaining site control of the investigator site files by using electronic Investigator Site Files. Note: all eTMF’s do not have the ability for sites to maintain its documents. Site access and control and electronic signature capabilities are key components in developing processes for remote document management. In many systems, sponsor-required documents can be automatically routed to the eTMF. Systems today can provide the appropriate level of access control to protect personal identifying information. Processes for the system set up, role-based access, training, management of site-



specific essential document checklists, and document review and archiving must all be developed, implemented, and managed. Through the use of e-informed consents (eIC) or the method in the FDA’s recommendation for converting paper documents to certified electronic source, monitors can conduct immediate, remote review of all informed consent documents. These approaches assure a key component of subject safety is present and completed correctly. This can be accomplished while protecting a subject’s personal health information (PHI). The FDA released guidance on the use of eIC in March, 2015 which allowed consent to be given remotely using electronic systems.. It also described the materials needed during an audit, which include “records and reports made by the investigator, including site-specific versions of the eIC, materials submitted to IRBs for review and approval, all amendments to the site-specific eICs and all subject-specific signed eICs.” 23 Data to support the remote oversight of investigational product (IP) can also be collected to assure proper management of IP as well as reconciliation. By having all data and documents available remotely, more comprehensive remote study management can be conducted. Different team members may assume different activities. For instance, the monitor can maintain the eISF essential document checklist, but a Clinical Trial assistant (at a lower hourly rate) can check informed consents to determine all documents are available and correct;; saving significant monitoring time and significantly decreasing the requirements for onsite monitoring time. While some Sponsors may worry that decreased time onsite may be a problem, there is no direct correlation to monitoring time spent onsite and site problems. Intensive training, at the site initiation visit, with the Onsite monitor can establish a strong working relationship with the site. Since the onsite monitors will not be traveling as much, the monitors can be more available and responsive. Setting up processes for remote monitoring visits with the Study Coordinator and Principal Investigator must be discussed at the time of site selection and incorporated into contracts and other control documents.



When onsite visits are scheduled, time is available to work with the sites on issues previously identified. Our findings confirm that time spent with the Principal Investigator (PI) is more productive when you provide metrics on site performance, rather than just generalities. In addition, if more remote study management is adopted, sites can use the existing storage space now available for revenue generating activities. Processes for decreasing the emphasis of SDV Assuring SOPs aligned with this new approach is crucial. This requires a careful review of all SOPs. For instance, review of the processes for database lock must not require that all data is SDV’ed. Cross-functional Review and Management In addition to developing the Quality Management Plan early, implementing RBM requires a cross-functional approach to study oversight. The project team must know who is responsible for each type of review, the timing of each review, and the method to escalate significant findings, if any, to the entire team. Issue logs, accessible by all team members, serve as a location for all team members to provide the issues they identify. This helpful tool facilitates communication and trend analysis. Review timing and integration The change in timing of the review can significantly affect workflow and team assignments. Traditionally, data managers do not review data until after the monitors have completed their onsite monitoring trips. In RBM, data management review begins immediately. It is not focused only on the edits incorporated into the EDC, but is a more comprehensive review of data across all data systems. Both onsite and remote monitors’ activities are also divided differently. The onsite monitor should begin the SDR immediately and start reviewing the informed consents. Since the Site monitor will perform more management remotely, beginning these processes early should not pose a problem.



Table 4 illustrates one approach to completing data review using RBM. The first tier of review is source data review of each research subject. Following that review, there is a review of data across all subjects at a site. This could include timing of assessments, dosing, or variability of data on fields such as vital signs). Following review for issues across subjects, a review of site performance (e.g., rate of screen failures, rate of early terminations, rate of deviations, queries and query response time) in comparison to all sites, allows identifications or sites that are outliers.

Risk%Based*Monitoring*Approach**

Site*Performance*Across*Sites*

Site*Performance*Across*Subjects*

Subject*SDR*

Subject*SDR*

Subject*SDR*

Subject*SDR*

Subject*SDR*

6*



Site Interactions Enhancing site interactions starts during site evaluation. The sites should receive a full explanation of study conduct from a technology and study oversight standpoint. Satisfying RBM expectations requires rapid data entry, quick query responses, direct data entry, and electronic Investigator Site Files (eISF). These expectations should be explained during site evaluations. More site personnel may have to interact with new technology and perform different functions than required in their traditional roles. For instance, if elSF are used, a person is needed to assume responsibility for learning how to use technology to upload documents. All site personnel will need to access important study documents electronically. Access to the internet within examination rooms may be needed for some direct data entry systems. With fewer onsite visits, teleconferences with the sites assume a greater importance. Assuring Study Coordinators and PI’s are available for these calls is critical. These issues are best approached at the time of study feasibility to assure everyone knows what is expected and study budgets can include these requirements. Recognizing the additional time in study budgets and setting expectations for training, data entry, query response, and regular calls are keys to successful RBM adoption. When Sponsors actively engage in implementing RBM, it yields a very positive effect on the site adopting RBM. Root Cause Analysis and Evaluation A key component of a Quality Management System involves determining the root cause(s) for issues identified. Those who provide oversight must understand why a specific issue occurred and have the ability to devise plans to correct the root cause. While the remote monitor will primarily identify issues, the onsite monitor must identify root causes at the site level. The onsite monitor will need to research and confirm root causes, design a mitigation plan, and an approach to assess its effectiveness.



Monitoring Reports and Issues logs Monitoring reports, or at the least, issue logs must be in electronic format to enable integration of the monitor’s findings into a comprehensive review of site conduct and trend analysis. Issue logs list issues, the extent of the issue (one site, multiple sites), and provide standard classifications (e.g., investigational product management, protocol compliance, SAE reporting) to facilitate comprehensive review of data issues. These logs should be accessible by all team members and are a key component of the trend analysis. Contracts Significant changes in contracts are required as follows: Site:

• Recognition of additional time needed for site training. • Time required for periodic phone calls with the Study Coordinator and PI. • Time constraints for data entry and query responses.

Monitor/CRO • Remote Monitoring time must be included. Remote monitor’s skills demand a higher billing rate than traditional monitors experienced only in SDV.

Onsite monitors will conduct more remote reviews. Paying only for onsite visits will offset this change.

Technology



21 CFR Part 1118 addressed the use of electronic signatures and defined the need for processes as well as specific technology requirements for any electronic data/documents submitted to the agency. Monitors must understand several key components discussed in the guidance as follows:

a) Managing users and their access to electronic systems b) Training on systems (and associated documentation) c) The use of electronic signatures in electronic systems and the components required for an electronic signature

d) Converting paper documents to certified electronic documents e) Use audit trail to determine which users entered data, timing of data entry, and any changes in data

f) Capabilities of each electronic system (what it can do and what it does not do) and how the systems interact.

The Common Data Interchange Standards Consortium (CDISC) has spearheaded data standardization in clinical trials. It developed standards for CRF creation (CDASH), for data submission (SDTM), and for analysis (ADAM). Using standard forms and naming conventions saves significant time and effort at every stage of the process. All submissions for drugs and biologics must be in SDTM and ADAM format. Many companies convert the data to SDTM format periodically throughout a trial. There are challenges using this data for RBM;; the data are merged and converted in SDTM. Raw data, audit trails, and query data are needed for RBM. Some companies create a modified SDTM format so fields needed for RBM are available for review and there is a mapping to the CRF data names. In these situations, more standardized reports (without the added costs for configurations/customization) are available immediately at the start of the study. Companies save significant time and costs by creating and maintaining as much standardization as possible throughout the study conduct, submission, and reporting processes.



Reports and data visualizations can also be standardized, enabling more efficient oversight and easier staff training because the team members can be trained how to interpret the data specifically on the reports they will be using. Differences in axes and data normalization can result in different interpretations when there are a wide variety of reports available across trials. Technology has enabled the remote review of more data and documents. Some technology solutions can be modified to enhance remote review and RBM. EDC and eSource—Direct data entry can be accomplished though web-based EDC solutions and tablets. It is imperative that sites have adequate internet access to use tablets for direct data entry. Sites benefit from eliminating transcription of documents. Monitors and data managers also benefit from having immediate access to the data. Questions that document GCP compliance can be incorporated into the EDC or eSource. These fields (e.g., detailing timing for vital signs, informed consent processes) enable monitors to conduct Source Data Review remotely. Many data managers may not be familiar with the additional questions the monitor will want to have documented, so cross-functional input into the EDC is needed during design. Tablet set up and testing ensures tablets work as needed by the site. The initiation visit should include site training on how to use the tablets to collect all data, including source data. While no system has “site level” organizational structure, EDC systems can be set up to collect key site data from the sites. Investigational product receipt and site delegation logs can be designed within EDC systems to support the reporting and oversight of these key components of trial oversight. eSource tablets are specifically designed to collect data directly at the time of a subject visit (direct data entry), rather than having the data entered onto paper and transcribed into an EDC system. Some focus on providing a “look and feel” like a paper chart. eSource tablets are not as robust in data exporting and data query capabilities as EDC systems, which can also be used for direct data entry. In addition, allowing sites to enter data anywhere on the electronic chart requires that the electronic charts must be reviewed frequently to assure there are not entries that would indicate a safety or protocol compliance issue. Understanding what queries are possible and what queries require manual entry is important for the data and monitoring team. Often the clinical team will assume a certain query will be generated by the data collection system, when that may not be the case.



IVR/IWR IVR/IWR outputs can be integrated into EDC or eSource systems using a web services interface. By integrating the data into the EDC and having the data come directly from the IVR/IWR, fewer changes are needed during reconciliation because the data are not entered separately into two different systems, yielding a faster time to database lock. It also allows the sites to continue working in a single system rather than having to switch systems and transcribe data—which contributes to data errors. When IVR/IWR is integrated, it is important for the project manager to plan time for integration testing and to assure both systems are ready simultaneously to facilitate integration testing during User Acceptance Testing. SAE reporting When electronic systems are used for data collection (EDC or eSource), no separate manual process is needed for faxing information to the company for manual entry into a separate database;; which is the usual practice. A more efficient and effective use of resources is to have the SAE data entered into the EDC /eSource system by the site, then transferred to the safety database, if present. All questions about the episode should be generated as queries in the EDC system so the EDC system contains the most current and complete SAE data. When data are integrated throughout the trial, with the EDC containing the most current version of SAEs, there is less time needed for SAE reconciliation resulting in a shorter time to database lock. In addition, there will be a common understanding of the SAE by all team members. ePRO and eDiaries- The FDA released guidance on the use of patient reported outcomes in 200921. That guidance specified the need to have data collected from subjects at protocol-specified times. Since this cannot be documented using paper diaries (which are subject to “parking lot syndrome” or completion of all diaries immediately before a study visit), there has been a significant move to electronic systems. When ePRO data from subjects are integrated into the EDC, it is easier for sites to evaluate data and identify any errors with completion when the site sees the subject in clinic than when the data are in separate databases.



RBM requires the comprehensive review of subject data and confirmation that investigator assessments align with patient-reported outcomes. eConsent/Informed Consent Form (ICF) errors routinely rank as one of the top 5 findings during site audits. The complexity of the Informed Consent process (e.g., multiple consents required for the study and sub studies, such as genetic testing), the frequency of amendments, the number of different languages required, and the number of organizations managing the trial all increase the likelihood of informed consent errors. eConsent systems are an excellent option for large trials that require the use of multiple languages because the translations can be incorporated directly into the eConsent system. This simplifies the process of informed consent review, particularly when there are amendments to the protocol. The systems that provide eConsent can also provide educational materials to help the research subject better understand the process. An audit trail confirms who signed the document and when. Ideally, on the same date the subject signs the informed consent, it is transferred to the EDC system. This step eliminates the risk for transcription errors. Costs increase when the ICF requires multiple amendments. The benefit is that there is a greater likelihood that the correct version of the ICF has been provided for the site to use because they are loaded and delivered from a central system, which has been through a validation process. An alternative to using an electronic system for reviewing ICF is to generate a certified copy of the signed ICF, upload it into the site’s eISF, and review it remotely shortly after the subject visit. This assures you have an executed ICF(s) for each research subject. Medical Coding is traditionally the domain of data management. It is usually performed infrequently during the trial. As part of the Within-Site and Cross-Site review, it is valuable to look at the AE’s by System, Organ, and Class (SOC) as part of the review. Simply looking at the rate of AE’s across subjects within a site and across sites, may not identify important safety issues - a key requirement of RBM. Reporting and Data Visualizations are vital components in successfully implementing RBM. Barnes et al. published an in-depth discussion of the technology requirements for technology systems used to aggregate and analyze data and metrics on site and staff performance for RBM oversight .22 In their discussion, the complexity of managing multiple data sources and technology systems (e.g., EDC, labs, eInformed consent,



IVR/IWR, CTMS, Payments, eTMF) was highlighted and the need for systems that integrate and aggregate the data. They further provided user requirements for reporting and visualization, a critical component to RBM. A crucial component of the Monitor’s role in quality oversight is evaluating the research subject’s appropriateness for the study and safety oversight at the site level. Figure 3 shows how a data visualization tool enables much faster identification of issues that would be difficult to see if a monitor has to “flip” across multiple pages within the CRF and across multiple systems. This is a Graphical Patient Profile generated using J Review. In this example, it is important to see that the subject’s reported AE of anemia pre-dated treatment with the study drug. This knowledge allows the monitor to query the AE to see if the anemia worsened and to have the anemia reported as Medical History. The Graphical Patient Profiles can be built to interface with many different inputs (e.g., Central ECG reading, ePRO vs. Investigator reports, etc.). It is imperative that all data sources are represented in the reports, including metadata, not just the EDC data for analysis. Many companies have a data repository for analysis datasets, but the data used is often converted to SDTM for analysis, which is problematic for real-time data review. SDTM conversion for submission of data to the regulators involves data manipulation, conversion, and merging of data into subfolders. Much of this converted data, including metrics that are not included in analysis datasets, are needed for quality oversight reporting. Therefore, new reports may need to be generated using the real-time data sets and metrics. For some companies with fewer trials, using a data repository with standard RBM and Medical Monitoring reporting included may provide the most expeditious solution rather than developing their own repository and reporting system. Development of internal data repositories and reporting tools requires significant resources for development and validation as well as managing the process of regular data imports from multiple systems. In addition, it is important to have a system that will enable ad hoc reporting to allow further evaluations if issues are identified. Having a system to develop and manage study-specific issue logs with input from multiple team members is another key requirement of technologies used for risk identification and analysis.



Figure 4. Graphical Patient Profile



Implementation Options and the Change Management Process Implementing RBM may seem daunting, but companies that make the commitment gain significant improvements in study quality and significant cost savings. Since there may be insufficient numbers of monitors available and trained to conduct the more complex RBM reviews required, Wilson et al. (TransCelerate)15 recommended a new role be defined for a central monitor. Figure 5 illustrates an alternative for adopting RBM when working with a standard CRO model with Onsite Monitors that may not have the training and skills needed for the central analytic role. Figure 5. Implementing RBM with centralized remote monitors.

Clinical'Data'Warehouse!J!Review!

QlickView!

Site!Enters!Data!into!EDC!!

Monitor!Reviews!data!in!EDC!!

and!performs!SDV!at!Site!

Laboratory!Data!Sets! Third!Party!Vendor!Sponsor!Project!Managers!

Risk!Based!Monitoring!

EvaluaFon/InterpretaFon/Trend!Analysis!

Issue!Log!Development/!PotenFal!Root!Cause!

Analysis!

ConfirmaFon!of!IntervenFon!Success/Issue!

resoluFon!!

Data!Manager!Review!

Monitor!works!with!Site!to!!

Confirm!Root!Cause,!Perform!!

IntervenFons!

AddiFonal!issues!idenFfied!with!

RBM!now!followed!to!conclusion!by!

Monitor!!

EDC!data!!PRO!Data!Sets!

Sponsor!Standard!Process!

New!Inputs!into!Sponsor!Process!

IVR!Data!

Risk!Free!RBM!Process!

Data!Mapping!to!SDTM!“Like”!



This approach enables the Onsite Monitor to focus on site interactions, evaluate root causes for problems, provide interventions, and evaluate its effectiveness. The training requirements are less and the role continues to be focused on Site Interactions. This still requires a mind shift and emphasis on eliminating SDV and demonstrating how quality will be assessed. Change Management Change Management is a critical step in successfully implementing RBM. Figure 6 provides a model of Change Management that was used to successfully implement IT changes within GSK (Dement personal communication). It identifies how each component affects the ultimate success of adoption. For example, if staff does not see how they will fit into the new operational model, they will find ways to sabotage its adoption. Helping staff understand how they benefit from new opportunities or new, expanded roles, creates a favorable environment to adopt RBM. Figure 6: Change Management Model for RBM Adoption

Quality Oversight

Change Management: Change Management

The$predictable,$reliable$and$measurable$cycle$of$People$who$Perform$Processes$which$Generate$Data$which$is$integrated$and$interpreted$with$Technology$and$

Delivered$back$to$People$to Make Decisions.$

Behavior

Talen

t

Quality/Standards

Conte

xt &

Reus

e Eff

icienc

y

End to End/ Enterprise

Infrastructure Interp

retati

on/

Analy

tics

PEOPLE

DATA / INFORMATION

TECHNOLOGY

PROCESSES Perform

Which Generate

Which are Integrated & Interpreted with

And then Delivered to DECISIONS



MANA’s RBM approach uses remote trial oversight methods as much as possible. Our approach enables much broader review of data, site performance, and documentation by the entire team. Implementing MANA’s RBM approach eliminates much of the risk associated with a poor performing site and study conduct personnel. The increased visibility and access (while safeguarding protected health information) enables remote audits in a very short time without the need for travel. Manasco et al.24 performed a remote informed consent review of 10% of informed consents across 12 sites (788 subjects enrolled) within two business days with no travel required.

Summary RBM provides an improved option for quality oversight of clinical trials. It removes the focus on SDV and expands the review of study performance more broadly. While a number of organizations still focus on using RBM to identify “when to send out a monitor to the site”, MANA considers RBM a comprehensive tool to identify issues at the user level, and manage interventions remotely as possible. Implementing RBM yields more transparency into trial conduct, better quality oversight of trial conduct and faster identification of issues. Sponsors also gain significant cost savings from the decreased number of monitoring costs, for travel and onsite visits now obviated by remote RBM. References: 1. Oversight of Clinical Investigations: A Risk-Based Approach to Monitoring. U.S. Department of HHS, FDA, August 2013 OMB Control No. 0910-0733.

2. Reflection paper on risk based quality management in clinical trials. European Medicines Agency. 18 November 2013 EMA/269011/2013.

3. Electronic Source Data in Clinical Investigations. U.S. Department of Health and Human Services, Food and Drug Administration. Sept 2013. http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm328691.pdf



4. Reflection paper on expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials. 09 June 2010. EMA/INS/GCP/454280/2010.

5. Ball, M.D, Leslie K. "Pfizer Warning Letter." Letter to Mr. Martin Mackay, Ph.D. 9 Apr. 2010.FDA U.S. Food and Drug Administration. U.S. Department of Health and Human Services. Web. <http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/ucm208976.htm>.

6. Ball, M.D, Leslie K. "Johnson & Johnson Pharmaceutical Research & Development Warning Letter." Letter to Karen Grosser, Ph.D., M.B.A. 10 Aug. 2009. FDA U.S. Food and Drug Administration. U.S. Department of Health and Human Services. Web. <http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2009/ucm177398.htm>.

7. Ball, M.D, Leslie K. "ICON Clinical Research, Inc. Warning Letter." Letter to John W. Hubbard, Ph.D. 27 Nov. 2009. FDA U.S. Food and Drug Administration. U.S. Department of Health and Human Services. Web. <http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2009/ucm193156.htm>.

8. http://www.ctti-clinicaltrials.org/what-we-do/study-conduct/monitoring/products 9. Position Paper: Risk-Based Monitoring Methodology. TransCelerate Biopharma. 2013.http://www.transceleratebiopharmainc.com/wp-content/uploads/2013/10/TransCelerate-RBM-Position-Paper-FINAL-30MAY2013.pdf

10. Sheetz N, Wilson B, Benedict J, Huffman E, Lawton A, Travers M, Nadolny P, Young S, Given K, Florin L. Evaluating Source Data Verification as a Quality Control Measure in Clinical Trials Therapeutic Innovation and Regulatory Science.48: 6: 671-680. November 2014

11. Smith CT, Stocken DD, Dunn J, Cox T, Ghaneh P, Cunningham D, Neoptolemos JP. (2012) The Value of Source Data Verification in a Cancer Clinical Trial. PLoS ONE 7(12):e51623. Dol:10.1371/journal.pone.0051623.

12. Are your data cleaning cycle times out of control? Blog.mdsol.com. 13. Mitchel JT, Cho T, Gittleman DA, Markowitz JMS, Kim YJ, Choi J, Hamrell MR, Carrara D, Nora SD. Time to Change the Clinical Trial Monitoring Paradigm. Applied Clinical Trials January 17, 2014

14. Cerullo L, Radovich C, Ghandi, S, Wilder B, Stubbs C, Riley-Wagenmann C, McKellar R, Manasco PK, Competencies for the Changing Role of the Clinical Study Monitor: Implementing A Risk-Based Approach to Monitoring. April 2014. http://www.appliedclinicaltrialsonline.com/appliedclinicaltrials/article/articleDetail.jsp?id=842042&cid=actresourcearticle-competencies



15. Wilson B, Provencher T, Gough J, Clark S, Abdrachitov R, de Roeck K, Constantine S, Knepper D, Lawton A. Defining a Central Monitoring Capability: Sharing the Experience of TransCelerate BioPharma’s Approach, Part 1. Therapeutic Innovation and Regulatory Science 2014 48:529. August 2014RACT Toolkit http://www.transceleratebiopharmainc.com/rbm-resources/

16. Guidance for Industry: Part 11: Electronic Records;; Electronic Signature—Scope and Application U.S. Dept. HHS, U.S. FDA, August 2003 http://www.fda.gov/downloads/RegulatoryInformation/Guidances/ucm125125.pdf

17. Nahm M, Pieper CF, Cunningham MM. (2008) Quantifying Data Quality for Clinical Trials Using Electronic Data Capture. PLoS ONE 3(8): e3049 doi:10.1371/journal.pone.0003049.

18. Mitchel JT, Cho T, Gittleman DA, Markowitz JMS, Kim YJ, Choi J, Hamrell MR, Carrara D, Nora SD, Time to Change the Clinical Trial Monitoring Paradigm. Applied Clinical Trials January 17, 2014.

19. Riley-Wagenmann C, Manasco PM, Implementing a Standard Report Set for Risk Based Monitoring Domains, Poster Presentation, DIA June 2014.

20. Manasco PK, Electronic Investigator Site Files: The Hidden Gem that Completes Remote Risk Based Monitoring, Poster Presentation, DIA June 2014.

21. Guidance for Industry: Patient Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. US Department of HHS, U.S. FDA, December 2009. http://www.fda.gov/downloads/Drugs/Guidances/UCM193282.pdf

22. Barnes S, Katta N, Sanford N, Staigers T, Verish T, Technology Considerations to Enable the Risk-Based Monitoring Methodology. Therapeutic Innovation and Regulatory Science 2014, Vol 48(5) 536-545. August 2014

23. Use of Electronic Informed Consent in Clinical Investigations: Questions and Answers;; Guidance for Industry. US HHS, FDA, March 2015.

24. Manasco PK, Riley-Wagenmann C, Flack M. Remote Informed Consent Review: Results of Implementation in a Phase III Trial. Poster, DIA Annual Meeting June 2013.

Introduction:+ …manarbm.com/userfiles/955/files/Implementing-Risk-Based-Monitori… · Implementing*Risk.Based*Monitoring:** APrimer!

Documents

Introduction:+ …manarbm.com/userfiles/955/files/Implementing-Risk-Based-Monitori… · ImplementingRisk.BasedMonitoring:** APrimer!