A method for estimating the probability of adverse drug reactions

Reprinted from C L I N I C A L PHARMACOLOGY A N D T H E R A P E U T I C S . S t . Louis Vo l . 30. No. 2 , Pages 239-245, August . 1981 (Printed in the U . S . A . ) (Copyright 0 1981 by The C . V . Mosby Company)

A method for estimating the probability of adverse

drug reactions

Plrc, o.~ri~rrtrriorr 111. rlrc, ~,rohtrhi l in. rlrtrr tr tlr.rr,q c.crrr.sc~tl t r r r trth.c,rsc, 1 lirtic~trl c,~.orrr i.v rt.srroll\ h(r.cc~tl

orr c.lirric~rr1 /rtc,qrrrc,rrr. L~rc k of tr r r rc~r l~o t l f i~ r t~>rtrhli.slrirr,q I crct.\trlir\ ,qc2rrortrrc~.s Itrr.gc, / ~ c ~ r ~ ~ ~ c ~ c ~ r ~ - ~ ~ c r r ( ~ r . s

crrrrl ~~irlrirr-~.(rrc,r.s ~~t r r i t r l~ i l i r y irr trs.\r~.\.~rtcorrr. Lr.\irr,q rhc, ( . r ~ r i \ ~ t~ r~~ io r~c r I c.rtrc,,qor.ic..s trrrtl rlt:/irririorr.s of

r l t , f~~rir(,. / ) r oh r r / J /~ . ~)o.s.\ihlr, trrrtl ~/orrl)t/irl trtl\.or.\r, clr.rrg r.c~trc~riorr.s (A DR.\). rlrc, hcrn~oc~rr-r.trrt~r.c

cr,qr~c~c~t~rc~rrr I I ~ I I I , I I ~~lr\.sic~itr~r.s trrrt l / i) irr ~,I~trr.~r~trc~i.srs \~. l ro iricl(~/)c.rrtlc~rrr!\. trssc,.s.\otl 63 r.trrrtlorrr1~~

.\ol(,c.rt,tl trl/(,gc,tl A1)R.v \~ . t r . s 38% ro 63%. ktrpl~tr ( K , tr c ~ l ~ t r ~ r c ~ t ~ - ( ~ o r r c ~ r ~ r ( ~ ( l irrt10.v I!( tr,qr.t,c,r~~c,~rt)

~~trr~/c~tl,/ i .orrr 0 .2 I 11) 0 .40. trrrtl rlrr, irrrr.trc~ltrs.s ~~~~ r ro l t r r i o r r c.oc;ffific.ic,rrr I!( r.clitrhiliry (R[osr]) wtrs

0 .49 . Si.v (~r~.\r irrg) trrr(1 -72 11.k (rt,rcsrirrg) Itrrc,r rlrc, .\trriit~ ohsrjr-~.c,r.s irrtlt~pc~rrrlt~rrr!\. rc~trrrtr1~:t~tl rlic~

63 r.rr.\r,.\ /I! tr\.\i,qrrirr,q t r 11.t~ig11rotl .sc.orcJ (.-1DR ~ ~ r o l ~ r r l ) i l i r \ >c,(rlt,) ro c~trc.lr I!/ rlrt, c~orrrporrorrrs rlrtrr

rrrrr.\r / I(, r~or~.s i ( I (~r~r~r l 111 o.s~(/l~l i ,slr i~~,q (~trr(.s(rI cr.s,~o(.i tr~io~~> I~c,r~~~t,t,r i t/r.rr,q/> 1 trrr(1 (r(I\~c,r,cc, 01~1~11r.s ( ( , .~q . ,

r c , ~ ~ ~ l ~ ~ r ( / l srPclrr('rr~.c,j. 7'/i(, c.tr.sc,s ~~,c.rc, rzrrrtloriii:orl 111 ririrrirrri:r rho ir!j/rrorrc~c~ ( ! f ' I ( ~ r ~ r r r i r r ~ . T l i r cJ~~c,rtr

1 1 c1.s tr.s.si,qrrr,cl tr ~~r.ohtrl>i l iry c,trrc,,qoy fiorrr I / / ( , tortrl sc~~rc, . 7'110 I ~c~r~ t~c~c~r r - r t r r~~r~s rc l / i t rh i / ih (,rrrr$c':

~X, I '~ . ( ' I I I ~ , q r ( ~ c ~ ~ i r r ~ ~ i r = X.Zc'4 ro 92'%; K = 0.69 10 0 .86 : r = 0.91 10 0.05; R(c..sl) = 0.92) crtrtl

~~.irlrirr-t~trrt~r-.v r .c~ I io /~ i l i~ \ (rrrrrgo: I,r8rc.c,rrr tr~rt,c,r~rc,rrr = SOr/r ro 97'2: K = 0.64 l o 0.9-5; r = 0.91

ro 0.981 irtrpro\~c,tl (1) < O.oi)IJ. 7'111, h(,r\~.c,c,rr-~.trrc.rs ro l i r r1) i l i~. \\.ti> rrrtrirrrtrirrc~tl orr rc,rc~.srirr,q

(rtrrrgc8: r = 0.84 ro 0.94: R((,.\rj = 0.871. Tlrc~ hcr i~~c~t~r r - t~ t r r (~r .~ rc,litrhilir~ ( ? / ' rIrrc,o trrrc,rrclitrg

~)li\~.vic~itrri.s 11'1ro i t r t lo l~t~rr t l t~rrr l~ tr.\sc,.s.\c,cl 28 orlrc,r ~)ro.sl,c,c.ri~.c,!\. (.ollt,t.rc.tl c,tr.sc,s (I( trllo,qc.d .4DRs

11.tr.s \.or\. lriglr (rcrrrgc,: r = 0.76 ro 0.87; R(t,.sr) = O.XO). 11 \\.tr.v ctlso .slro~\~rr rlrtcr rlrr ADR

~>ro l x rh i l i h .sc,trlc 11tr.s c~~rrsc~rr.\rrtrl. c~orrrc~~rr, curt1 ~.~~rrc.rrrr-orrr ~ . t r l i ~ l i h . T1ri.s s\~.\rc,rrrtrric rrrc,rl~otl (![fim

t r .sc,rr.siri\.c, \\.ti! ro ~trotriror ADRs trrrrl r r r t r \ ho trl,plic~trhlc~ 10 ~ ~ u ~ r ~ r c r r k c ~ ~ i r r , q clrrr,q .srrr~~oi/ l t~rrt~o.

C. A. Naranjo, M.D., U. Busto, Pharm.D., E. M. Sellers, M.D., Ph.D., P. Sandor, M.D., I. Ruiz, Pharm.D.,* E. A. Roberts, M.D., E. Janecek, B.Sc. Phm., C. Domecq, Pharm.D.,* and D. J. Greenblatt, M.D.** Toronto. Orrrtrrio

Clirrir~rrl Plrtrrrrrcr~~olo,qy Pr~~,qrtrrtr. .-ltlclir~~iorr Rc.st,trrc,/r Forrrr(1trriori Clirric,crl Irr.srirrrrt~, r r r r t l

Dol)trr.rrirt,rrr.s of M(~clic,irrc, trrrtl Phtrr~rroc.olo,y~. Urri\.t~r..\ih ( ! / ' Tororrro

Received for publication Aprll 1.5. 19RO. The most important problem in assessing ad- A~.cepted for publication M X C ~ 20. 1981. verse drug reactions (ADRs) is whether there is Reprint requests to: C . A . Naranjo. M . D . Clinical Pharmacology

Program. Addiction Research Foundation Clinical Institute. 33 Rus- a causal relationship between the drug and the sell St.. Toronto. Ontario M5S 2 ~ 1 . Canada. untoward clinical event. The use of the con-

'Clinical Pharmacy Group. Faculty of Chemical Sciences, Uni- ventional definitions and of de- wrsidad de Chile. Sant~ago. Chile.

"'D~vision of Clinical Pharmacology. New England Medical finite. probable, possible, and doubtful ADRs;' Centre Hospital. Bosron. MA. generates wide variability in assessment. Koch-

0009-9236iRli080239+07$00.7010 0 1981 The C . V . Mosby C o . 239

Table I . .4DR p r o h ~ b i l i y scule II

7h trssess /\re trdverse tlrrrg t-twc,rior~, plecrse trrr.s;~.rr rhr fbllo~virrg qrtr.\riot~r~trirr ~rtrll ,yir.c rhc' perrrrirr~r score.

( Yrs I ,"\.') 1 11)) ,lor Xtlon 1 Score

I . Are there previous conc.lrisire reports on this reaction'!

2 . Did the adverse event appear after the uspected drug was adn~inistered!

3. Did the adverse reaction improve when the drug mas discontinued or a spec,~/ic ~lntagonist was administered?

1. Did the adverse reaction reappear when the tlrug was readministered!

5. Are there alternative causes (other than the drug) that could on their own have caused the reaction'!

6 . Did the reaction reappear when ;i placebo was given'! 7. Was the drug detected in the blood (or other Huids) In

concentrations known to be toxic'? 8. Was the reaction more severe when the close was in-

creased. or less severe when the dose was decreased'? 9. Did the patient have a similar reaction to the same or

similar drugs in ( ~ t i j . previous exposure? 10. Was the adverse event confirmed by any objective

evidence'? 'Total c o r e

Weser et al.' found that clinical pharmacolo- gists frequently disagreed when analyzing the causality of ADRs, and others1. ' have come to similar conclusions. Manifestations of ADRs are nonspecific. The suspected drug is usually confounded with other causes, and often the adverse clinical event cannot be distinguished from manifestations of the disease. Recently there have been attempts to systematize the assessment of causality of ADRs, applying operational definitions such as those proposed by Karch and Lasagna6 and by Kramer et al.!' The application of these methods in routine clinical practice has been limited, perhaps because they are too detailed and time consuming. We developed a simple method to assess the causality of ADRs in a variety of clinical situations, and its systematic application to different cases of alleged ADRs has provided reliable answers.

Materials a n d methods

To test the reliability and validity of the ADK probability scale (Table I ) several studies were conducted. In the main study, on three occa- sions (phases 1 , 2. and 3) six observers (two physicians and four pharmacists) independently

assessed 63 randomly selected alleged ADRs. These cases composed a stratified random sample ( 1 8 . 8 9 ) of 335 cases of ADRs published during 1978 in the U~Yrish hfr,rlic,trl Jo~rr - tltrl ( 2 2 cases), Ltr~lc,rr r 17 cases), A11r1~1ls of

I~irrrrrul Mc~tlici~ir ( 1 2 cases), .lorrr11~/1 of rlir . - I I N C ~ ~ L . L I I I M ~ d i c ~ r I . 1 ~ ~ o c . i ~ r i o r i ( 8 cases), and Nrlr) Engltrtitl Jorrrrlul (4' Mrtlir.ir~r ( 4 cases).* i The cases were randomized to minimize learn- ing. and the sequence was kept blind tv the observers.

In the first assessment (phase I ) an 'adverse drug reaction" (ADR) was defined as any nox- ious, unintended, and undesired effect of a drug after doses used in humans for prophylaxis, diagnosis, or therapy. This definition excludes therapeutic failures, intentional and accidental poisolling, and drug abuse. '" The probability that the adverse event was related to drug therapy was classified as definite. probable, possi-

i

ble, or doubtful.", I% '"delinite" reaction was j one that ( 1 ) followed a reasonable temporal sequence after a drug or in which a toxic drug level had been established in body fluids o r tis-

sues, (2) followed a recognized response to the suspected drug. and (3) was contirmed by improvement on withdrawing the drug and reap- peared on reexposure. A "probable" reaction ( 1 ) followed a reasonable temporal sequence after a drug. (2) followed a recognized response to the suspected drug, (3) was confirmed by withdrawal but not by exposure to the drug, and (4) could not be reasonably explained by the known characteristics of the patient's clinical state. A "possible" reaction (I) followed a temporal sequence after a drug. (2) possibly followed a recognized pattern to the suspected drug. and (3) could be explained by characteristics of the patient's disease. A reaction was defined as "doubtful" i f it was likely related to factors other than a drug.

Six weeks later the 63 c;laes were reordered randomly and reanalyzed (phase 2). The observers independently assigned a weighted score to the components used to establish a causal association betwecn drugs and adverse events (temporal sequence. pattern of response. withdrawal, reexposure, alternative causes, placebo re- $ponse, drug levels in body fluids or tissues, dose-response relationship. previous patient experience with the drug, and confirmation by ob-

1 jective evidence). These factors were analyzed and scored using the ADR probability scale (Table I ) . Each question could be answered positive (yes), negative (no). or unknown or inapplicable (do not know). The raters were in- structed to use the questionnaire for about 20 min.* The ADR was assigned to a probability category from the total score as follows: definite 2-9, probable 5 to 8, possible 1 to 4, doubtful 5 0 . The between-raters reliability to use the categorical classification of ADR probability was measured using percent agreement and kappa ( K , a chance-corrected inde .~ of agreement).'-' Kappa was calculated as follows:

P,, - P,. K = - I - P,.

where P , = proportion of observed agreement

*.4n a p p n d l x with instructions for using o u r ADR pmbab~lity scale will be supplied with reprints and will 3lw he available from the National A u x ~ l i a r y Publication Service. American Society of Inl;?rmation Services. 1010 16th St. N.W.. Washinelon, D.C. 20036.

and P, = proportion of agreement expected by chance. Kappa ranged from - 1 (complete disagreement) to + l (perfect agreement). Correla- tion coefficients between ADR scores were also used to test between-raters and within-raters reliability in phases 2 and 3. The intraclass correlation coefficient of reliability (R[est]) was also calculated:

where Sg = variance from the cases, S: =

variance generated by the raters, and S: = re- sidual variance or error. This coefficient is the ratio of the variance associated with true case- to-case variability to the sum of all the components of variance. R(est) varies from zero ( i .e . , no intercase variation is detected by the ratings, the ratings are the result only of measurement error and between-rater differences) to a maxi- mum of unity ( i .e . , intercase variation is cor- rectly detected by the ratings, there is no contam- ination by measurement error or rater-to-rater variation). I - ' The R(est) was calculated in phase 1 , assuming a score of 1 (doubtful), 2 (possible), 3 (probable), or 4 (definite). The actual ADR scores were used in phases 2 and 3 .

T o determine whether the improvement in reliability found in phase 2 had occurred by chance the cases were again reordered randomly and reanalyzed independently by the six raters 4 mo later (phase 3). This allowed us to assess within-rater and between-rater retest reliability. The between-rater reliability of practicing physicians was also tested. Three attending physicians independently rated 28 other prospectively collected cases of alleged ADR observed in the Toronto Western Hospital.

Validity. T o establish validity comparison with a standard is necessary. Because there is no method that can determine which adverse events are truly ADR, we studied the validity of the ADR probability scale in several ways. Consensual validity was tested as follows. ( 1 ) The consensus assessment of three "experts" ( C . A. N . , E. M. S . , D. J . G . ) using the conventional categories of ADR probabilities was the external standard with which physicians- pharmacists assessments were compared. Their expertise is supported by p ~ b l i c a t i o n s . ~ l o . l 2

Clin. Phurmocol. Ther Augrrsr I Y X l

Table 11. Between-rc~rers agreement

lntraclass correlation coefficient R(est) = 0.49 of reliability

Pairs of raters

Table 111. Within-raters agreement

Phuse 2

phase 3

Rater

(2) One of the experts (C. A. N.) assessed the reactions using the ADR probability scale, and his ratings were compared with those by the physicians-pharmacists in phase 2. Content validity was tested in the 63 reported cases and in the 28 prospectively collected cases, comparing the variations in the ADR scores of reactions considered possible, probable, or definite and those classified as definite nondrug adverse events. Concurrent validity was tested by comparing the correlation of the scores of the 63 ADRs obtained by our method with those derived by the algorithm described by Kramer et

Phase I

Results

%

Table I1 shows that there was poor between- raters agreement when the conventional definitions of ADRs were used (phase 1). Percent agreement ranged from 41% to 57% (kappa =

0.21 to 0.37, R(est) = 0.49). When the ob- %.

servers applied the ADR probability scale (phase 2) there was a rise in percent agreement (83% to 92%), K (0.69 to 0.86), and r (0.91 to 0.95) (sign test, p < 0.001). The intraclass correlation coefficient of reliability (R[est] = 0.92) indicates high reproducibility. The between- raters reliability was maintained on phase 3 retesting (r = 0.84 to 0.93, R(est) = 0.87). The high within-raters reliability using the ADR probability scale (phase 2 versus phase 3) is shown in Table 111. The percent agreement ranged from 80% to 97% ( K = 0.64 to 0.95, r = 0.91 to 0.98). The between-raters reliability of the three attending physicians who rated the 28 prospectively collected ADRs was also high (r = 0.76 to 0.87, R[est] = 0.80).

Validity. Percent agreement between the consensus of experts and the physicians-pharmacists assessments ranged from 7% to 84% (K = 0.64 to 0.71). Percent agreement with the expert (C. A. N.) who used the ADR probability scale ranged from 86% to 95% (K = 0.75 to

Phase 3

r K

Phase 2

% K r

Cases

Fig. I. Distribution of ADR scores in 63 cases of alleged ADRs.

0.91. r = 0.94 to 0.96). The ADR scores obtained rating the 63 reported cases with our method correlated with those derived using the algorithm described by Kramer et al.' (r =

0.82, p < 0.001).

Discussion

Our data indicated a marked improvement in between-raters and within-raters agreement when the adverse events were assessed with our ADR probability scale. The intraclass correlation coefficient of reliability (R[est] = 0.92) suggests that the method can discriminate ADRs of different probabilities. The reproducibility was maintained on retesting, and results of the same order were obtained when physicians rated a different set of prospectively collected cases of ADR. The ADR probability scale is a simple questionnaire that can be answered rapidly.

A major problem in drug-monitoring studies is lack of a reliable method of assessing the causal relation between drugs and adverse events. Such a method is needed because the incidence of adverse events can be estimated only from cases identified as definite or probable ADRs." Our data and those of others have demonstrated large interobserver variations in assessments when the conventional cate- goricaI definitions of probability of ADRs were used." '. Our ADR probability scale led to improved reproducibility in assessments. Using our scale, pairs of raters had scores that

were within the same diagnostic category or only one category apart. When there was disagreement it was usually not substantial, as indicated by the small standard deviation of the ADR scores (Fig. 1 ) and the high correlation coefficients between scores (Tables I1 and 111). A 3-point between-raters disagreement occurred in only one very complicated case.

It is possible that high reproducibility could occur without using the ADR probability scale, but the poor within-raters (phase 112 and phase 113; Table 111) and between-raters agreements (phase 1: Table 11) using the conventional definitions rule out this possibility. Perhaps the high agreement occurred because the 63 cases were selected from published reports and in- cluded only three ADR categories (possible, probable, and definite), which generated spuri- ously high reproducibility, but this seems un- likely. Fig. 1 shows that the cases represented a broad spectrum of ADRs (scores ranged from -2 to + 12). The good correlation between the actual ADR scores reflecting between-raters reliability (r = 0.91 to 0.95; Table 11) and within-raters reliability (r = 0.91 to 0.98; Table 111) and the high K values suggest that the ADR probability scale was the basis of a genuine improvement in reproducibility. When attending physicians used our method to rate a different set of reactions the between-raters agreement was good (R[est] = 0.80).

Using the ADR probability scale we were also able to identify the origin of the interrater

dis;ipreenlmts. The :r>s~:>alrlent (lf quebtion 5 (aitern:itive ~, ; iuszr) led to the most disagree- !,lent. In view of the cunlplex clir~ical situatiolls :~nd the differences in triiir~ing o f the observers, this should have been ;~nticipated. Pharmacists in general n.ere more likely to ansuer "I d o not k ~ ~ o w " to this question. Hutchinson et al. ' !clund that this c o ~ ~ l d he a ~ n i ~ j o r source of dis- aqeerrlent evcn though \.cry 8.letaileti instructions were given. In $oms complicated cases n o i~lgori th~n can snhstitute for clinical experience.

Evcn though the repruducibilitv of ;In instru- r r~er~t is ilnportant. its validity rnust also he con5itlered. The ohservers could agree among thelr~sclves, but they cnuld also all be wrong. In cases of ;ldverse events there is no definite standard again\t which to test the validity of new operational definitions of XDRs. We therefore assessed the validity of our AI>R probability sc;~le in several \bays. The agreement of the six raters mith the consensus o f three experts was \cry high, suggesting that our instrument has consensual validity. Although the experts may not always accurately classify reactions. the probability that the consensus of three experts would be corr~pletely wrong all the time is small. The high agreement between the physicians-pharmacists and one o f the experts using the RDR probability scale also indicates consensual validity. The concurrent validity of our instrument is suggested by the good correlation between the ADR scores generated by our method and those of another recently pl~blished a l g o r i t h m . T h e negative scores in the definite nondrug adverse events and the positive scores in the "true" ADR indicate that our method had content validity. Our tindings indicate that our hl1R probability scale is reliable and valid.

Important potential ;~pplications of the ADK probability scale are the analysis of adverse $drug-related events published in medical jour- nals as well as the assessment of reports submit- ted to national tlrllg monitoring centers. Many countries ore interested in developi~ig postmar- keting drug surveilla~ice programs.'' The reliability of the ADR assessments in case studies could improve if operational delinitions such as ours and similar procedures are used." Advan- tages of our method are simplicity and wide applilvability. Sonie minor modifications rriay be

required in special circumstances. In analyzing adverse drug interactions suspect interacting 4 drugs rather than a particular drug must be assessed. When a patient receives several drugs at the same time the ADR scale must be applied to each of the possible causes: the most likely will he the drug with the highest score. In reactions that appear during drug withdrawal. withdrawal corresponds to reinstituting treatment and rep- etition corresponds to disccntinuing the suspect drug.

The conventional classilicntion of definite. probable, possible, and doubtful ADRs. as proposed by Seidl et al."' in 1966, assumes four discrete categories For which there is no empiri- cal demonstration, It is therefore reasonable to postulate that some of the unreliability of the conventional detinitions or operational detinitions of ADRs could result. because such categories are not unique ( i . e . , the unreliability could reflect the overlap between nondiscrete categories). Thus the higher correlation co- cfticients of the actual ADR scores (r = 0.91 to 0.94), as compared with the kappa values when using four categories ( K = 0.69 to 0.83) (Table 11). support this view and indic,ate the need to characterize the probability spectrum of ADRs empirically. We suggest that it is preferable 1 ,: !

to classify the prohability using the actuaI ADR scores by our and \imilar operational methods.

Notwithstanding our encouraging results, it is unrealistic to expect that our relatively simple procedure will solve all the complex problems of identification and classification of ADRs. Further experience will provide the rationale for refinements and improvements and will confirm its utility in clinical practice. Our findings suggest that its systematic application can improve the quality of the assessment of ADRs in a variety of clinical situations.

The collaboration of Doctors M. Spino. H . Wang. and M . Rudyk and of S. Schachter. B .Sc.. in some of the phases of the study i s grnteti~lly itcknowledged.

Rufirur~cua I . Blanc S, Leuenberger P. Berger JP. Brooke EM.

Schelliny JL: Judgments of trained observers on adverse drug reaction<. C1.i~ PH A R M ~ C O L THER 25:493-498. 1979.

1. B u t o I!. ?I :~r~ ln jo C"4. Sellera ELI: i ~ ~ i ~ p a r r : ~ ~ ) ~ ~ 1 o l two recen~ly published .ilgorithr~ls to .iare,s

the prcibabrlit). a d ~ c r s e drug rcai;:tloils. C I IS

P~~.-\KzI-\~oI ' ~ ~ I E R 20:2.3h. I'jR1 3 . Gross F t i , Inrnan WtiW: Dl-11s i n ~ c i n ~ t o ~ - i i ~ g . ?.I?:+-

YOI-k. 1077. .\c;idcn~~i;: Press. 1. Flut~~hinst111 T.A. l.e\.cnthal J h l . K~,ur~i;:r >IS.

K;~ri;:h t.E. I.rplnan .A(;. F;c'in\tc.in .AR: .An al- ~ o r i t h m lo r rhe ~ ~ p e l a t i o n a l d>sesblllent o i .IJ- \else tlr-ug ~cacriona. 11. I)crnt)nsrr;irion of 1.2-

pr~ ldt lc ih i l i ty JII~ \ a l i J i t y . JX?rlA 242:hi i h3X. 1979.

5 . Karch FE. L.,I\,I~II~ 1-: :\d\c~.,e drug rt.ii~.tlona: 1 crit ical revle\c. J A b I A 2.74: 1236- 12-11. 1075.

0 . Karch FE. Ldbagnlr L : 'TowarJ rhc' opc.rario~~,rl i de r~~ i r i ca t ion o i ad~c ' rbe (Ir~rg re;~ctions. Ci 1%

P I ~ ~ K M - \ ~ ~ I ~ H L - R 21:217-25-1, 1 ~ 7 7 . 7 . Karsh FE. Smith C'L. l i e r rner B. h l u ~ ~ u l l o .iM.

Weintraub 51. Laaagrra L: ,Advcrsc d n r IeaL,- ~i~,ns: A matter OF opinron. C'I..IN PIIAK~IA~ OL

T t l t . ~ 19:1X9-492. 1076. X . Koch-\Veser J. Sellers EM. Z~i;:e\t R: 'The a111-

ttiguity o l a d ~ c r s e ~ I - L I ~ rcacrionr. L!~.rr J C l l n Pharnlacol 11:7.5-78, 1977.

9 . Krluuer h I S . I.t.\;el~thal JIM, Hutchinson 'T.1. Feinsrein AR: : i n a lgor i thm for the operat io l~al ;cssc.asrnent u t adbsrse drug reactions. I . Hack- ground, drhcript ion. and instrui;:tions for LI~S. JAMA 2-42:623-631, 19.79.

i l l , * l i l l c ~ [ < I < . <~~ICL.II~;I:II~ 1 ) ) . I . j tug CI~ 'CC[ \ III I I%I \~J~- rdl irct l p d ~ c n t \ . \etv 'L'nrk. 1976. lohn \i'ilt?)'S s011r.

i I . ? q , ~ ~ . d ~ ~ l o C 1. [~II\IO I . ..',b~l .I(;. S c l l ~ r s 1131: t<mp i r r i ~ le l i~ rea t ion ,)t the probcthrlity \l,c.ctrllnl 111' < i d \ c r ~ c t1r11g I-~~IL.IIOI~~. CI ib t ' t i - \u\ l~c 01.

~ ~ I I I R 2 ~ : ~ 0 7 ~ ~ ~ ~ . lost 12 . ?,~ii~..~nlo C \. F'OIIII~O E. \ d1dcncg1-o ( '. &)n-

i * ~ I c f G. KIIIL I. BLI~II) F~~ r , ) sc~ l i i de - i r~~ I~~~ . : c i ; ~ i l ~ ~ c ~ s c iec1L~it'nh i n ~ i r ~ t i t \ a r r , ) t ' [he l i ~ . c r . L'I IS

I'II.\R!,IA\ a IL l ' i i t . ~ 25: 134- 100. ! c l ~ ' l . 13. SLILII 1 .G T'11o1.11ton (;F:. S1111th .It+'. ( l u f f 1 E;

Stuilieh on [he ep ic l c~n i~ l losy o i .icl\cr\c J ~ L I ~ ~ i ~ t i i . lil. Kc' i rc f ion~ i r ~ pdrienrs ttn a gcller:kl r ~ ~ c d i ~ . , i l ,er\.r~,c. IOIIIIS l i o ~ h i ~ ~ s 5,lcd 1 I 19:2'iLJ- 315. 19h6.

1-1. Spirter R l . . t . 1 ~ 1 5 ~ J L . E r ~ ~ i i c ~ r ~ t I: f'~.ciblt.ms r l t '

, las>~licar~urr: Re l iab i l~ ty and , l l i c l i t > . !,r l . i l)[un %lA, LIihla>cro A. K i l l i ~ r l ~ DF. etlitorh: P\y- L ~ I I L I ~ > I ~ ~ I I ~ I ~ L o I ~ I ~ ~ ; '1 g?n~r.dt ion or progress. Nc:w YOI-A. 1978. K i i \ c n Prcas. p l , 657-869.

l i . Wardel l \Vhl. Tatar~~.o \IC. .1na\.ckar SPJ. ()a\ is t I T P~)srrnarkcr~l lg >11r\ eillancs u i new dr.ug5. \I. Care \(ci<lles. J C'lin Phdrl~~ai;:ol lY:16Y-1x4. 1979.

! 6. Wor l t l He:llrh Organiration; I ~ i t c i ~ ~ i a t i o r ~ ; ~ I dr119 111onitcrring: 'rile rc~lc. of tire hoaoital. LL 110 Tech Rep Scr No. 125. 1Y69

Naranjo CA, Busto U, S e l l e r s EM, e t al : A method f o r e s t ima t ing t h e p r o b a b i l i t y o f adverse drug r eac t ions . Cl in Pharmacol Ther 30: 239-245, 1981.

L APPENDIX I Instructions for using the ADR probability scale.

A. General Instructions

Collect all t he relevant information about the alleged ADR, before using the

questionnaire. Please, read each question carefully. If you do not understand a

particular question, consult the specific instructions given below. Use the "do not

knoww answer only when the quality of the data does not allow an affirmative (yes)

or negative (no) answer. Similarly, you should answer "do not know" if the

information is unavailable or the question is inapplicable. Some minor modifications

a r e required in certain circumstances. For analyzing adverse drug interactions, the

suspected interacting drugs, instead of a particular drug must be assessed. When the

patient is receiving several drugs, the ADR probability scale must be applied t o L

every one of the suspected etiological agents. The presumed etiology will be the

drug with the highest score. In reactions which appear during drug withdrawal,

dechallenge (Question 3) refers t o reinstituting treatment with the suspected drug

whereas rechallenge (Question 4) consists of discontinuing the alleged drug.

Similarly, question 2 must be reworded: Did the adverse e f fec t appear af ter the

suspected drug was discontinued? ( See ADR P r o b a b i l i t y Sca l e : Drug Withdrawal ) .

0. Specific Instructions

Question 1: It assesses the previous general experience with drug use. You should

answer "yes" only if there a r e two or more published reports which

have previously described in detail the adverse clinical event (ACE).

Similarly, you can accept as a reliable source of information the

listing of the ACE in a classic textbook on adverse reactions such as

Meyler's Side Effects of Drugs, Vol. 9, MNG Dukes, Amsterdam,

Excerpta Medica, 1979. You should answer "no" when the ACE has

not been previously described. The answer "do not know" is only

applicable when the information is either unavailable or inconclusive.

Question 2: It assesses the t ime sequence of the reaction. The answer must be

"yes" only if there is definite evidence tha t the ACE occurred a f te r

the suspected drug was administered. "No" is applicable when the

ACE developed prior t o drug administration. You must answer "do

not knoww if the information available does not allow a certain

decision.

Question 3: It assesses the effect of dechallenge. The answer must be "yesw only

if the ACE diminish or disappear at any t ime af te r discontinuation of

the suspected drug. Similarly, you can answer "yes" if the reaction

disappear a f te r a specific pharmacologic antagonist was administered

(e.g. an anticholinergic drug, such as atropine, antagonized the effect

of a chalinergic drug such as physostgmine). You must answer "no" if

the reaction did not disappear or i t did not diminish a f te r drug

discontinuation. Similarly, you must answer "no" if the ACE

improved af ter a non-specific pharmacologic antagonist was admin-

istered (e.g. potassium salts in a diuretic-induced hypokalemia). You

must answer "do not know" in the following circumstances: the

suspected drug was not discontinued, the information is unavailable

or the results of dechallenge a re inconclusive. If t he ACE is

transitory (e.g. convulsions), and the clinical circumstances do not

allow an unequivocal assessment of the effect of dechallenge, the

answer must be "do not know".

Question 4: I t assesses t h e e f f e c t of re-challenge. You must answer llyesll only if

t h e following conditions were met: a) t h e suspected drug was

discontinued, b) a new baseline of t h e ACE was determined, c ) t h e r e

was a n unequivocal reappearance when t h e drug was re-administered

in similar doses and by identical route. Similarly, a positive answer

can be given in those cases in which t h e other factors suggest a

strong causal association and t h e ACE is a well known e f f e c t of a

drug (e.g. furosemide-induced hypokalemia) and for clinical and/or

e thical reasons t h e re-challenge is unwise. You must answer "now

when one o r a l l of t h e conditions of re-challenge listed in a, b and c

were not met. You must answer "do not know" if t h e information is

ei ther unavailable o r equivocal.

Question 5: It assesses alternative etiologic candidates of t h e ACE. Since usually

ADR are non-specific manifestations of disease you must carefully

consider t h e eventual contribution of etiologies o ther than t h e

suspected drug. You must answer "no" only if a f t e r a systemat ic and

exhaustive investigation no a l t e rna te causes were detected, thus

implying t h e suspected drug more strongly. You must bear in mind

t h a t a contributing fac to r t o t h e ACE does not qualify as al ternat ive

etiologic candidate (e.g. If a patient with prior renal fai lure develop

a n impairment of renal function secondary t o therapy with genta-

mycin, t h e prior renal failure is a contributing fac to r not an

a l ternat ive etiology of t h e ADR). You must answer "yes" if the re is

an alternative etiologic candidate. You must answer "do not know" if

t h e investigation of other causes was incomplete, inconclusive, o r not

performed at all.

Question 6: It is important when the ACE is subjective. The answer must be L

l1yesl1 only if t he following conditions were met: a) the suspected

drug was discontinued, b) a new baseline of the clinical manifestation

was established, and c) there was unequivocal re-appearance of t he

ACE af te r a single or double-blind placebo administration. The

answer must be "no" if re-administration of placebo, as described

above, did not elicit the same reaction. You must answer "do not

knowv1 when a placebo test was not performed, the information is

unavailable, or t he placebo e f fec t is inconclusive.

Question 7: I t is very important in cases of dose-related ADRs, and i t assesses

the information provided by drug concentrations in blood, other body

fluids or tissues. The answer must be llyesll only if t he concentration

of the alleged drug is in the accepted toxic range. The answer must

be l1nol1 if drug concentration is below the toxic range; and the answer

must be "do not knowt1 if this information is either unavailable or t he

question is inapplicable (i.e. in ADRs which a r e not dose-related).

Question 8: I t assesses clinically the dose-response relationship. You must

answer only if t he ACE was more severe when the dose of the

alleged drug was increased. Similarly, you must answer l1yesl1 if the

ACE was less severe a f te r reduction in the dose of the alleged drug.

The answer must be llnoll if there was no clinically detectable

variation in t he ACE when the dose of the suspected drug was

changed. You must answer "do not know" when either the dose did

not change or t he information is unavailable.

Question 9: It assesses the personal experience of t he patient in previous

administrations of t h e same or chemically related drugs. You must

answer "yestt only if t h e patient report a similar ACE t o t h e drug(s).

Similarly, you must answer "yesv if there is documentation in t h e

patient's record of such prior reaction (e.g. documentation of prior

manifestations of allergy t o penicillin such as skin rash or anaphyl-

axis.) You must answer "nott when either t h e patient has no previous

exposure t o t h e drug or if in a previous exposure t o t h e same or

similar drugs t h e patient did not develop t he ACE. The answer must

be "do not knowN when either t h e information is unavailable o r it is

inconclusive.

Question 10: It assesses the quality of t he information in which we a r e supporting

our judgments. This is crucial because even though the patient's

report is a very important source of information, some patients can

be unreliable, for instance, because of memory impairment. You

must answer "yes" only in t he following circumstances: a) a

laboratory test documented t h e ADR (e.g. in t h e case of a suspected

drug-induced hepatitis, a compatible liver biopsy). b) t h e ACE was

directly observed by qualified personnel (e.g. t he attending physicians

or t h e nurses observed manifestations such as the skin rash, jaundice,

etc). The answer must be "no" when either t he laboratory tests or

t h e direct clinical observation did not document the reaction. The

answer must be "do not know" when t he ACE is subjective, t h e

information is unavailable or i t is inconclusive.

Once you have answered all t h e questions, you must add t he individuals scores. The

probability of t h e ADR i s given by t h e tota l score.

ADR PROBABILITY SCALE : DRUG IJITHDRAWAL

L To assess adverse drug reac t ions which appear during "drug withdrawal", please answer the ~_ollowing questionnaire and give the per t inent score:

... -..

YES NO DON'T KNOW SCORE

1. Are t he r e previous conclusive r epo r t s on t h i s react ion?

2. Did t he adverse event +2 - 1 appear a f t e r t h e suspected drug was withdram?

3. Did t h e adverse react ion improve when t he drug vas readministered?

4. Did the adverse event reappear when t he drug was withdrawn again?

- 5 . Are the re a l t e r n a t i v e

L causes (other than the drug) t h a t could on t h e i r own have caused t he react ion?

6. Did t h e adverse event appear a f t e r placebo withdrawal?

7. Did t he pa t i en t previously +1 use t h e drug chronically?

8. lias t h e r e a t t i o n l e s s +1 severe whcn the dose was increased o r more severe when t he dose was decreased?

9. Did the pa t i en t have a +1 .- 0 react ion t o the same o r s imi la r drug i n any previous exposure?

10. tias t h e adverse event +1 0 0 confirmed by any ob jec t ive evidence?

L

11 . TOTAL SCORE

Table 1. - ADR Probability Scale

To assess t h e adverse drug reaction, please answer t h e following questionnaire and give t h e pert inent score:

Yes No Do Not Know Score

1. Are t h e r e previous conclu- + 1 0 0 sive reports on th is reaction? -

2. Did t h e adverse even t +2 - 1 0 appear a f t e r t h e suspected drug was administered?

3. Did t h e adverse reaction + 1 0 improve when t h e drug was discontinued, o r an specific antagonist was administered?

4. Did t h e adverse reaction +2 - 1 0 reappear when t h e drug was re-administered? .

5. Are t h e r e a l ternat ive -1 +2 0 causes (other than t h e drug) t h a t could on thei r d

own have caused t h e reaction? f

6. Did t h e reaction reappear - 1 + 1 0 when a placebo was given?

7. Was t h e drug de tec ted +1 0 in t h e blood (or o the r fluids) in concentrations known t o be toxic?

-

8. Was t h e reaction more +I 0 severe when t h e dose was increased, o r less severe when t h e dose was decreased?

9. Did t h e pat ient have a +I 0 similar reaction t o t h e s a m e o r similar drugs in any previous exposure?

10. W a s t h e adverse even t + 1 0 confirmed by any objective evidence?

11. TOTAL SCORE d

A method for estimating the probability of adverse drug reactions

Documents