A Quality Indicator Set for Systemic Lupus Erythematosus
Jinoos Yazdany, MD, MPH1, Pantelis Panopalis, MD1, Joann Zell Gillis, MD2, GabrielaSchmajuk, MD3, Catherine MacLean, MD, PhD4, David Wofsy, MD1, Edward Yelin, PhD1,and SLE Quality Indicators Project Expert Panels*
1University of California, San Francisco, Division of Rheumatology2National Jewish Hospital, Division of Rheumatology3Stanford University, Division of Rheumatology4David Geffen School of Medicine at the University of California, Los Angeles, Division ofRheumatology
AbstractObjective—To systematically develop a quality indicator (QI) set for systemic lupuserythematosus (SLE).
Methods—We used a validated process that combined available scientific evidence and expertconsensus to develop a QI set for SLE. First, we extracted 20 candidate indicators from asystematic literature review of clinical practice guidelines pertaining to SLE. An advisory panelrevised and augmented these candidate indicators, and through two rounds of voting, arrived at 25QIs. These QIs advanced to the next phase of the project, in which we employed a modification ofthe RAND/UCLA Appropriateness Method. A systematic review of the literature was performedfor each QI, linking the proposed process of care to potential improved health outcomes. Afterreviewing this scientific evidence, a second interdisciplinary expert panel convened to discuss theevidence and provide final ratings on the validity and feasibility of each QI.
Results—The final expert panel rated 20 QIs as both valid and feasible. Areas covered includediagnosis, general preventive strategies (e.g. vaccinations, sun avoidance counseling, screening forcardiovascular disease), osteoporosis prevention and treatment, drug toxicity monitoring, renaldisease, and reproductive health.
Conclusions—We employed a rigorous multi-step approach with systematic literature reviewsand two expert panels to develop QIs for SLE. This new set of indicators provides an opportunityto assess health care quality in SLE, and represents an initial step toward the important goal ofimproving care in this patient population.
Long-term survival of patients with SLE has improved greatly over the last several decades,but morbidity from the disease and complications of medical therapy remain importantconcerns. Although many studies have explored risk factors associated with poor outcomesin SLE,1-4 few studies have investigated the quality of health care received by patients withthis condition.5, 6 One important barrier to such research is the lack of consensus on healthcare processes constituting high quality care in SLE. We aimed to address this gap bydeveloping a quality indicator (QI) set for SLE.
Correspondence: Jinoos Yazdany, M.D., M.P.H., Rosalind Russell Medical Research Center for Arthritis, UCSF Box 0920, SanFrancisco, California 94143-0920, Phone 415-476-0622, Fax 415-476-9030, [email protected].*Members of the SLE Quality Indicators Project Expert Panels included: Eliza Chakravarty, MD, MSc., Maria Dall'Era, MD, JamesDavis, MD, John C. Davis, Jr., MD, MPH, Rashmi B. Dixit, MD, PhD, Richard Furie, MD, Jennifer Grossman, MD, KennethKalunian, MD, Lester Miller, MD, Rosalind Ramsey-Goldman, MD, DrPh, Brad Rovin, MD, FACP, FASN, Kenneth Saag, MD, NinaD. Schwartz, MD, Jorge Sanchez-Guerrero, MD, MS, Daniel J. Wallace, MD, FACP, FACR, Michael M. Ward, MD.
NIH Public AccessAuthor ManuscriptArthritis Rheum. Author manuscript; available in PMC 2010 March 15.
Published in final edited form as:Arthritis Rheum. 2009 March 15; 61(3): 370–377. doi:10.1002/art.24356.
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
Over the last decade, research focusing on quality measurement in health care hasburgeoned, partly in response to the initiative launched by the Institute of Medicine [IOM]in 1996 to assess and improve health care quality. The IOM defined quality as “the degree towhich health services for individuals and populations increase the likelihood of desiredhealth outcomes and are consistent with current professional knowledge.”7 In the UnitedStates, the most commonly used tools to measure quality have taken the form of QIs,defined as “retrospectively measurable elements of practice performance for which there isevidence or consensus that can be used to assess the quality of care provided and hencechange it”.8 Unlike clinical guidelines or recommendations that often aim to define optimalhealth care practices in the context of complex clinical decision making, QIs are meant torepresent a minimally acceptable standard of care across a specific patient population.9
Indicators that assess health care quality have traditionally been grouped into three relatedcategories, including structural measures (e.g. innate characteristics of providers and thesystem), process measures (e.g. what health care providers do in delivering care), oroutcome measures (e.g. what happens to patients, particularly with respect to their health).10, 11 For many chronic diseases, quality assessment has focused primarily on processrather than outcome measures for a number of reasons, including that health outcomes oftenrequire years to develop and their measurement may therefore delay timely interventions,and that there remains limited consensus on the correct outcome measures to assess formany conditions. Likely as a result of these limitations, QI sets pertaining to severalprevalent rheumatic diseases have also focused on process measures. A multidisciplinarypanel for the Arthritis Foundation Quality Indicator Project developed process indicators toassess quality for rheumatoid arthritis, osteoarthritis, and analgesic use,9 and indicators havealso been developed for gout.12 For this project, we relied solely on indicators measuringprocesses of care as well.
Although the prevalence of SLE is lower than that of conditions previously targeted for QIdevelopment, quality assessment may be especially desirable in SLE. First, as a multi-organdisease with varied manifestations, SLE often requires care by multiple providers across arange of health care settings. This fragmented structure of care may contribute todeficiencies in quality. Second, despite improved therapies, the relative burden of morbidityand mortality from SLE remains high, and a majority of patients accumulate significantorgan damage and disability over time.13-15 The need to improve patient outcomes in SLEtherefore remains an important priority, and quality measurement and enhancement effortshave potential to contribute toward this goal. Finally, significant disparities in healthoutcomes exist in SLE, with racial/ethnic minorities and those with low socioeconomicstatus demonstrating consistently poorer outcomes over time.16 The relative contribution ofprocesses of care to the poorer outcomes in these vulnerable populations remains uncertain,and until now, the lack of validated process measures has encumbered efforts to furtherinvestigate these relationships.
We present here the methods used to develop process indicators for SLE. Like otherinvestigators who have previously developed QIs for general medical conditions andrheumatic diseases, we adhered to a validated method of combining scientific evidence fromthe literature and expert consensus to create indicators for SLE.
MethodsFigure 1 summarizes the methods used to develop the SLE QI set. The general approach,developed at the RAND Corporation based on several decades of research,17 has been usedextensively in the literature to develop QIs for a variety of health care practices andconditions.
Yazdany et al. Page 2
Arthritis Rheum. Author manuscript; available in PMC 2010 March 15.
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
Recognizing that SLE is among the most complex conditions targeted for QI development todate, we decided a priori to limit the scope of our work to several general topics. Theseincluded diagnosis, general preventive care practices, and areas of disease-specificmanagement with either high prevalence or high morbidity (i.e. osteoporosis, renal disease,cardiovascular disease, and pregnancy and reproductive health).
Phase 1: Devising preliminary QIsTo devise a preliminary list of QIs, we performed a systematic review of the literature toretrieve all recommendations and guidelines relevant to SLE. We used a broad strategy,including searches in the MEDLINE, EMBASE, and Cochrane databases, as well asinternet-based searches of medical societies and national organizations doing health carequality work (e.g. the National Quality Measures Clearinghouse, the National KidneyFoundation, the National Guidelines Clearinghouse). We excluded editorials or reviews,including only efforts that had used a consensus-based approach relying on scientificevidence. From this search, we drafted 20 preliminary QIs, each of which had to represent atopic area of interest, derive support either from scientific evidence or professionalconsensus, and deal with processes that are under the influence of health care providers.
We devised the preliminary QIs in an IF-THEN-BECAUSE format, where the IF componentdefined the eligible population, the THEN component explicitly defined the process of careto be performed by health care providers, and the BECAUSE component summarized thehealth benefits accrued by the patient.9 Our research team performed preliminary literaturereviews using the MEDLINE database to define how the processes of care in each QI werelinked to patient outcomes. This information was compiled in a document and formed thescientific basis for the SLE Quality Indicators Project advisory panel meeting in October2007.
Phase 2: SLE QI Advisory Panel MeetingThe project advisors identified 8 individuals to serve on an advisory panel, drawing fromspecialists in SLE research and patient care, health services researchers, and rheumatologistspracticing in non-academic settings (Table 1). During a one-day meeting, the advisory paneldeleted two QIs secondary to lack of scientific evidence; nine QIs underwent minorrevisions, such as changes in wording or clarification, and 9 QIs underwent major revisions,such as adding or subtracting content or separation into multiple QIs. Using a modificationof the nominal group technique, the panel also added 9 additional QIs. In total, 29 QIsresulted from the meeting.
The revised QI set was then reviewed by additional outside consultants, including experts inSLE and quality measurement. Further adjustments to wording were made, and the revisedQI set was then re-presented to the advisory panel for two rounds of electronic voting.Panelists rated the validity of each indicator using a 9-point scale (1 representing “invalid”,and 9 representing “definitely valid”). Twenty-five indicators had a high mean validityrating (7-9), and progressed to the next phase of the project.
Phase 3: Systematic Literature ReviewsOur research team (J.Y., P.P., J.Z.G., G.S.) then undertook systematic literature reviews tosummarize the scientific evidence linking the advocated processes of care to patientoutcomes. We largely adhered to methods outlined in the Cochrane Collaborative WorkingGroup on Systematic Reviews,18 except that we used a single reviewer to assess studies formost QIs. In cases where questions arose, a second reviewer also assessed studies.Professional librarians set up formal searches at our respective institutions in the MEDLINEand EMBASE databases.
Yazdany et al. Page 3
Arthritis Rheum. Author manuscript; available in PMC 2010 March 15.
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
The search results were reviewed in several steps. First, we screened retrieved titles. Next,abstracts of relevant titles were reviewed in detail, and full-length manuscripts wereobtained for studies with potential information linking the process of care advocated in theQI to patient outcomes. Non-English articles, and those not reporting original research wereexcluded. Finally, we hand-searched the references of retrieved studies as well as reviewarticles for completeness. In total, our research team examined over 6000 search results inMEDLINE and over 9000 search results in EMBASE.
We summarized these literature reviews in a monograph. Each QI review had the followingcomponents: 1) a summary of previous consensus-based guidelines or recommendations onthe topic; 2) details of the systematic review strategy and results; 3) specification of the typeof evidence (meta-analysis, randomized controlled trial, observational studies, or other); 4) asummary of the direct evidence (i.e. where adequate clinical trial or observational datalinked the process of care to health outcomes in SLE specifically), and indirect evidence(e.g. summaries of professional consensus statements, data from other chronic diseases, orcircumstantial evidence in SLE). Where evidence was lacking, this was also clearly stated.The resulting 260-page monograph served as the scientific basis for the next phase of theproject.
Phase 4: The RAND/UCLA Appropriateness Method PanelFor the final phase of the project, we employed a modification of the RAND/UCLAAppropriateness Method, a technique with content, construct and predictive validity, togenerate a final indicator set for SLE.17, 19-22 This method has previously been describedas the best systematic method of combining expert opinion and evidence.23 In brief, thisapproach entails two rounds of anonymous ratings on a standardized scale by an expertpanel. The second round of ratings takes place after a face-to-face discussion amongpanelists. Research has demonstrated that the reproducibility of this method is consistentwith that of common diagnostic tests, such as the interpretation of mammograms.22Adaptation of this method to the SLE QI project is discussed in detail below.
The project advisors assisted in identifying 9 physicians to sit on the final expert panel; 5members were rheumatologists with special expertise in SLE, 2 members wererheumatologists in private practice, 1 member was a nephrologist, and 1 member was arheumatologist/health services researcher with experience in QI development. Prior to theface-to-face meeting in May 2008, we mailed the monograph summarizing the scientificevidence to panelists and asked them to rate the feasibility and validity of each QI using aweb-based survey tool.
Panelists rated each potential QI on two 1-9 scales, one for validity and one for feasibility (1representing “not valid” or “not feasible, and 9 representing “definitely valid” or “definitelyfeasible”). For validity, panelists were instructed to consider the following questions: 1) Isthere adequate scientific evidence or professional consensus to support the indicator? 2) Arethere identifiable health benefits to patients who receive care specified by the indicator? 3)Based on your professional experience, would you consider physicians with significantlyhigher rates of adherence to the indicator higher quality providers? 4) Are the majority offactors that determine adherence to the indicator under the control of the physician (or arethey subject to influence by the physician)? For feasibility, panelists were asked to considerthe following questions: 1) Is the information necessary to determine adherence possible tofind in an average medical record (or is failure to document such information itself a markerof poor quality)? 2) Is the estimate of adherence to the indicator based on medical recorddata likely to be reliable and unbiased?24
Yazdany et al. Page 4
Arthritis Rheum. Author manuscript; available in PMC 2010 March 15.
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
Results of these preliminary ratings were compiled and sent to panelists approximately oneweek before the meeting, allowing them to compare their responses to their colleagues.During the meeting, each panelist again received an anonymous summary of the rankings ofother members of the group, as well as their own ranking. The meeting moderator used thesedata to guide discussion, focusing on areas with greatest disagreement. No attempt wasmade to force the panel to consensus; instead, the discussion attempted to determine whetherdivergent ratings resulted from real clinical disagreement, or simply reflected differentunderstandings of the indicators that might require clarification. After several minorrevisions, panelists anonymously re-rated the validity and feasibility of each item using thesame scale described above.
Data AnalysisOur analytic plan adhered to methods previously used by other investigators at the RANDCorporation.9, 17 Indicators with a mean validity rating of ≥7.0 and without disagreementwere considered valid. Disagreement was defined as two or more panelists rating theindicator in the highest tertile (rating of 7, 8 or 9), and two in the lowest tertile (rating of 1,2, or 3). Indicators with a mean feasibility rating ≥4.0 were considered feasible. Onlyindicators rated as both valid and feasible by panelists are included in the final QI set.24 Asummary of the criteria for indicator inclusion in the final SLE QI set is provided in Table 2.
ResultsThe final QI ratings are listed in Tables 3. Twenty-five candidate QIs were presented to theexpert panel. In some cases, minor revisions or clarification of wording were made prior tovoting, although all content areas remained constant. For the pharmacologic therapymonitoring QIs listed in Table 4, the validity and feasibility of each drug were ratedseparately, and in some areas, where disagreement was present in the pre-meeting ratings,on the individual laboratory tests or procedures suggested for each drug.
Panelists rated 20, or 80%, of the statements as valid and feasible indicators of quality inSLE. Five statements had either a mean validity rating <7 or disagreement among panelistsand were excluded from the final set; no QIs were excluded based on low feasibility alone(for deleted QIs, see Appendix). As outlined in Table 3, the final set includes QIs related todiagnosis (2), general preventive strategies, including sun protection counseling andvaccinations (3), osteoporosis (3), drug monitoring (4), renal disease (4), prevention ofcardiovascular disease (1), and pregnancy and reproductive health (3).
DiscussionUsing a validated approach that relies on a combination of scientific evidence and expertconsensus, we have developed a QI set for SLE. The 20 QIs cover several important aspectsof SLE care including diagnosis, general preventive strategies (e.g. vaccinations, sunavoidance counseling), osteoporosis prevention and treatment, screening for cardiovasculardisease, drug toxicity monitoring, renal disease, and reproductive health. The QIs provide aninitial tool for assessing health care quality in SLE, a condition in which serious health caredisparities among demographic and socioeconomic groups have been reported.16
The potential impact of applying SLE QIs to practice will depend partly on their technicalcharacteristics (face/content validity, reproducibility, acceptability, feasibility, reliability,sensitivity to change, predictive validity).8 Performing the requisite basic methodologicalresearch to define these properties is an important step in validating the QI set. Thereafter,the impact will depend largely on uptake of the indicators by researchers and organizationsinterested in measuring and improving health care quality. The SLE QIs provide both a
Yazdany et al. Page 5
Arthritis Rheum. Author manuscript; available in PMC 2010 March 15.
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
unique opportunity and challenge on this latter front, as the complexity and lowerprevalence of the condition may require more innovative research designs and larger patientor administrative databases than traditionally required in quality measurement efforts.
It is important to note that the QIs presented here reflect either the current scientificevidence or professional consensus in SLE. However, QIs are not meant to be static. Asstronger clinical evidence becomes available over time, this initial set of indicators mayrequire revisions and updating. In the meantime, the initial QI set provides a starting pointfor quality measurement in SLE.
Although we used a validated approach to develop the SLE QI set, several caveats should betaken into consideration. It is essential to keep in mind that QIs represent a minimalacceptable standard of care, and are not meant to represent best practices or to serve asguidelines for patient management. Moreover, QIs have traditionally been used to assessquality retrospectively, a function distinct from guidelines or recommendations, which oftendefine optimal care for a condition and assist clinical decision-making prospectively.Accordingly, while QIs are often limited in scope, guidelines more comprehensively addressclinical care, and may intentionally leave room for clinician judgment. Many importantaspects of care for patients with SLE are therefore not covered by the QI set, such as themanagement of specific disease manifestations (e.g. neuropsychiatric disease, hematologicalmanifestations, etc.), or special populations (e.g. pediatric SLE). Moreover, importantaspects of disease assessment, such as comprehensively evaluating SLE activity, were alsoexcluded because the expert panels felt further scientific evidence was needed beforeadvocating the use of specific disease activity indices in routine clinical practice. As morescientific evidence becomes available, future efforts may focus on developing QIs for theseareas.
Our effort is the first to our knowledge to systematically develop QIs for a less commonrheumatic condition. Whereas indicators for rheumatoid arthritis or osteoarthritis rely on alarger evidence base of randomized clinical trials, robust clinical trial evidence in SLE islimited for many aspects of disease, necessitating greater reliance on observational studies,studies in other related conditions, and expert consensus. We attempted to address thislimitation by incorporating two face-to-face panel meetings rather than one, with distinctpanelists at each meeting. In order to “pass” to the second meeting, indicators had to reach ahigh level of agreement by the first panel. Although this methodology likely allowed us toadequately capture expert consensus, our research also highlighted that further clinical trialevidence is needed for many aspects of SLE management. It is our hope that disseminationof the QI set may help spur such efforts.
In conclusion, we have created an initial quality indicator set for SLE using a validatedtechnique, a modification of the RAND/UCLA Appropriateness Method. We hope that theavailability of QIs will allow researchers and organizations to engage in qualitymeasurement efforts in SLE. Further study in this area also has the potential to affect healthoutcomes in SLE not only by identifying demographic groups and health care settings wherecare is deficient, but also by serving as the basis for clinical trials and policy interventionsthat aim to improve quality.
AcknowledgmentsThe authors wish to thank participating librarians, including Gloria Won at the University of California, SanFrancisco, Kimberly Schwartz at Stanford University, and Rosalind Dudden at National Jewish Hospital. Inaddition, the authors thank Caroline Gordon, MD, R. Adams Dudley, MD, Chi-yuan Hsu, MD, MSc, and VirginiaD. Winn, MD, PhD for their review of draft indicators, Ann Clarke, MD for assistance in identifying members of
Yazdany et al. Page 6
Arthritis Rheum. Author manuscript; available in PMC 2010 March 15.
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
the expert panels, and Ted Mikuls, MD, Jennifer Barton, MD, Laura Trupin, MPH and Jim Calvert for generalassistance with the project.
Supported by the Arthritis Foundation, the American College of Rheumatology/Research and EducationFoundation and NIAMS P60-AR-053308. Additional support provided by the Rosalind Russell Medical ResearchCenter for Arthritis at the University of California, San Francisco.
Appendix. Items deleted from SLE Quality Indicator Set
Quality Indicator
Original QIs deleted during SLE QI project advisory panel meeting
IF a patient with SLE is receiving immunosuppressive therapy,
THEN live vaccines including those for polio, typhoid, yellow fever, varicella, MMR, influenza (live) andtuberculosis (BCG) should not be given.
IF a patient with SLE is on a combination of high dose prednisone (equivalent ≥20 mg/day for ≥1 month) andcytotoxic therapy, and has lymphopenia (<600 cells/mm3),
THEN prophylaxis with TMP/SMZ or alternative agent should be prescribed unless the patient declines or suchtherapy is contraindicated.
Additional QIs suggested by advisory panel and deleted during formal voting after meeting
IF a patient has SLE,
THEN hydroxychloroquine should be added to the medication regimen, unless the provider documents that thedisease is inactive, the patient declines or contraindications exist.
IF a patient presents with an apparent flare of SLE that includes any of the following features: fever, respiratorysymptoms, chest pain or abdominal pain,
THEN an alternative explanation for these symptoms should be considered.
IF a patient is newly diagnosed with SLE,
THEN a complete history and physical examination should be performed and presence or absence of findings inthe following systems should be documented: constitutional, musculoskeletal, skin, renal, pulmonary, cardiac,gastrointestinal and central nervous system.
IF a patient with SLE is treated with cyclophosphamide,
THEN a discussion with the patient about the potential for infertility as well as treatment alternatives should bedocumented in the medical record, unless the patient is unable to conceive (e.g. a woman has had a hysterectomy,oopherectomy, tubal ligation, or is post-menopausal, or a man has had a vasectomy).
Final QIs deleted during SLE QI project RAND/UCLA Appropriateness Method panel ratings
IF a patient has SLE,
THEN routine follow-up for SLE should occur at ≤6-month intervals, unless the provider documents that thedisease has been inactive without therapy for two or more years.
IF a patient has SLE,
THEN the following laboratory studies should be obtained at least every 6 months: CBC with differential, plateletcount, serum creatinine measurement, and urinalysis, unless the provider documents that the disease has beeninactive without therapy for two or more years.
IF a patient with SLE is taking hydroxychloroquine,
THEN the dose of this medication should be ≤ 6.5 mg/kg/day after three months.
IF a woman between 18 and 45 years of age is newly diagnosed with SLE or new to a clinical practice,
THEN a discussion about the risks associated with conceiving while the disease is active should be documentedwithin one year, unless the patient is unable to conceive (e.g. has had a hysterectomy, oopherectomy, tuballigation, or is post-menopausal).
IF a woman between 18 and 45 years old or a man ≥18 years old is treated with cyclophosphamide for SLE,
THEN fertility-sparing modalities (e.g. leuprolide in women and sperm banking in men) should be discussed withthe patient and documented, unless contraindicated or the patient is unable to conceive (e.g. a woman has had ahysterectomy, oopherectomy, tubal ligation, or is post-menopausal, or a man has had a vasectomy).
Yazdany et al. Page 7
Arthritis Rheum. Author manuscript; available in PMC 2010 March 15.
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
References1. Sutcliffe N, Clarke AE, Gordon C, Farewell V, Isenberg DA. The association of socio-economic
status, race, psychosocial factors and outcome in patients with systemic lupus erythematosus.Rheumatology (Oxford, England). 1999; 38(11):1130–7.
2. Ward MM, Lotstein DS, Bush TM, Lambert RE, van Vollenhoven R, Neuwelt CM. Psychosocialcorrelates of morbidity in women with systemic lupus erythematosus. The Journal of rheumatology.1999; 26(10):2153–8. [PubMed: 10529132]
3. Karlson EW, Daltroy LH, Lew RA, et al. The relationship of socioeconomic status, race, andmodifiable risk factors to outcomes in patients with systemic lupus erythematosus. Arthritis andrheumatism. 1997; 40(1):47–56. [PubMed: 9008599]
4. Friedman AW, Alarcon GS, McGwin G Jr, et al. Systemic lupus erythematosus in three ethnicgroups. IV. Factors associated with self-reported functional outcome in a large cohort study.LUMINA Study Group. Lupus in Minority Populations, Nature versus Nurture. Arthritis Care Res.1999; 12(4):256–66. [PubMed: 10689990]
5. Bernatsky S, Pineau C, Joseph L, Clarke A. Adherence to ophthalmologic monitoring forantimalarial toxicity in a lupus cohort. The Journal of rheumatology. 2003; 30(8):1756–60.[PubMed: 12913931]
6. Bernatsky SR, Cooper GS, Mill C, Ramsey-Goldman R, Clarke AE, Pineau CA. Cancer screeningin patients with systemic lupus erythematosus. The Journal of rheumatology. 2006; 33(1):45–9.[PubMed: 16331804]
7. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century.National Academy Press; Washington D.C., USA: 2001.
8. Campbell SM, Braspenning J, Hutchinson A, Marshall MN. Research methods used in developingand applying quality indicators in primary care. BMJ (Clinical research ed. 2003; 326(7393):816–9.
9. MacLean CH, Saag KG, Solomon DH, Morton SC, Sampsel S, Klippel JH. Measuring quality inarthritis care: methods for developing the Arthritis Foundation's quality indicator set. Arthritis andrheumatism. 2004; 51(2):193–202. [PubMed: 15077259]
10. Donabedian, A. Explorations in quality assessment and monitoring Vol 1 The definition of qualityand approaches to its assessment. Health Administration Press; Ann Arbor, MI: 1980.
11. Brook RH, McGlynn EA, Shekelle PG. Defining and measuring quality of care: a perspective fromUS researchers. Int J Qual Health Care. 2000; 12(4):281–95. [PubMed: 10985266]
12. Mikuls TR, MacLean CH, Olivieri J, et al. Quality of care indicators for gout management.Arthritis and rheumatism. 2004; 50(3):937–43. [PubMed: 15022337]
13. Yelin E, Trupin L, Katz P, et al. Work dynamics among persons with systemic lupuserythematosus. Arthritis and rheumatism. 2007; 57(1):56–63. [PubMed: 17266065]
14. Becker-Merok A, Nossent HC. Damage accumulation in systemic lupus erythematosus and itsrelation to disease activity and mortality. The Journal of rheumatology. 2006; 33(8):1570–7.[PubMed: 16845708]
15. Gladman DD, Urowitz MB, Rahman P, Ibanez D, Tam LS. Accrual of organ damage over time inpatients with systemic lupus erythematosus. The Journal of rheumatology. 2003; 30(9):1955–9.[PubMed: 12966597]
16. Odutola J, Ward MM. Ethnic and socioeconomic disparities in health among patients withrheumatic disease. Current opinion in rheumatology. 2005; 17(2):147–52. [PubMed: 15711226]
17. Brook, RH. The RAND/UCLA appropriateness method. In: McCormick, KA.; Moore, SR.; Siegel,RA., editors. Methodology Perspectives. Rockville, MD: Public Health Service, U.S. Departmentof Health and Human Services; 1994. p. 59-70.AHCPR Pub No 95-009
18. Higgins, J.; Green, S., editors. Cochrane Handbook for Systematic Reviews of Intervnetions 4.2.5.Cochrane Library. Chichester, UK: John Wiley & Sons, Ltd.; 2005. 3
19. Kravitz RL, Park RE, Kahan JP. Measuring the clinical consistency of panelists' appropriatenessratings: the case of coronary artery bypass surgery. Health policy (Amsterdam, Netherlands).1997; 42(2):135–43.
Yazdany et al. Page 8
Arthritis Rheum. Author manuscript; available in PMC 2010 March 15.
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
20. Shekelle PG, Chassin MR, Park RE. Assessing the predictive validity of the RAND/UCLAappropriateness method criteria for performing carotid endarterectomy. International journal oftechnology assessment in health care. 1998; 14(4):707–27. [PubMed: 9885461]
21. Kravitz RL, Laouri M, Kahan JP, et al. Validity of criteria used for detecting underuse of coronaryrevascularization. Jama. 1995; 274(8):632–8. [PubMed: 7637144]
22. Shekelle PG, Kahan JP, Bernstein SJ, Leape LL, Kamberg CJ, Park RE. The reproducibility of amethod to identify the overuse and underuse of medical procedures. The New England journal ofmedicine. 1998; 338(26):1888–95. [PubMed: 9637810]
23. Campbell SM, Hann M, Hacker J, Durie A, Thapar A, Roland MO. Quality assessment for threecommon conditions in primary care: validity and reliability of review criteria developed by expertpanels for angina, asthma and type 2 diabetes. Quality & safety in health care. 2002; 11(2):125–30. [PubMed: 12448803]
24. Mangione-Smith R, DeCristofaro AH, Setodji CM, et al. The quality of ambulatory care deliveredto children in the United States (Appendix). The New England journal of medicine. 2007; 357(15):1515–23. [PubMed: 17928599]
25. Marmor MF, Carr RE, Easterbrook M, Farjo AA, Mieler WF. Recommendations on screening forchloroquine and hydroxychloroquine retinopathy: a report by the American Academy ofOphthalmology. Ophthalmology. 2002; 109(7):1377–82. [PubMed: 12093666]
Yazdany et al. Page 9
Arthritis Rheum. Author manuscript; available in PMC 2010 March 15.
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
Figure 1. Methods for Developing the SLE Quality Indicator Set
Yazdany et al. Page 10
Arthritis Rheum. Author manuscript; available in PMC 2010 March 15.
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
Yazdany et al. Page 11
Table 1Members of SLE Quality Indicators Project Advisory Panel and RAND/UCLAAppropriateness Method Expert Panel
SLE QI Project Advisory Panel Primary Practice Setting
Moderator
David Wofsy, MD University of California, San Francisco
Panelists
Eliza Chakravarty, MD, MSc Stanford University
Maria Dall'Era, MD University of California, San Francisco
James Davis, MD* Sutter Health, San Francisco
John C. Davis, Jr., MD, MPH† University of California, San Francisco
Kenneth Kalunian, MD University of California, San Diego
Nina D. Schwartz, MD Kaiser Permanente, South San Francisco
Daniel J. Wallace, MD David Geffen School of Medicine at UCLA
Michael M. Ward, MD NIH/NIAMS
SLE QI Project RAND/UCLA Appropriateness Method Panel
Moderator
Catherine MacLean, MD, PhD‡ University of California, Los Angeles
Panelists
Rashmi B. Dixit, MD, PhD Walnut Creek, California
Richard Furie, MD North Shore-Long Island Jewish Health System
Jennifer Grossman, MD David Geffen School of Medicine at UCLA
Lester Miller, MD Santa Cruz, California
Rosalind Ramsey-Goldman, MD, DrPh Feinberg School of Medicine, Northwestern University
Brad Rovin, MD* Ohio State University
Kenneth Saag, MD University of Alabama at Birmingham
Jorge Sanchez-Guerrero, MD, MS National Institute of Medical Sciences and Nutrition Salvador Zubirán,Mexico
David Wofsy, MD University of California, San Francisco
*James Davis practices Internal Medicine and Rheumatology. Brad Rovin is a nephrologist. All others primarily practice Rheumatology.
†John C. Davis, Jr. is currently employed at Genentech, Inc. He was employed at the University of California, San Francisco when asked to sit on
the panel.
‡Catherine MacLean is also Director of Programs in Clinical Excellence at Wellpoint, Inc.
Arthritis Rheum. Author manuscript; available in PMC 2010 March 15.
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
Yazdany et al. Page 12
Table 2Indicator inclusion criteria for the final SLE QI set
Mean Validity Rating* Mean Feasibility Rating Result
1-3 1-3 Exclude
1-3 4-6 Exclude
1-3 7-9 Exclude
4-6 1-3 Exclude
4-6 4-6 Exclude
4-6 7-9 Exclude
7-9 1-3 Exclude
7-9 4-6 Include
7-9 7-9 Include
*To be valid, indicators required a high mean validity rating, as well as no evidence of disagreement. Disagreement was defined as two or more
ratings in the highest tertile for validity, and two or more ratings in the lowest tertile.
Arthritis Rheum. Author manuscript; available in PMC 2010 March 15.
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
Yazdany et al. Page 13
Tabl
e 3
Fina
l SL
E q
ualit
y in
dica
tors
rat
ed a
s val
id a
nd fe
asib
le
Qua
lity
Indi
cato
rV
alid
ity(M
ean,
Ran
ge)
Feas
ibili
ty(M
ean,
Ran
ge)
Dia
gnos
is
IF
a p
atie
nt h
as a
susp
ecte
d di
agno
sis o
f SLE
,
TH
EN
an
initi
al w
ork-
up sh
ould
incl
ude
the
follo
win
g: A
NA
, CB
C w
ith d
iffer
entia
l, pl
atel
et c
ount
, ser
um c
reat
inin
e, a
nd u
rinal
ysis
.8.
2 (4
-9)
5.0
(1-9
)
IF
a p
atie
nt is
new
ly d
iagn
osed
with
SLE
,
TH
EN
the
follo
win
g la
bora
tory
test
s sho
uld
be o
rder
ed w
ithin
6 m
onth
s of d
iagn
osis
: ant
i-dsD
NA
, com
plem
ent l
evel
s, an
d an
ti-ph
osph
olip
id a
ntib
odie
s.8.
1 (6
-9)
8.6
(7-9
)
Gen
eral
Pre
vent
ive
Stra
tegi
es
IF
a p
atie
nt h
as S
LE,
T
HE
N e
duca
tion
abou
t sun
avo
idan
ce sh
ould
be
docu
men
ted
at le
ast o
nce
in th
e m
edic
al re
cord
(e.g
. wea
ring
prot
ectiv
e cl
othi
ng, a
pply
ing
suns
cree
ns w
hene
ver
outd
oors
, and
avo
idin
g su
nbat
hing
).
7.7
(5-9
)5.
6 (3
-9)
IF
a p
atie
nt w
ith S
LE is
on
imm
unos
uppr
essi
ve th
erap
y,
TH
EN
an
inac
tivat
ed in
fluen
za v
acci
natio
n sh
ould
be
adm
inis
tere
d an
nual
ly, u
nles
s pat
ient
refu
sal o
r con
train
dica
tions
are
not
ed.
7.7
(6-9
)6.
7 (5
-9)
IF
a p
atie
nt w
ith S
LE is
on
imm
unos
uppr
essi
ve th
erap
y,
TH
EN
a p
neum
ococ
cal v
acci
ne sh
ould
be
adm
inis
tere
d, u
nles
s pat
ient
refu
sal o
r con
train
dica
tions
are
not
ed.
7.6
(6-9
)6.
7 (2
-9)
Ost
eopo
rosi
s
IF
a p
atie
nt w
ith S
LE h
as re
ceiv
ed p
redn
ison
e ≥
7.5
mg/
day
(or o
ther
glu
coco
rtico
id e
quiv
alen
t) fo
r ≥3
mon
ths,
T
HE
N th
e pa
tient
shou
ld h
ave
bone
min
eral
den
sity
(BM
D) t
estin
g re
cord
ed in
the
med
ical
reco
rd* ,
unl
ess t
he p
atie
nt is
cur
rent
ly re
ceiv
ing
anti-
reso
rptiv
e or
anab
olic
ther
apy.
8.0
(7-9
)7.
9 (7
-9)
IF
a p
atie
nt w
ith S
LE h
as re
ceiv
ed p
redn
ison
e ≥
7.5
mg/
day
(or o
ther
glu
coco
rtico
id e
quiv
alen
t) fo
r ≥3
mon
ths,
T
HE
N su
pple
men
tal c
alci
um a
nd v
itam
in D
shou
ld b
e pr
escr
ibed
or r
ecom
men
ded
and
docu
men
ted.
7.6
(6-9
)7.
1 (5
-9)
IF
a p
atie
nt w
ith S
LE h
as re
ceiv
ed p
redn
ison
e ≥
7.5
mg/
day
(or o
ther
glu
coco
rtico
id e
quiv
alen
t) fo
r ≥1
mon
th, a
nd h
as a
cen
tral t
-sco
re ≤
-2.5
or a
his
tory
of
frag
ility
frac
ture
,
TH
EN
the
patie
nt sh
ould
be
treat
ed w
ith a
n an
ti-re
sorp
tive
or a
nabo
lic a
gent
, unl
ess p
atie
nt re
fusa
l or c
ontra
indi
catio
ns a
re n
oted
.
8.0
(6-8
)8.
0 (6
-9)
Dru
g M
onito
ring
IF
a p
atie
nt is
pre
scrib
ed a
new
med
icat
ion
for S
LE (e
.g. N
SAID
s, D
MA
RD
s or g
luco
corti
coid
s),
T
HE
N a
dis
cuss
ion
with
the
patie
nt a
bout
the
risks
ver
sus b
enef
its o
f the
cho
sen
ther
apy
shou
ld b
e do
cum
ente
d.8.
3 (7
-9)
8.3
(7-9
)
IF
a p
atie
nt w
ith S
LE is
new
ly p
resc
ribed
an
NSA
ID, D
MA
RD
, or g
luco
corti
coid
,
TH
EN
bas
elin
e st
udie
s sho
uld
be d
ocum
ente
d w
ithin
an
appr
opria
te p
erio
d of
tim
e fr
om th
e or
igin
al p
resc
riptio
n (T
AB
LE 4
).Se
e Ta
ble
4Se
e Ta
ble
4
IF
a p
atie
nt w
ith S
LE h
as e
stab
lishe
d tre
atm
ent w
ith a
n N
SAID
, DM
AR
D, o
r glu
coco
rtico
id,
T
HE
N m
onito
ring
for d
rug
toxi
city
shou
ld b
e pe
rfor
med
(TA
BLE
4).
See
Tabl
e 4
See
Tabl
e 4
IF
a p
atie
nt w
ith S
LE is
taki
ng p
redn
ison
e ≥
10 m
g (o
r oth
er st
eroi
d eq
uiva
lent
) for
≥3
mon
ths,
T
HE
N a
n at
tem
pt sh
ould
be
mad
e to
tape
r the
pre
dnis
one,
add
a st
eroi
d-sp
arin
g ag
ent,
or e
scal
ate
the
dose
of a
n ex
istin
g st
eroi
d-sp
arin
g ag
ent,
unle
ss p
atie
ntre
fusa
l or c
ontra
indi
catio
ns a
re n
oted
.
7.7
(4-9
)7.
7 (5
-9)
Ren
al D
isea
se
IF
a p
atie
nt h
as h
ad e
vide
nce
of S
LE re
nal d
isea
se (i
ncre
asin
g pr
otei
nuria
, act
ive
urin
ary
sedi
men
t, a
risin
g cr
eatin
ine,
or d
isea
se a
ctiv
ity o
n re
nal b
iops
y) in
the
past
two
year
s,8.
0 (5
-9)
7.8
(5-9
)
Arthritis Rheum. Author manuscript; available in PMC 2010 March 15.
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
Yazdany et al. Page 14
Qua
lity
Indi
cato
rV
alid
ity(M
ean,
Ran
ge)
Feas
ibili
ty(M
ean,
Ran
ge)
T
HE
N th
e fo
llow
ing
shou
ld b
e ob
tain
ed a
t 3-m
onth
inte
rval
s: C
BC
, ser
um c
reat
inin
e, u
rinal
ysis
with
mic
rosc
opic
eva
luat
ion,
and
mea
sure
men
t of u
rine
prot
ein
usin
g a
quan
titat
ive
assa
y.
IF
a p
atie
nt is
dia
gnos
ed w
ith p
rolif
erat
ive
SLE
neph
ritis
(WH
O o
r ISN
/RPS
Cla
ss II
I or I
V),
T
HE
N th
erap
y w
ith c
ortic
oste
roid
s com
bine
d w
ith a
noth
er im
mun
osup
pres
sant
age
nt sh
ould
be
prov
ided
and
doc
umen
ted
with
in o
ne m
onth
of t
his d
iagn
osis
,un
less
pat
ient
refu
sal o
r con
train
dica
tions
are
not
ed.
8.7
(8-9
)8.
6 (8
-9)
IF
a p
atie
nt w
ith S
LE h
as re
nal d
isea
se (p
rote
inur
ia ≥
300
mg/
day
or e
GFR
<60
mL/
min
) and
≥2
bloo
d pr
essu
re re
adin
gs, i
nclu
ding
the
last
read
ing,
whe
resb
p>13
0 or
dbp
>80
over
3 m
onth
s,
TH
EN
pha
rmac
olog
ic th
erap
y fo
r hyp
erte
nsio
n sh
ould
be
initi
ated
or t
he c
urre
nt re
gim
en sh
ould
be
chan
ged
or e
scal
ated
, unl
ess p
atie
nt re
fusa
l or
cont
rain
dica
tions
are
not
ed.
8.7
(8-9
)8.
6 (7
-9)
IF
a p
atie
nt w
ith S
LE h
as p
rote
inur
ia≥
300
mg/
day,
T
HE
N th
e pa
tient
shou
ld b
e tre
ated
with
an
AC
E in
hibi
tor o
r AR
B, u
nles
s pat
ient
refu
sal o
r con
train
dica
tions
are
not
ed.
8.2
(7-9
)8.
6 (8
-9)
Car
diov
ascu
lar
Dis
ease
IF
a p
atie
nt h
as S
LE,
T
HE
N ri
sk fa
ctor
s for
car
diov
ascu
lar d
isea
se, i
nclu
ding
smok
ing
stat
us, b
lood
pre
ssur
e, B
MI,
diab
etes
, and
seru
m li
pids
(inc
ludi
ng to
tal c
hole
ster
ol, H
DL,
LD
Lan
d tri
glyc
erid
es),
shou
ld b
e ev
alua
ted
annu
ally
.
8.3
(6-9
)7.
6 (5
-9)
Preg
nanc
y an
d R
epro
duct
ive
Hea
lth
IF
a p
atie
nt w
ith S
LE is
pre
gnan
t,
TH
EN
ant
i-ssA
, ant
i-ssB
**, a
nd a
nti-p
hosp
holip
id a
ntib
odie
s† sh
ould
be
docu
men
ted
in th
e m
edic
al re
cord
.8.
4 (7
-9)
8.1
(7-9
)
IF
a p
atie
nt h
as h
ad p
regn
ancy
com
plic
atio
ns a
s a re
sult
of th
e an
ti-ph
osph
olip
id a
ntib
ody
synd
rom
e (A
PS) ‡
,
TH
EN
the
patie
nt sh
ould
be
offe
red
aspi
rin a
nd h
epar
in (i
.e. h
epar
in o
r low
mol
ecul
ar w
eigh
t hep
arin
) dur
ing
subs
eque
nt p
regn
anci
es.
8.0
(5-9
)7.
6 (6
-9)
IF
a w
oman
bet
wee
n 18
and
45
year
s of a
ge is
star
ted
on a
ny o
f the
follo
win
g m
edic
atio
ns fo
r SLE
: chl
oroq
uine
, qui
nacr
ine,
met
hotre
xate
, aza
thio
prin
e,le
fluno
mid
e, m
ycop
heno
late
mof
etil,
cyc
losp
orin
e, c
yclo
phos
pham
ide,
or t
halid
omid
e,
TH
EN
a d
iscu
ssio
n w
ith th
e pa
tient
abo
ut th
e po
tent
ial t
erat
ogen
ic ri
sks o
f the
rapy
and
abo
ut c
ontra
cept
ion
shou
ld b
e do
cum
ente
d pr
ior t
o dr
ug in
itiat
ion,
unl
ess
the
patie
nt is
una
ble
to c
once
ive
(e.g
. has
had
a h
yste
rect
omy,
oop
here
ctom
y, tu
bal l
igat
ion,
or i
s pos
t-men
opau
sal).
7.7
(4-9
)7.
6 (4
-9)
* The
pane
l rec
omm
ends
that
BM
D te
stin
g be
reco
rded
in th
e m
edic
al re
cord
eith
er w
ithin
the
12 m
onth
s pre
cedi
ng o
r the
6 m
onth
s afte
r ini
tiatio
n of
glu
coco
rtico
id th
erap
y.
**Th
e pa
nel r
ecom
men
ds th
at a
nti-s
sA a
nd a
nti-s
sB a
ntib
odie
s sho
uld
be re
cord
ed in
the
med
ical
reco
rd d
urin
g th
e fir
st tr
imes
ter a
nd re
flect
cur
rent
test
ing
or re
sults
from
the
prio
r tw
o ye
ars.
† The
pane
l rec
omm
ends
ant
i-pho
spho
lipid
ant
ibod
ies s
houl
d be
reco
rded
in th
e m
edic
al re
cord
with
in 4
wee
ks o
f the
dia
gnos
is o
f pre
gnan
cy a
nd re
flect
cur
rent
test
ing
or re
sults
from
the
prio
r six
mon
ths.
‡ APS
pre
gnan
cy c
ompl
icat
ions
incl
ude
unex
plai
ned
feta
l dea
th a
fter 1
0 w
eeks
ges
tatio
n, b
irth
befo
re 3
4 w
eeks
as a
resu
lt of
seve
re p
reec
lam
psia
, ecl
amps
ia o
r pla
cent
al in
suff
icie
ncy,
or t
hree
or m
ore
unex
plai
ned
cons
ecut
ive
spon
tane
ous a
borti
ons b
efor
e th
e 10
th w
eek
of g
esta
tion,
in th
e se
tting
of p
ositi
ve a
nti-p
hosp
holip
id a
ntib
ody
test
ing.
Arthritis Rheum. Author manuscript; available in PMC 2010 March 15.
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
NIH
-PA Author Manuscript
Yazdany et al. Page 15
Tabl
e 4
Qua
lity
Indi
cato
rs fo
r Ph
arm
acol
ogic
The
rapy
Mon
itori
ng in
SL
E
A. B
asel
ine
stud
ies f
or p
harm
acol
ogic
ther
apy.
Dru
gC
BC
and
pla
tele
tsSe
rum
Cr
Seru
m g
luco
seA
ST o
r A
LT
Alb
umin
Alk
Pho
sU
AO
ther
NSA
IDs &
salic
ylat
es†
X*
XB
lood
pre
ssur
e
Glu
coco
rtico
ids†
XB
lood
pre
ssur
e
Hyd
roxy
chlo
roqu
ine
Fund
osco
pic
exam
with
in o
ne y
ear
Aza
thio
prin
e‡X
XX
X
Cyc
loph
osph
amid
e‡X
XX
X
Met
hotre
xate
‡X
XX
XX
CX
R§
Myc
ophe
nola
te m
ofet
il‡X
X
B. I
nter
val s
tudi
es fo
r ph
arm
acol
ogic
ther
apy
(num
ber
repr
esen
ts in
terv
al in
wee
ks).
NSA
IDs &
salic
ylat
es52
*52
52B
lood
pre
ssur
e (5
2)
Glu
coco
rtico
ids
52B
lood
pre
ssur
e (5
2)
Hyd
roxy
chlo
roqu
ine
Low
risk
pat
ient
s (ag
e 18
to 6
0 yr
s):
—
one
exa
m e
very
two
year
s.†H
igh-
risk
patie
nts (
age
>60
yrs,
dose
>6.
5 m
g/kg
, im
paire
d re
nal
or h
epat
ic fu
nctio
n, o
besi
ty, m
acul
ar d
egen
erat
ion,
or p
rior a
nti-
mal
aria
l dru
g us
e):
—
ann
ual s
cree
ning
.†
Aza
thio
prin
e12
12
Cyc
loph
osph
amid
e4
4
Met
hotre
xate
1212
1212
12
Myc
ophe
nola
te m
ofet
il12
* Onl
y if
risk
fact
ors f
or G
I ble
edin
g pr
esen
t: ag
e ≥
75,
pep
tic u
lcer
dis
ease
, gas
troin
test
inal
ble
edin
g, o
r glu
coco
rtico
id u
se.
† Rec
omm
ende
d in
terv
al fo
r bas
elin
e te
stin
g: 3
mon
ths p
rior o
r 1 m
onth
afte
r dru
g in
itiat
ion.
‡ Rec
omm
ende
d in
terv
al fo
r bas
elin
e te
stin
g: w
ithin
1 m
onth
of d
rug
initi
atio
n.
§ Rec
omm
ende
d in
terv
al fo
r bas
elin
e te
stin
g: w
ithin
the
prio
r yea
r or 1
mon
th a
fter d
rug
initi
atio
n.
† Mod
ified
Am
eric
an A
cade
my
of O
phth
alm
olog
y re
com
men
datio
ns.2
5
Arthritis Rheum. Author manuscript; available in PMC 2010 March 15.