MEASUREMENT IN HEALTH BEHAVIOR Methods for Research and Education Colleen Konicki Di Iorio Y
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Colleen Konicki Di Iorio
File Attachment
Measurement in Health Behavior
MEASUREMENT IN HEALTH BEHAVIOR
Colleen Konicki Di Iorio
Copyright © 2005 by John Wiley & Sons, Inc. All rights reserved.
Published by Jossey-Bass A Wiley Imprint 989 Market Street, San Francisco, CA 94103-1741 www.josseybass.com
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permis- sion of the publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400, fax 978-646-8600, or on the Web at www.copyright.com. Requests to the publisher for permission should be addressed to the Permis- sions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, 201-748-6011, fax 201- 748-6008, or online at http://www.wiley.com/go/permissions.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.
Readers should be aware that Internet Web sites offered as citations and/or sources for further information may have changed or disappeared between the time this was written and when it is read.
Jossey-Bass books and products are available through most bookstores. To contact Jossey-Bass directly call our Customer Care Department within the U.S. at 800-956-7739, outside the U.S. at 317-572-3986, or fax 317- 572-4002.
Jossey-Bass also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books.
Library of Congress Cataloging-in-Publication Data Di Iorio, Colleen Konicki, 1947-
Measurement in health behavior : methods for research and education / Colleen Konicki Di Iorio.
p. ; cm. Includes bibliographical references and index. ISBN 0-7879-7097-2 (alk. paper) 1. Health behavior--Measurement. 2. Health
behavior--Research--Methodology. [DNLM: 1. Health Behavior. 2. Data Collection. 3. Reproducibility of
Results. W 85 D575m 2005] I. Title. RA776.9.D54 2005 362.1072--dc22
2005027390
Printed in the United States of America FIRST EDITION
PB Printing 10 9 8 7 6 5 4 3 2 1
01_970972 ffirs.qxd 9/30/05 9:31 AM Page vi
Preface xxi
1 Introduction to Measurement 1
Role of Measurement in Health Education and Health Behavior Research 1
Brief History of Psychosocial Measurement 3
Conceptualization of Measurement 6
Scales of Measurement 9
Major Concepts of Measurement: Reliability and Validity 16 • Reliability 16
• Validity 17
Summary 17
2 Types of Measures 18
Self-Report 19 • Interviews 19
• Indexes 31
Observations 33
• Systematic Error 39
Response Sets 42
Social Desirability 42
Positive Skew 45
Strategies to Reduce Errors During Interviews 51
• Situational Factors 51
Measurement Rules 53
4 Survey Development 54
• Objectives 56
• Respondents 59
General Principles of Item Writing 59 • The Do’s of Item Writing 60
• The Don’ts of Item Writing 68
Survey Format 70
Survey Administration 72
Test Construction 76 • State the Purpose of the Test 76
• State the Test Objectives 78
• Review Content 79
• Domain-Sampling Model 80
• True/False Items 90
Average Congruency Percentage 98
6 Theory and Measurement 103
Linking Measurement to Theory-Based Health Practice and Research 104 • Gibbs’s Model 106
• Theoretical and Operational Definitions 109
Conceptualization Issues in Scale Development 110 • Survey Versus Scale 110
• Single-Item Versus Multiple-Item Scales 111
• Concept Selection 113
02_970972 ftoc.qxd 9/19/05 6:01 PM Page ix
Concept Analysis 114 • Identify Definitions and Uses of the Concept 114
• Identify Critical Attributes of the Concept 116
• Identify Dimensions of the Concept 117
• Identify Similar and Different Concepts 117
• Identify Antecedents and Consequences of the Concept 118
• Write a Model Case 118
• State the Variables 119
Item Development Process 122 • State the Theoretical Definition 123
• Identify Dimensions of the Concept 123
• Create a Content-Domain by Concept-Dimension Matrix 123
• Decide How Many Items to Include for Each Matrix Cell 124
• Write Items for Each Content-Domain by Concept-Dimension Cell 126
Literature Review 126
Qualitative Interviews 126
Other Instruments 127
Summated Rating Scales 128 • Types of Response Options 128
• Number of Response Options 130
• Odd or Even Number of Categories 131
• Meanings of Response Option Adjectives or Adverbs 132
• Use of Adjectives, Adverbs, or Numbers, or Combinations Thereof 132
• Positively and Negatively Worded Items 133
• Scoring 133
Summary 135
Basic Statistical Concepts 137 • Frequency Distribution 137
Measures of Central Tendency 139
Measures of Dispersion 139
• SPSS Commands 140 Recode Negatively Worded Items Using SPSS 144
Compute Total Scale Scores Using SPSS 145
x Contents
Review of Correlation 146 • Interpretation of the Correlation Coefficient 150
• Correlation Matrix 152
• Correlation Issues 152
Analysis of Variance 157 • Definition and Use 157
• SPSS Commands for a One-Way ANOVA 158
• Interpreting a One-Way ANOVA 158
Summary 162
Classical Test Theory 165 • Observed, True, and Error Scores 165
• Assumptions and Characteristics 167
• Calculation of Variance Due to the True Score 173
• Calculation of the Reliability Index 174
Summary 174
• Stability 178
Factors Associated with Coefficient Alpha 191 • Test Length and Interitem Correlation 191
• Test Adjustments to Increase Alpha 191
• Standards of Reliability 193
Item Analysis 193
Intraclass Correlation Coefficient 203
Generalizability Theory 206
• Content Validity Index 218
Think Aloud 221
Verbal Probe 222
• Construct Validity 229
• Multitrait-Multimethod Matrix 233
• Validity Issues 235
• Conceptual Basis 241
Steps in Factor Analysis 244 • Basic Requirements for FA 246
• Initial Assessment 247
Test for an Identity Matrix 251
• Tests of Sampling Adequacy 252
• Selection of the Type of Factor Analysis 253
Initial Extraction 255
Communality 257
Disadvantages of Classical Test Theory 277
Item Response Theory Basics 278
Polytomous Models 285
Technical Issues 288
TABLES, FIGURES, AND EXHIBITS
Tables
2.1 The Rosenberg Self-Esteem Scale. 5.1 Sample Table of Test Specifications. 5.2 Sample Table of Test Specifications Using Individual Learning Objectives. 5.3 Table of Test Specifications for an HIV Knowledge Test. 5.4 Item-Objective Congruency Evaluation Conducted by Three Content
Experts for Ten Items. 5.5 Example of Item Analysis for a True/False HIV Knowledge Test. 5.6 P Values for Answer Choices on Three Multiple-Choice Items on an HIV
Knowledge Test. 7.1 Example of a 1 × 1 Matrix. 7.2 Example of an 8 × 1 Matrix. 7.3 Matrix for Outcome Expectations for Healthy Eating. 7.4 Items on the Outcome Expectations for Healthy Eating Scale. 8.1 Frequency and Percentage of Participants’ Benefit Ratings (N = 100). 8.2 Calculation of Variance Using Benefit Ratings of 100 Participants. 8.3 SPSS Printout of Measures of Central Tendency and Dispersion. 8.4 Calculation of the Correlation Coefficient for the Association Between
Exercise and Oxygen Consumption.
8.5 Correlation Matrix Showing Relationships Among Benefit Ratings, Hours of Exercise, VO2, and Weight Change.
8.6 SPSS Printout Using the One-Way ANOVA Descriptive Command. 8.7 SPSS Printout for Test of Homogeneity of Variances. 8.8 SPSS Printut of ANOVA Summary Table. 8.9 SPSS Printout Showing Comparisons Among the Groups. 9.1 Josh’s Observed, True, and Error Scores for Ten Attempts on a Research
Methods Quiz. 9.2 Descriptive Statistics for True Scores, Parallel Test Scores, and Error Scores
for a Sample of Students. 9.3 Correlations Among True Scores, Parallel Test Scores, and Error Scores for
a Sample of Students. 10.1 SPSS Printout Showing the Results of Parallel Test Reliability Assessment. 10.2 SPSS Printout Showing the Results of Test-Retest Reliability Assessment. 10.3 Correlation Matrix for the Maternal Involvement Scale. 10.4 Variance-Covariance Matrix. 10.5 Variance Covariance for the Maternal Involvement Scale. 10.6 Values of Cronbach’s Alpha for Various Combinations of Different Num-
ber of Items and Different Average Interitem Correlations. 10.7 Ratings of Two Observers on Ten Videotape Segments. 10.8 Cross-Tabulations for Agreement Among Observers. 10.9 Amount of Variance and Percentage of Total Variance for the Social Support
Scale. 11.1 New and Old Terminology for Validity. 11.2 Hypothetical Scores of Participants on a Readiness Scale and Percentage of
Doses Taken Correctly. 11.3 Correlation Coefficients for Relationships Between Self-Efficacy and Outcome
Expectations, Social Support, Difficult Life Circumstances, and Age. 11.4 SPSS Printout of ANOVA for Hypothetical Data Comparing Differences
Between Two Groups on a Self-Efficacy Scale. 11.5 Expected Pattern of Relationships for a Multitrait-Multimethod Matrix. 11.6 Multitrait-Multimethod Validity Matrix for QOL and Stigma Scales Mea-
sured by Self-Report and Interview. 12.1 Categories of Physical Activities Formed by Conceptual Similarities. 12.2 Correlation Matrix for Eight Items on the SE Scale. 12.3 SPSS Printout with Descriptive Statistics for the Twelve-Item SE Scale. 12.4 SPSS Printout of the Correlation Matrix for the Twelve-Item SE Scale. 12.5 SPSS Printout of the KMO and Bartlett’s Test of Sphericity for the Twelve-
Item SE Scale. 12.6 SPSS Printout of the Anti-Image Correlation Matrix for the Twelve-Item SE
Scale.
02_970972 ftoc.qxd 9/19/05 6:01 PM Page xvi
12.7 SPSS Printout of the Initial Solution for the Twelve-Item SE Scale. 12.8 SPSS Printout for the Unrotated Factor for the Twelve-Item SE Scale. 12.9 SPSS Printout of the Communalities Table for the Twelve-Item SE Scale.
12.10 SPSS Printout of Goodness-of-Fit Test for the Twelve-Item SE Scale. 12.11 SPSS Printout of the Unrotated Factor Matrix for Eight Items on the SE
Scale. 12.12 SPSS Printout of the Rotated Factor (Pattern) Matrix for Eight Items on the
SE Scale. 12.13 SPSS Printout of the Rotated Factor Matrices and Factor Correlation Matrix
for the Twelve-Item SE Scale. 12.14 Factor Correlation Matrix. 12.15 SPSS Printout of the Rotated Pattern Matrix for the Twelve-Item SE Scale
Using the Sort and Suppression Commands Under Options. 13.1 Items for Two Hypothetical Tests. 13.2 Equations for the One-, Two-, and Three-Parameter Logistic IRT Models. 13.3 CCC for a Three-Category Graded Response Model Where a = 1.5,
b1 = –1.0, and b2 = 1.0. 13.4 ICRFs for a Three-Category Item Where a = 1.5, b1 = –1, and b2 = 1.
Figures
1.1 Conceptualization of the Measurement Process. 1.2 Interval Scale for Weights of Newborns Using the Mean Weight as the Origin. 1.3 Schema for Levels of Measurement. 2.1 Response Format from a Worksheet Completed by a First-Grade Student. 2.2 Wong-Baker Faces Pain Rating Scale. 2.3 General Form of a Visual Analog Scale. 3.1 Response Options. 4.1 Steps in Survey Development. 5.1 Steps in the Development of a Knowledge Test. 5.2 Diagram of the Domain-Sampling Method. 6.1 Steps in the Scale Development Process. 6.2 The Measurement Process in Health Behavior Research. 6.3 Example of Relational Statements in Social Cognitive Theory. 6.4 Gibbs’s Relational Model (1972) as Modified by Dulock and Holzemer (1991). 6.5 Steps in Conducting a Concept Analysis. 7.1 Steps in the Item-Writing Process. 8.1 Graphic Representation of Benefit Ratings for a Sample of 100 Participants. 8.2 SPSS Data Editor Screen. 8.3 SPSS Dialog Box for Frequency.
Tables, Figures, and Exhibits xvii
02_970972 ftoc.qxd 9/19/05 6:01 PM Page xvii
8.4 Relationship Between Number of Hours Exercised and VO2 for 100 Participants. 8.5 Relationship Between Number of Hours Exercised and Change in Weight for
100 Participants. 8.6 Range and Sign of Correlation Coefficients. 8.7 Possible Causal Relationships Between Two Variables. 8.8 Relationship Between Number of Hours Exercised and VO2 for Those Who
Exercise Fewer Than 5 Hours per Week. 8.9 Venn Diagram Showing Variance for Two Separate Variables.
8.10 Venn Diagram Showing the Shared Variance Between Two Variables. 9.1 Histogram of Josh’s Scores on 100 Repeated Attempts on a Research Methods
Test. 9.2 Histogram of Josh’s Scores on 100 Repeated Attempts on a Health Behavior
Theory Test. 9.3 Relationship Among Observed-, True-, and Error-Score Variance.
10.1 Diagram of the Components in a G Study. 10.2 Venn Diagram for P ×O ×I Design. 11.1 SPSS Printout of Scatterplot Showing the Relationship Between Readiness
Scores and Medication Adherence for a Hypothetical Set of Data. 11.2 Hypothesized Relationships Among Variables Within Social Cognitive Theory. 11.3 Hypothesized Relationships Among Variables to Be Examined. 12.1 Structure of the Outcome Expectations for Healthy Eating Scale. 12.2 Reconfiguration of a Correlation Matrix into a Factor (Structure) Matrix
for Eight Items on the SE Scale. 12.3 Steps in Factor Analysis. 12.4 SPSS Dialog Box for Descriptives Command. 12.5 SPSS Dialog Box for Extraction Commands. 12.6 SPSS Printout of the Scree Plot for the Twelve-Item SE Scale. 12.7 Scatterplot of Factor Loadings for the Unrotated Factor Matrix for the Eight-
Item SE Scale. 12.8 Scatterplot of Factor Loadings for the Rotated Factor (Pattern) Matrix for the
Eight-Item SE Scale. 12.9 SPSS Dialog Box for Rotation Commands.
12.10 SPSS-Generated Scatterplot of Rotated Factor Matrix for the Twelve-Item SE Scale.
12.11 SPSS Dialog Box for the Options Commands. 12.13 Latent Variable Model for SEM Analysis.
13.1 Example of an Item Characteristic Curve. 13.2 Theoretical ICCs for One-, Two-, and Three-Parameter Logistic IRT Mod-
els for Four Items with Given Parameter Values.
xviii Tables, Figures, and Exhibits
02_970972 ftoc.qxd 9/19/05 6:01 PM Page xviii
Exhibits
1.1 Transformation of an Ordinal Variable by Addition and by Multiplication. 1.2 Conversion of Temperature from Fahrenheit to Celsius Degrees. 2.1 Examples of Response Formats. 2.2 Example of a Thurstone Scale, Using Hypothetical Items and Weights for an
Epilepsy Attitude Scale. 2.3 Examples of Items for an Index of Socioeconomic Status. 3.1 Example of a Knowledge Test Demonstrating Instrument-Related Errors. 4.1 Example of Survey Items on Hand-Hygiene Practices. 5.1 Materials for Review of Knowledge Items by Content Experts 6.1 Sample Survey and Scale Items, Demonstrating Differences. 7.1 Examples of Response Options.
10.1 SPSS Printout Showing the Results of a Split-Half Reliability Assessment for the Maternal Involvement Scale.
10.2 SPSS Printout for the Alpha Coefficient for the Maternal Involvement Scale. 10.3 SPSS Printout of the Results of a KR20 for an HIV Knowledge Test. 10.4 SPSS Printout for Use in Item Analysis for the Maternal Involvement Scale. 10.5 SPSS Printout with the Results of Kappa 10.6 SPSS Printout of an Intraclass Correlation Coefficient. 11.1 Example of Materials for Conducting a Content Validity Assessment. 11.2 Example of Verbal Probes Used in an Assessment. 12.1 Items on the Epilepsy Stigma Scale. 13.1 Equations for the One-, Two-, and Three-Parameter Logistic IRT Models.
Tables, Figures, and Exhibits xix
02_970972 ftoc.qxd 9/19/05 6:01 PM Page xix
To the memory of my parents, Thaddeus M. Konicki and Mary Hogan Konicki
03_970972 fbetw.qxd 9/19/05 5:56 PM Page xx
PREFACE
This text is written primarily for public health students, particularly those in health education and health behavior programs. Students in other health disciplines,
including nursing, medicine, applied health sciences, and psychology, may also find it useful. This book is intended to be an introductory text for students and public health professionals who have a limited background in statistics and mathematics, yet need a basic understanding of measurement issues. It emphasizes classical measurement strategies for constructing instruments, for testing for reliability and validity, and for interpreting results. A major portion of the text deals with scale development and test- ing, but chapters on survey development and the construction of knowledge tests are also included. Likewise, even though classical test theory is the major theoretical approach, generalizability theory and item response theory are also introduced.
The approach used stresses the practical application of concepts and principles. With that in mind, I have used data from research and programs that I have been involved in during the past ten years to provide realistic examples from public health that demonstrate the concepts. This book also includes illustrations of materials used to gather evaluative data, and instructions for conducting tests of reliability and validity using SPSS. Samples of SPSS printouts are provided, along with detailed interpretations of the results.
The overall objectives of this text are to improve the student’s (1) knowledge of instrument development and testing using classical test theory, (2) understanding of reports of reliability and validity testing found in articles and reports, (3) ability
xxi
04_970972 fpref.qxd 9/19/05 6:14 PM Page xxi
to conduct basic tests for reliability and validity, and (4) ability to interpret the results of data analysis. To accomplish these aims, the book is divided into thirteen chapters. The first three chapters provide a background in measurement theory. In Chapter One, the definition of measurement is presented, along with a brief history of mea- surement in the psychological and health sciences and a discussion of the advan- tages of measurement. Chapter Two presents a brief description of the different types of measures that might be used to collect data about health behavior or factors asso- ciated with it. Chapter Three presents a discussion of the types of measurement error and strategies to reduce error.
Chapters Four and Five present overviews of survey development and knowledge test construction, respectively. Chapter Six begins a discussion of scale development, which is the focus for the remainder of the book. This chapter presents important information on the analysis of concepts. In Chapter Seven, the principles of item writ- ing and of formatting summated rating scales are discussed. Chapter Eight presents basic statistical concepts that are necessary for understanding the material in the fol- lowing chapters. Chapters Nine and Ten focus on reliability: Chapter Nine discusses classical test theory, and Chapter Ten examines the different types of reliability as- sessment. The final three chapters of the book focus on validity testing. In Chapter Eleven, the concept of validity is introduced and several strategies for assessing va- lidity are discussed. In Chapter Twelve, the basic principles of factor analysis are pre- sented. The final chapter, Chapter Thirteen, provides an elementary description of item response theory.
Examples and data analyses described in several of the chapters rely heavily on my research in HIV prevention and self-management of chronic illnesses. This work has been supported by several agencies including the National Institute of Mental Health, the National Institute of Nursing Research, the National Institute of Child Health and Human Development, the Centers for Disease Control and Prevention, the Epilepsy Foundation, and the Emory University Research Committee.
Acknowledgments
There are many people who have helped bring this book to completion. First are the students in my classes at the Rollins School of Public Health, who provided the in- spiration for the book. I especially want to thank the students in the measurement classes of spring 2004 and spring 2005. These students used drafts of the book for their class text and provided invaluable feedback on the content and the presentation, for which I am grateful. I also want to thank Amanda Whatley and Andrea Landis, who critiqued each chapter from a student’s perspective and provided comments to
xxii Preface
04_970972 fpref.qxd 9/19/05 6:14 PM Page xxii
improve the clarity of the text. Many thanks also to those who read drafts of the book and provided feedback, especially Stanley A. Mulaik, Frances McCarty, Ronda Sinkowitz-Cochran, and Amy Lansky. I am indebted to Stanley Mulaik who provided many helpful comments throughout the book, in particular on reliability assessment and factor analysis. His perspective on the use of factor analysis for instrument de- velopment is clearly evident in Chapter Twelve. Frances McCarty combed the chap- ters and recommended a number of changes that helped clarify difficult concepts. She also contributed to the section on analysis of variance and wrote the final chapter on item response theory. Ronda Sinkowitz-Cochran and Amy Lansky reviewed the chap- ters from the practitioner’s perspective, and their suggestions for revisions improved the link between theory and practice. I appreciate the support of those who helped with the compilation of the book, and I especially want to thank Pamela Kennedy, Regina Daniel, and Sara Di Iorio for their many hours of assistance with gathering references, formatting chapters, and other tasks. Not forgotten are the many stu- dents who made numerous trips to the library for materials.
I am also indebted to my family for their patience and support. Special thanks go to my husband, Ralph; my children, Matt and Sara; and my sisters, Susan Arrowood, Doris Konicki, Mary Morin, Carol Konicki, Nancy Konicki, and Pamela Kennedy.
August 2005 Colleen Konicki Di Iorio Atlanta, Georgia
Preface xxiii
THE AUTHOR
Colleen Konicki Di Iorio is a professor in the Department of Behavioral Sci- ences and Health Education at Rollins School of Public Health and holds a sec-
ondary appointment in the Department of Family and Community Nursing at the Nell Hodgson Woodruff School of Nursing. She earned her Ph.D. degree in nurs- ing research and theory development from New York University, her master’s de- gree in nursing from New York University, and her bachelor’s degree in nursing from the University of Iowa. She is a registered nurse and a Fellow of the American Acad- emy of Nursing. She has extensive experience in health promotion research and in- strument development. Her research on health behavior has been funded by the Centers for Disease Control and Prevention, the National Institutes of Health, and the Epilepsy Foundation, among others. Her work focuses on two broad areas of health: adherence and self-management, and HIV prevention. Di Iorio has published widely in the health behavior literature. She has served as a consultant to and been a member of panels of professional and community organizations and governmen- tal agencies. She currently serves as the associate editor of the Journal of Nursing Measurement. She recently received the Distinguished Nurse Scholar Award from New York University and the Distinguished Alumni Award from the University of Iowa.
xxv
Measurement in Health Behavior
File Attachment
Measurement in Health Behavior
MEASUREMENT IN HEALTH BEHAVIOR
Colleen Konicki Di Iorio
Copyright © 2005 by John Wiley & Sons, Inc. All rights reserved.
Published by Jossey-Bass A Wiley Imprint 989 Market Street, San Francisco, CA 94103-1741 www.josseybass.com
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permis- sion of the publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400, fax 978-646-8600, or on the Web at www.copyright.com. Requests to the publisher for permission should be addressed to the Permis- sions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, 201-748-6011, fax 201- 748-6008, or online at http://www.wiley.com/go/permissions.
Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.
Readers should be aware that Internet Web sites offered as citations and/or sources for further information may have changed or disappeared between the time this was written and when it is read.
Jossey-Bass books and products are available through most bookstores. To contact Jossey-Bass directly call our Customer Care Department within the U.S. at 800-956-7739, outside the U.S. at 317-572-3986, or fax 317- 572-4002.
Jossey-Bass also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books.
Library of Congress Cataloging-in-Publication Data Di Iorio, Colleen Konicki, 1947-
Measurement in health behavior : methods for research and education / Colleen Konicki Di Iorio.
p. ; cm. Includes bibliographical references and index. ISBN 0-7879-7097-2 (alk. paper) 1. Health behavior--Measurement. 2. Health
behavior--Research--Methodology. [DNLM: 1. Health Behavior. 2. Data Collection. 3. Reproducibility of
Results. W 85 D575m 2005] I. Title. RA776.9.D54 2005 362.1072--dc22
2005027390
Printed in the United States of America FIRST EDITION
PB Printing 10 9 8 7 6 5 4 3 2 1
01_970972 ffirs.qxd 9/30/05 9:31 AM Page vi
Preface xxi
1 Introduction to Measurement 1
Role of Measurement in Health Education and Health Behavior Research 1
Brief History of Psychosocial Measurement 3
Conceptualization of Measurement 6
Scales of Measurement 9
Major Concepts of Measurement: Reliability and Validity 16 • Reliability 16
• Validity 17
Summary 17
2 Types of Measures 18
Self-Report 19 • Interviews 19
• Indexes 31
Observations 33
• Systematic Error 39
Response Sets 42
Social Desirability 42
Positive Skew 45
Strategies to Reduce Errors During Interviews 51
• Situational Factors 51
Measurement Rules 53
4 Survey Development 54
• Objectives 56
• Respondents 59
General Principles of Item Writing 59 • The Do’s of Item Writing 60
• The Don’ts of Item Writing 68
Survey Format 70
Survey Administration 72
Test Construction 76 • State the Purpose of the Test 76
• State the Test Objectives 78
• Review Content 79
• Domain-Sampling Model 80
• True/False Items 90
Average Congruency Percentage 98
6 Theory and Measurement 103
Linking Measurement to Theory-Based Health Practice and Research 104 • Gibbs’s Model 106
• Theoretical and Operational Definitions 109
Conceptualization Issues in Scale Development 110 • Survey Versus Scale 110
• Single-Item Versus Multiple-Item Scales 111
• Concept Selection 113
02_970972 ftoc.qxd 9/19/05 6:01 PM Page ix
Concept Analysis 114 • Identify Definitions and Uses of the Concept 114
• Identify Critical Attributes of the Concept 116
• Identify Dimensions of the Concept 117
• Identify Similar and Different Concepts 117
• Identify Antecedents and Consequences of the Concept 118
• Write a Model Case 118
• State the Variables 119
Item Development Process 122 • State the Theoretical Definition 123
• Identify Dimensions of the Concept 123
• Create a Content-Domain by Concept-Dimension Matrix 123
• Decide How Many Items to Include for Each Matrix Cell 124
• Write Items for Each Content-Domain by Concept-Dimension Cell 126
Literature Review 126
Qualitative Interviews 126
Other Instruments 127
Summated Rating Scales 128 • Types of Response Options 128
• Number of Response Options 130
• Odd or Even Number of Categories 131
• Meanings of Response Option Adjectives or Adverbs 132
• Use of Adjectives, Adverbs, or Numbers, or Combinations Thereof 132
• Positively and Negatively Worded Items 133
• Scoring 133
Summary 135
Basic Statistical Concepts 137 • Frequency Distribution 137
Measures of Central Tendency 139
Measures of Dispersion 139
• SPSS Commands 140 Recode Negatively Worded Items Using SPSS 144
Compute Total Scale Scores Using SPSS 145
x Contents
Review of Correlation 146 • Interpretation of the Correlation Coefficient 150
• Correlation Matrix 152
• Correlation Issues 152
Analysis of Variance 157 • Definition and Use 157
• SPSS Commands for a One-Way ANOVA 158
• Interpreting a One-Way ANOVA 158
Summary 162
Classical Test Theory 165 • Observed, True, and Error Scores 165
• Assumptions and Characteristics 167
• Calculation of Variance Due to the True Score 173
• Calculation of the Reliability Index 174
Summary 174
• Stability 178
Factors Associated with Coefficient Alpha 191 • Test Length and Interitem Correlation 191
• Test Adjustments to Increase Alpha 191
• Standards of Reliability 193
Item Analysis 193
Intraclass Correlation Coefficient 203
Generalizability Theory 206
• Content Validity Index 218
Think Aloud 221
Verbal Probe 222
• Construct Validity 229
• Multitrait-Multimethod Matrix 233
• Validity Issues 235
• Conceptual Basis 241
Steps in Factor Analysis 244 • Basic Requirements for FA 246
• Initial Assessment 247
Test for an Identity Matrix 251
• Tests of Sampling Adequacy 252
• Selection of the Type of Factor Analysis 253
Initial Extraction 255
Communality 257
Disadvantages of Classical Test Theory 277
Item Response Theory Basics 278
Polytomous Models 285
Technical Issues 288
TABLES, FIGURES, AND EXHIBITS
Tables
2.1 The Rosenberg Self-Esteem Scale. 5.1 Sample Table of Test Specifications. 5.2 Sample Table of Test Specifications Using Individual Learning Objectives. 5.3 Table of Test Specifications for an HIV Knowledge Test. 5.4 Item-Objective Congruency Evaluation Conducted by Three Content
Experts for Ten Items. 5.5 Example of Item Analysis for a True/False HIV Knowledge Test. 5.6 P Values for Answer Choices on Three Multiple-Choice Items on an HIV
Knowledge Test. 7.1 Example of a 1 × 1 Matrix. 7.2 Example of an 8 × 1 Matrix. 7.3 Matrix for Outcome Expectations for Healthy Eating. 7.4 Items on the Outcome Expectations for Healthy Eating Scale. 8.1 Frequency and Percentage of Participants’ Benefit Ratings (N = 100). 8.2 Calculation of Variance Using Benefit Ratings of 100 Participants. 8.3 SPSS Printout of Measures of Central Tendency and Dispersion. 8.4 Calculation of the Correlation Coefficient for the Association Between
Exercise and Oxygen Consumption.
8.5 Correlation Matrix Showing Relationships Among Benefit Ratings, Hours of Exercise, VO2, and Weight Change.
8.6 SPSS Printout Using the One-Way ANOVA Descriptive Command. 8.7 SPSS Printout for Test of Homogeneity of Variances. 8.8 SPSS Printut of ANOVA Summary Table. 8.9 SPSS Printout Showing Comparisons Among the Groups. 9.1 Josh’s Observed, True, and Error Scores for Ten Attempts on a Research
Methods Quiz. 9.2 Descriptive Statistics for True Scores, Parallel Test Scores, and Error Scores
for a Sample of Students. 9.3 Correlations Among True Scores, Parallel Test Scores, and Error Scores for
a Sample of Students. 10.1 SPSS Printout Showing the Results of Parallel Test Reliability Assessment. 10.2 SPSS Printout Showing the Results of Test-Retest Reliability Assessment. 10.3 Correlation Matrix for the Maternal Involvement Scale. 10.4 Variance-Covariance Matrix. 10.5 Variance Covariance for the Maternal Involvement Scale. 10.6 Values of Cronbach’s Alpha for Various Combinations of Different Num-
ber of Items and Different Average Interitem Correlations. 10.7 Ratings of Two Observers on Ten Videotape Segments. 10.8 Cross-Tabulations for Agreement Among Observers. 10.9 Amount of Variance and Percentage of Total Variance for the Social Support
Scale. 11.1 New and Old Terminology for Validity. 11.2 Hypothetical Scores of Participants on a Readiness Scale and Percentage of
Doses Taken Correctly. 11.3 Correlation Coefficients for Relationships Between Self-Efficacy and Outcome
Expectations, Social Support, Difficult Life Circumstances, and Age. 11.4 SPSS Printout of ANOVA for Hypothetical Data Comparing Differences
Between Two Groups on a Self-Efficacy Scale. 11.5 Expected Pattern of Relationships for a Multitrait-Multimethod Matrix. 11.6 Multitrait-Multimethod Validity Matrix for QOL and Stigma Scales Mea-
sured by Self-Report and Interview. 12.1 Categories of Physical Activities Formed by Conceptual Similarities. 12.2 Correlation Matrix for Eight Items on the SE Scale. 12.3 SPSS Printout with Descriptive Statistics for the Twelve-Item SE Scale. 12.4 SPSS Printout of the Correlation Matrix for the Twelve-Item SE Scale. 12.5 SPSS Printout of the KMO and Bartlett’s Test of Sphericity for the Twelve-
Item SE Scale. 12.6 SPSS Printout of the Anti-Image Correlation Matrix for the Twelve-Item SE
Scale.
02_970972 ftoc.qxd 9/19/05 6:01 PM Page xvi
12.7 SPSS Printout of the Initial Solution for the Twelve-Item SE Scale. 12.8 SPSS Printout for the Unrotated Factor for the Twelve-Item SE Scale. 12.9 SPSS Printout of the Communalities Table for the Twelve-Item SE Scale.
12.10 SPSS Printout of Goodness-of-Fit Test for the Twelve-Item SE Scale. 12.11 SPSS Printout of the Unrotated Factor Matrix for Eight Items on the SE
Scale. 12.12 SPSS Printout of the Rotated Factor (Pattern) Matrix for Eight Items on the
SE Scale. 12.13 SPSS Printout of the Rotated Factor Matrices and Factor Correlation Matrix
for the Twelve-Item SE Scale. 12.14 Factor Correlation Matrix. 12.15 SPSS Printout of the Rotated Pattern Matrix for the Twelve-Item SE Scale
Using the Sort and Suppression Commands Under Options. 13.1 Items for Two Hypothetical Tests. 13.2 Equations for the One-, Two-, and Three-Parameter Logistic IRT Models. 13.3 CCC for a Three-Category Graded Response Model Where a = 1.5,
b1 = –1.0, and b2 = 1.0. 13.4 ICRFs for a Three-Category Item Where a = 1.5, b1 = –1, and b2 = 1.
Figures
1.1 Conceptualization of the Measurement Process. 1.2 Interval Scale for Weights of Newborns Using the Mean Weight as the Origin. 1.3 Schema for Levels of Measurement. 2.1 Response Format from a Worksheet Completed by a First-Grade Student. 2.2 Wong-Baker Faces Pain Rating Scale. 2.3 General Form of a Visual Analog Scale. 3.1 Response Options. 4.1 Steps in Survey Development. 5.1 Steps in the Development of a Knowledge Test. 5.2 Diagram of the Domain-Sampling Method. 6.1 Steps in the Scale Development Process. 6.2 The Measurement Process in Health Behavior Research. 6.3 Example of Relational Statements in Social Cognitive Theory. 6.4 Gibbs’s Relational Model (1972) as Modified by Dulock and Holzemer (1991). 6.5 Steps in Conducting a Concept Analysis. 7.1 Steps in the Item-Writing Process. 8.1 Graphic Representation of Benefit Ratings for a Sample of 100 Participants. 8.2 SPSS Data Editor Screen. 8.3 SPSS Dialog Box for Frequency.
Tables, Figures, and Exhibits xvii
02_970972 ftoc.qxd 9/19/05 6:01 PM Page xvii
8.4 Relationship Between Number of Hours Exercised and VO2 for 100 Participants. 8.5 Relationship Between Number of Hours Exercised and Change in Weight for
100 Participants. 8.6 Range and Sign of Correlation Coefficients. 8.7 Possible Causal Relationships Between Two Variables. 8.8 Relationship Between Number of Hours Exercised and VO2 for Those Who
Exercise Fewer Than 5 Hours per Week. 8.9 Venn Diagram Showing Variance for Two Separate Variables.
8.10 Venn Diagram Showing the Shared Variance Between Two Variables. 9.1 Histogram of Josh’s Scores on 100 Repeated Attempts on a Research Methods
Test. 9.2 Histogram of Josh’s Scores on 100 Repeated Attempts on a Health Behavior
Theory Test. 9.3 Relationship Among Observed-, True-, and Error-Score Variance.
10.1 Diagram of the Components in a G Study. 10.2 Venn Diagram for P ×O ×I Design. 11.1 SPSS Printout of Scatterplot Showing the Relationship Between Readiness
Scores and Medication Adherence for a Hypothetical Set of Data. 11.2 Hypothesized Relationships Among Variables Within Social Cognitive Theory. 11.3 Hypothesized Relationships Among Variables to Be Examined. 12.1 Structure of the Outcome Expectations for Healthy Eating Scale. 12.2 Reconfiguration of a Correlation Matrix into a Factor (Structure) Matrix
for Eight Items on the SE Scale. 12.3 Steps in Factor Analysis. 12.4 SPSS Dialog Box for Descriptives Command. 12.5 SPSS Dialog Box for Extraction Commands. 12.6 SPSS Printout of the Scree Plot for the Twelve-Item SE Scale. 12.7 Scatterplot of Factor Loadings for the Unrotated Factor Matrix for the Eight-
Item SE Scale. 12.8 Scatterplot of Factor Loadings for the Rotated Factor (Pattern) Matrix for the
Eight-Item SE Scale. 12.9 SPSS Dialog Box for Rotation Commands.
12.10 SPSS-Generated Scatterplot of Rotated Factor Matrix for the Twelve-Item SE Scale.
12.11 SPSS Dialog Box for the Options Commands. 12.13 Latent Variable Model for SEM Analysis.
13.1 Example of an Item Characteristic Curve. 13.2 Theoretical ICCs for One-, Two-, and Three-Parameter Logistic IRT Mod-
els for Four Items with Given Parameter Values.
xviii Tables, Figures, and Exhibits
02_970972 ftoc.qxd 9/19/05 6:01 PM Page xviii
Exhibits
1.1 Transformation of an Ordinal Variable by Addition and by Multiplication. 1.2 Conversion of Temperature from Fahrenheit to Celsius Degrees. 2.1 Examples of Response Formats. 2.2 Example of a Thurstone Scale, Using Hypothetical Items and Weights for an
Epilepsy Attitude Scale. 2.3 Examples of Items for an Index of Socioeconomic Status. 3.1 Example of a Knowledge Test Demonstrating Instrument-Related Errors. 4.1 Example of Survey Items on Hand-Hygiene Practices. 5.1 Materials for Review of Knowledge Items by Content Experts 6.1 Sample Survey and Scale Items, Demonstrating Differences. 7.1 Examples of Response Options.
10.1 SPSS Printout Showing the Results of a Split-Half Reliability Assessment for the Maternal Involvement Scale.
10.2 SPSS Printout for the Alpha Coefficient for the Maternal Involvement Scale. 10.3 SPSS Printout of the Results of a KR20 for an HIV Knowledge Test. 10.4 SPSS Printout for Use in Item Analysis for the Maternal Involvement Scale. 10.5 SPSS Printout with the Results of Kappa 10.6 SPSS Printout of an Intraclass Correlation Coefficient. 11.1 Example of Materials for Conducting a Content Validity Assessment. 11.2 Example of Verbal Probes Used in an Assessment. 12.1 Items on the Epilepsy Stigma Scale. 13.1 Equations for the One-, Two-, and Three-Parameter Logistic IRT Models.
Tables, Figures, and Exhibits xix
02_970972 ftoc.qxd 9/19/05 6:01 PM Page xix
To the memory of my parents, Thaddeus M. Konicki and Mary Hogan Konicki
03_970972 fbetw.qxd 9/19/05 5:56 PM Page xx
PREFACE
This text is written primarily for public health students, particularly those in health education and health behavior programs. Students in other health disciplines,
including nursing, medicine, applied health sciences, and psychology, may also find it useful. This book is intended to be an introductory text for students and public health professionals who have a limited background in statistics and mathematics, yet need a basic understanding of measurement issues. It emphasizes classical measurement strategies for constructing instruments, for testing for reliability and validity, and for interpreting results. A major portion of the text deals with scale development and test- ing, but chapters on survey development and the construction of knowledge tests are also included. Likewise, even though classical test theory is the major theoretical approach, generalizability theory and item response theory are also introduced.
The approach used stresses the practical application of concepts and principles. With that in mind, I have used data from research and programs that I have been involved in during the past ten years to provide realistic examples from public health that demonstrate the concepts. This book also includes illustrations of materials used to gather evaluative data, and instructions for conducting tests of reliability and validity using SPSS. Samples of SPSS printouts are provided, along with detailed interpretations of the results.
The overall objectives of this text are to improve the student’s (1) knowledge of instrument development and testing using classical test theory, (2) understanding of reports of reliability and validity testing found in articles and reports, (3) ability
xxi
04_970972 fpref.qxd 9/19/05 6:14 PM Page xxi
to conduct basic tests for reliability and validity, and (4) ability to interpret the results of data analysis. To accomplish these aims, the book is divided into thirteen chapters. The first three chapters provide a background in measurement theory. In Chapter One, the definition of measurement is presented, along with a brief history of mea- surement in the psychological and health sciences and a discussion of the advan- tages of measurement. Chapter Two presents a brief description of the different types of measures that might be used to collect data about health behavior or factors asso- ciated with it. Chapter Three presents a discussion of the types of measurement error and strategies to reduce error.
Chapters Four and Five present overviews of survey development and knowledge test construction, respectively. Chapter Six begins a discussion of scale development, which is the focus for the remainder of the book. This chapter presents important information on the analysis of concepts. In Chapter Seven, the principles of item writ- ing and of formatting summated rating scales are discussed. Chapter Eight presents basic statistical concepts that are necessary for understanding the material in the fol- lowing chapters. Chapters Nine and Ten focus on reliability: Chapter Nine discusses classical test theory, and Chapter Ten examines the different types of reliability as- sessment. The final three chapters of the book focus on validity testing. In Chapter Eleven, the concept of validity is introduced and several strategies for assessing va- lidity are discussed. In Chapter Twelve, the basic principles of factor analysis are pre- sented. The final chapter, Chapter Thirteen, provides an elementary description of item response theory.
Examples and data analyses described in several of the chapters rely heavily on my research in HIV prevention and self-management of chronic illnesses. This work has been supported by several agencies including the National Institute of Mental Health, the National Institute of Nursing Research, the National Institute of Child Health and Human Development, the Centers for Disease Control and Prevention, the Epilepsy Foundation, and the Emory University Research Committee.
Acknowledgments
There are many people who have helped bring this book to completion. First are the students in my classes at the Rollins School of Public Health, who provided the in- spiration for the book. I especially want to thank the students in the measurement classes of spring 2004 and spring 2005. These students used drafts of the book for their class text and provided invaluable feedback on the content and the presentation, for which I am grateful. I also want to thank Amanda Whatley and Andrea Landis, who critiqued each chapter from a student’s perspective and provided comments to
xxii Preface
04_970972 fpref.qxd 9/19/05 6:14 PM Page xxii
improve the clarity of the text. Many thanks also to those who read drafts of the book and provided feedback, especially Stanley A. Mulaik, Frances McCarty, Ronda Sinkowitz-Cochran, and Amy Lansky. I am indebted to Stanley Mulaik who provided many helpful comments throughout the book, in particular on reliability assessment and factor analysis. His perspective on the use of factor analysis for instrument de- velopment is clearly evident in Chapter Twelve. Frances McCarty combed the chap- ters and recommended a number of changes that helped clarify difficult concepts. She also contributed to the section on analysis of variance and wrote the final chapter on item response theory. Ronda Sinkowitz-Cochran and Amy Lansky reviewed the chap- ters from the practitioner’s perspective, and their suggestions for revisions improved the link between theory and practice. I appreciate the support of those who helped with the compilation of the book, and I especially want to thank Pamela Kennedy, Regina Daniel, and Sara Di Iorio for their many hours of assistance with gathering references, formatting chapters, and other tasks. Not forgotten are the many stu- dents who made numerous trips to the library for materials.
I am also indebted to my family for their patience and support. Special thanks go to my husband, Ralph; my children, Matt and Sara; and my sisters, Susan Arrowood, Doris Konicki, Mary Morin, Carol Konicki, Nancy Konicki, and Pamela Kennedy.
August 2005 Colleen Konicki Di Iorio Atlanta, Georgia
Preface xxiii
THE AUTHOR
Colleen Konicki Di Iorio is a professor in the Department of Behavioral Sci- ences and Health Education at Rollins School of Public Health and holds a sec-
ondary appointment in the Department of Family and Community Nursing at the Nell Hodgson Woodruff School of Nursing. She earned her Ph.D. degree in nurs- ing research and theory development from New York University, her master’s de- gree in nursing from New York University, and her bachelor’s degree in nursing from the University of Iowa. She is a registered nurse and a Fellow of the American Acad- emy of Nursing. She has extensive experience in health promotion research and in- strument development. Her research on health behavior has been funded by the Centers for Disease Control and Prevention, the National Institutes of Health, and the Epilepsy Foundation, among others. Her work focuses on two broad areas of health: adherence and self-management, and HIV prevention. Di Iorio has published widely in the health behavior literature. She has served as a consultant to and been a member of panels of professional and community organizations and governmen- tal agencies. She currently serves as the associate editor of the Journal of Nursing Measurement. She recently received the Distinguished Nurse Scholar Award from New York University and the Distinguished Alumni Award from the University of Iowa.
xxv
Measurement in Health Behavior
Related Documents
