DOCUMENT RESUME ED 426 101 TM 029 331 AUTHOR Moore, Jenny TITLE Some Basic Concepts in Meta-Analysis. PUB DATE 1999-01-00 NOTE 20p.; Paper presented at the Annual Meeting of the Southwest Educational Research Association (San Antonio, TX, January 21-23, 1999). PUB TYPE Reports Descriptive (141) Speeches/Meeting Papers (150) EDRS PRICE MF01/PC01 Plus Postage. DESCRIPTORS *Comparative Analysis; *Effect Size; *Meta Analysis; Regression (Statistics) IDENTIFIERS Generality ABSTRACT The business of science is formulating generalizable insight. No one study, taken singly, establishes the basis for such insight. Meta-analysis, however, can be used to determine if results generalize and to estimate the mean and the variance of effect sizes across studies (J. Hunter and F. Schmidt, 1990) . Meta-analysis inquiries treat studies (rather than people) as the units of analysis, and then use regression or other methods to determine the study features that explain or predict variability in detected effects in a given area of inquiry. There are two primary approaches to meta-analysis that are used: the comparison approach and the correlation approach. This paper explains the basic processes of doing a meta-analysis from the mean difference (comparison) approach to determine which of two independent groups or methods scores higher on the variable of interest. (Contains 31 references.) (Author/SLD) ******************************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. ********************************************************************************
21
Embed
DOCUMENT RESUME ED 426 101 TM 029 331 Moore, Jenny … · Jenny Moore. Texas A&M University. Paper presented at the annual meeting of the Southwest Educational Research. Association,
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
DOCUMENT RESUME
ED 426 101 TM 029 331
AUTHOR Moore, JennyTITLE Some Basic Concepts in Meta-Analysis.PUB DATE 1999-01-00NOTE 20p.; Paper presented at the Annual Meeting of the Southwest
Educational Research Association (San Antonio, TX, January21-23, 1999).
PUB TYPE Reports Descriptive (141) Speeches/Meeting Papers (150)EDRS PRICE MF01/PC01 Plus Postage.DESCRIPTORS *Comparative Analysis; *Effect Size; *Meta Analysis;
Regression (Statistics)IDENTIFIERS Generality
ABSTRACTThe business of science is formulating generalizable
insight. No one study, taken singly, establishes the basis for such insight.Meta-analysis, however, can be used to determine if results generalize and toestimate the mean and the variance of effect sizes across studies (J. Hunterand F. Schmidt, 1990) . Meta-analysis inquiries treat studies (rather thanpeople) as the units of analysis, and then use regression or other methods todetermine the study features that explain or predict variability in detectedeffects in a given area of inquiry. There are two primary approaches tometa-analysis that are used: the comparison approach and the correlationapproach. This paper explains the basic processes of doing a meta-analysisfrom the mean difference (comparison) approach to determine which of twoindependent groups or methods scores higher on the variable of interest.(Contains 31 references.) (Author/SLD)
Carver, R. (1978). The case against statistical significance testing. Harvard
Educational Review, 48, 378-399.
Cohen, J. (1994). The earth is round (p < .05). American Psychologist, 49, 997-
1003.
Daniel, L. G. (in press). Statistical significance testing: A historical overview of
misuse and misinterpretation with implications for the editorial policies of educational
journals. Research in the Schools.
Glass, G. V. (1977). Integrated findings: The meta-analysis of research. In L. S.
Shulman & S. Mowry (Eds.), Review of Research in Education, 5, 351-379.
Kalain, H. A., & Raudenbush, S. W. (1996). A multivariate mixed linear model
for meta-analysis. Psychological Methods, 1, 227-235.
Kavale, K. (1982). The efficacy of stimulant drug treatment for hyperactivity: A
meta-analysis. Journal of Learning Disabilities, 15, 280-289.
Kirk, R. E. (1996). Practical significance: A concept whose time has come.
Educational and Psychological Measurement, 56,
Hedges, L. V., & Becker, B. J. (1986). Statistical methods in the meta-analysis
of research on gender difference. In S. H. Hyde & M. C. Linn (Eds.), The psychology of
17
Meta-analysis 17
gender: Advances through meta-analysis (pp.14-50). Baltimore, MD: John Hopkins
University Press.
Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting
error and bias in research findings. Sage Publications: Newbury Park, CA.
Hyde, J. S. (1986). Introduction: Meta-analysis and the Psychology of Gender.
In S. H. Hyde & M. C. Linn (Eds.),The Psychology of Gender: Advances through meta-
analysis (pp.1-13). Baltimore, MD: John Hopkins University Press.
Kulik, J. & Kulik, C. (1992). Meta-analysis: Historical origins and
contemporary practice. In B. Thompson (Ed.), Advances in social science methodology
(Vol. 2, pp. 53-80). Greenwich, CT: JAI Press.
McLean, J. E., & Ernest, J. M. (in press). The role of statistical significance
testing in educational research. Research in the Schools.
McNamara, J. F. (1997). Meta-Analysis: A strategy to improve integrative
research reviews. International Journal of Educational Reform, 6, 490-495.
McNamara, J. F. (in press). Conducting your first meta-analysis: An illustrated
guide. International Journal of Education Reform.
Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald,
and the slow progress of soft psychology. Journal of Consulting and Clinical
Psychology, 46, 806-834.
Nix, T. W., & Barnette, J. J. (in press). The data analysis dilemma: ban or
abandon. A review of null hypothesis significance testing. Research in the Schools.
Rosenthal, R., & Rubin, D. B. (1982). Comparing results of independent studies.
Psychological Bulletin, 85, 500-504.
18
Meta-analysis 18
Rosenthal, R., & Rubin, D. B. (1986). Meta-analytic procedures for combining
studies with multiple effect sizes. Psychological Bulletin, 99, 400-406.
Schimdt, F.L., & Hunter, J.E. (1997). Development of a general solution to the
problem of validity generalization. Journal of Applied Psychology, 62, 529-540.
Synder, P. & Lawson, S. (1993). Evaluating results using corrected and
uncorrected effect size estimates. Journal of Experimental Education.. 61, 334-349.
Thompson, B. (1996). AERA editorial policies regarding statistical significance
testing: Three suggested reforms. Educational Researcher, 25, 26-30.
Thompson, B. (1997). Computing effect sizes. http://acs.tarnu.eduk-bbt6 147/
Thompson, B. (1998a, April). Five methodology errors in educational research:
The pantheon of statistical significance and other faux pas. Invited address presented at
the annual meeting of the American Educational Research Association, San Diego.
Thompson, B. (1998b). In praise of brilliance: Where that praise really belongs.
American Psychologist, 53, 799-800.
Thompson, B. (1998c). Review of What if there were no significance tests? by
L. Harlow, S. Mulaik & J. Steiger (Eds.). Educational and Psychological
Measurement, 58, 332-344.
Thompson, B. (in press-a). If statistical significance tests are broken/misused,
what practices should supplement or replace them?. Theory & Psychology. [Invited
address presented at the 1997 annual meeting of the American Psychological Association,
Chicago.]
Thompson,.B. (in press-b). Journal editorial policies regarding statistical
significance tests: Heat is to fire as p is to importance. Educational Psychology Review.
19
Meta-analysis 19
Thompson, B. (in press-c). Statistical significance tests, effect size reporting, and
the vain pursuit of pseudo-objectivity. Theory & Psychology.
Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in
measurement error affecting score reliability across studies. Educational and
Psychological Measurement, 58, 6-20.
2 0
U.S. DEPARTMENT OF EDUCATIONOffice of Educational Research and improvement MEM
Educational Rsources tnformation Cntr (ERIC)
REPRODUCTION RELEASE(Specif ic Document)
I. DOCUMENT IDENTIFICATION:
E IC
Tille:
5,144S1 ?Dca SiC Con matsAmorist:
--32YIAL\ rirl.o:hCorporate Source: 4
ftlei-c, na I (3.5 s
REPRODUCTION RELEASE:
Publication Date:
1/99
In order to disseminate as widely as possithe timely ana significant materials ot interest to me educational community. cocumentsannounceo in me monthly aostract lournal of the ERIC system. Resources in Education IRIE1. are usually made available to usersin microfiche. reoroauced paper copy, and electronic:opt:cat mein, and sota througn me ERIC Document Reproduction Service
IEORSI or wrier ERIC venaors. Credit is given to tne source 01 eacti document. ano. It reproduction finesse is grantec, one oithe following notices is affixed to Me document.
II perintssion is grantee to reproduce the identified document. please CHECK ONE of the following options ono sign tne releasebees"
z 00 Semple sticker to be alluded to document Sampie sticker to be affixed to document 0
"PERMISSION TO REPRODUCE THISMATERIAL HAS BEEN GRANTED BY
JENNY MOORE
TO THE EDUCATIONAL RESOURCESINFORMATION CENTER fERICI:'
"PERMISSION TO REPRODUCE THISMATERIAL IN OTHER THAN PAPER
ZOPY HAS BEEN GRANTED BY
.ANte_ftiv5
TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC):*
Wiwi 2
or here
Permittingreproaucsonin otner manpatter cobv.
Sign Here, PleaseDocuments will be oroceSsed as indicated provideo repioduction auality permits. If permission to reproduce is granted, but
neither oot is checeeo, aocuments will be processea at Level 1.
nerepy grant to the Educational Resources information Center (ERIC) nonexclusive permission to reoroduce trio document asinocateo soave. Reomouction from me ERIC microfiche or electronicioottcal meals Idy persons other man ERIC emPonvnen ancrItssystem contractors mores permission from me copyright holder. Exception is mace tor non-prom reproduction by homes and otherservice agencies to sausty information neeos ot educators in response to discrete mouiries:'