Page 1
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved.
Educational Research: Fundamentals for the Consumer
Woolfolk / PerryChild and Adolescent Development © 2012 Pearson Education, Inc. All rights reserved.
Sixth Edition
Page 2
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved.
The Intelligent Consumer: Putting It All Together
Chapter 14
Page 3
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 3
Discussion Topics
The purpose of the text Questions for quantitative studies Questions for qualitative studies Questions for mixed-method studies Reviewing two studies
Page 4
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 4
Text Purpose
Present and explain the fundamental principles of educational research• The quality of published educational research varies
greatly• There is a need to have the knowledge and skills to
critically evaluate educational research Rephrasing the consumer tips from each
chapter into questions to provide guidelines for the evaluation of research articles
Page 5
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 5
Quantitative Criteria
Topics for evaluation questions for quantitative studies (see pages 380 to 382 for a full list of questions)• Research problem• Review of the literature• Research hypotheses• Selection of participants• Instrumentation• Design• Results and analysis• Discussion and conclusions
Page 6
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 6
Questions for Quantitative Studies
Research problem• Is the problem researchable?• Is the problem significant? Will the results
have practical or theoretical importance?• Does the problem indicate the variables and
population studies?
Page 7
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 7
Questions for Quantitative Studies
Review of the literature• Does the review of literature seem
comprehensive? • Is the review up-to-date?• Have studies been critically reviewed, flaws
noted, and results summarized?• Does the review explicitly relate previous
studies to the problem?• If appropriate, does the review establish a
basis for research hypotheses?
Page 8
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 8
Questions for Quantitative Studies
Research hypotheses• Is the hypothesis stated in declarative form?• Does the hypothesis follow from the
literature?• Does the hypothesis state expected
relationships or differences?• Is the hypothesis testable?• Is the hypothesis clear and concise?
Page 9
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 9
Questions for Quantitative Studies
Selection of participants• Are the participants clearly described?• Is the population and sampling clearly
defined?• What is the return rate in a survey study?• Are volunteers used?• Is there an adequate number of participants?
Page 10
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 10
Questions for Quantitative Studies
Instrumentation• Is evidence for validity and reliability clearly
stated and adequate? • If an instrument is designed for a study by the
researchers, is there a description of its development?
• Are the procedures for gathering data clearly described?
• Do the scores distort the reality of the findings?• Do response sets or faking influence the
results?
Page 11
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 11
Questions for Quantitative Studies
Design - nonexperimental• Do graphic presentations distort the findings?• Are causative conclusions reached from
correlational findings?• Is the correlation affected by restriction in the range
and reliability of the instruments?• If causal-comparative, has the causal condition
already occurred? How comparable are the subjects in the groups being compared?
Page 12
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 12
Questions for Quantitative Studies
Design - experimental• Is there direct manipulation of an
independent variable?• Is the design clearly described?• What extraneous variables are not
controlled in the design?• Is each replication of the treatment
independent of the other replications?
Page 13
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 13
Questions for Quantitative Studies
Single subject• Is the measurement of the target behavior reliable?• Is the target behavior clearly defined?• Are there enough measures of the behavior to
establish stability?• Are procedures, subjects, and settings described in
detail?• Are there experimenter or observer effects?
Page 14
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 14
Questions for Quantitative Studies
Results• Is there an appropriately descriptive statistical
summary?• Is statistical significance confused with practical
significance?• Is statistical significance confused with internal or
external validity?• Are appropriate statistical tests used?• Is there a sufficient number of subjects to give valid
statistical results?• Are data clearly and accurately presented in graphs
and tables?
Page 15
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 15
Questions for Quantitative Studies
Discussion and conclusions• Are the results discussed in relation to previous
research, methodology, and the research problem?• Do the conclusions follow from the interpretation of
the results? • Are the conclusions appropriately limited by the
nature of the subjects, treatments, and measures?• Are the limitations of the findings reasonable? • Are the recommendations and implications specific?
Page 16
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 16
Qualitative Criteria
Topics for evaluation questions for qualitative studies (see pages 382 and 383 for a full list of questions)• Introduction and problem• Review of the literature• Methodology• Results and Analysis• Discussion and conclusions
Page 17
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 17
Questions for Qualitative Studies
Introduction and problem• Are the researcher’s background, interests,
and potential biases clear from the outset?• Is the problem significant?• Is there a clear conceptual and theoretical
framework for the problem?• Does the introduction include an overview
of the design of the study?• Is the purpose of the study clearly stated?
Page 18
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 18
Questions for Qualitative Studies
Review of the literature• Does the review indicate that the researcher is
knowledgeable about previous work in the area?• Is the review up-to-date?• Does the review establish an adequate background
and theoretical framework for the study?• Is the review well organized?• Is the literature analyzed as well as summarized?
Page 19
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 19
Questions for Qualitative Studies
Methodology• Will the participants selected provide a
credible answer to the research question?• Will the researcher’s involvement affect the
findings?• How detailed are the field notes?• Are multiple methods of data collection used?• Is the study long enough?
Page 20
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 20
Questions for Qualitative Studies
Discussion and conclusions• Are descriptions clearly separate from
interpretations and researcher’s opinions?• Is the credibility of the findings addressed?• Are the results discussed in relation to previous
research?• Do the conclusions follow from the interpretation of
the results? Are the conclusions consistent with what is known from previous research?
• Are appropriate limitations indicated?• Are appropriate recommendations and implications
indicated?
Page 21
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 21
Mixed-method Criteria
Topics for evaluation questions for qualitative studies (see pages 384 and 385 for a full list of questions)• Introduction and problem• Review of the literature• Selection of participants• Instrumentation• Design• Results• Discussion and conclusions
Page 22
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 22
Questions for Mixed-method Studies
Research problem• Is the problem significant? Will the results
have practical or theoretical importance?• Are there separate questions for each part
of the study?• Is it clear if the study is primarily
quantitative, qualitative, or convergent?
Page 23
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 23
Questions for Mixed-method Studies
Review of the literature• Does the review of literature seem
comprehensive? • Is the review up-to-date?• Have studies been critically reviewed, flaws
noted, and results summarized?• Does the review establish an adequate
background and theoretical framework for the study?
Page 24
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 24
Questions for Mixed-method Studies
Selection of participants• Are the participants clearly described?• Is it likely that the participants will provide
credible data?• Are the participants likely to be biased
Page 25
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 25
Questions for Mixed-method Studies
Instrumentation• Is evidence for validity and reliability clearly
stated and adequate? • Are the procedures for gathering data clearly
described?• Is there likely to be bias from participants,
interviewers, and/or observers?• Are multiple methods of data collection used?
Page 26
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 26
Questions for Mixed-method Studies
Design• Is the design explanatory, exploratory, or
convergent?• Is the design clearly described?• Is the quantitative part nonexperimental or
experimental? How do you know?• Is it clear how the research gained entry in to the
field and his or her role?• Is the design likely to provide credible information?
Page 27
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 27
Questions for Mixed-method Studies
Results• Are data clearly and adequately described?• Is statistical significance confused with
practical significance?• Are verbatim transcriptions and quotations
used?• Are appropriate statistical tests used?• Are results clearly separate from researcher
opinion?
Page 28
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 28
Questions for Mixed-method Studies
Discussion and conclusions• Are the results discussed in relation to
previous research, methodology, and the research problem?
• Do the conclusions follow from the interpretation of the results?
• Are the limitations of the findings indicated?
• Are the recommendations appropriate?
Page 29
McMillanEducational Research: Fundamentals for the Consumer, 6e© 2012 Pearson Education, Inc. All rights reserved. 29
Evaluation of Studies
Evaluation of a qualitative research article• Article 1 – How Elementary School Counselors Can
Meet the Needs of Students with Disabilities by Frye, H.N.
Evaluation of a quantitative, experimental research article• Article 2 – The Effect of Reciprocal Peer Counseling
in the Enhancement of Self-Concept Among Adolescents by Egbochuku, E.O., & Obiunu, J.J.