Educational Research: Educational Research: Causal- Causal- Comparative Studies Comparative Studies EDU 8603 Educational Research Richard M. Jacobs, OSA, Ph.D.
Jan 17, 2016
Educational Research: Educational Research: Causal-Comparative StudiesCausal-Comparative Studies
EDU 8603
Educational Research
Richard M. Jacobs, OSA, Ph.D.
Research...Research...
The systematic application of a family of methods employed to provide trustworthy information about problems…an ongoing process based on many
accumulated understandings and explanations that, when taken together lead to generalizations about problems and the development of theories
The basic steps of research...The basic steps of research...
Scientific and disciplined inquiry is an orderly process, involving:
description and execution of procedures to collection information (“methodmethod”)
objective data analysisanalysis statement of findings (“resultsresults”)
recognition and identification of a topic to be studied (“problemproblem”)
Research methods...Research methods...
QuantitativeQuantitative……collects and analyzes numerical datadata
obtained from formal instrumentsinstruments
Quantitative methods...Quantitative methods...
descriptive research (“survey research”)correlational researchcausal-comparative research
(“ex post facto research”)experimental research
causal-comparative research (“causal-comparative research (“ex post ex post factofacto research”) research”) …at least two different groups are
compared on a dependent variabledependent variable or measure of performance (called the “effect”) because the independent variableindependent variable (called the “cause”) has already occurred or cannot be manipulated
Research variables...Research variables...
IndependentIndependent……an activity of characteristic
believed to make a difference with respect to some behavior
…(syn.) experimental variable, cause, treatment
dependent variabledependent variable……the change or difference occurring
as a result of the independent variable
…(syn.) criterion variable, effect, outcome, posttest
A causal-comparative study…A causal-comparative study…
…a study in which the researcher attempts to determine the cause, or reason, for pre-existing differences in groups of individuals
…called an “ex post facto” study because both the effect and the alleged cause have already occurred and must be studied in retrospect
Differences in causal-comparison Differences in causal-comparison and correlational studies…and correlational studies…
causal-comparative studies……attempt to identify cause-effect
relationships correlational studies…
…attempt to identify relationships
causal-comparative studies……involve two (or more) groups and one
independent variable correlational studies…
…typically involve two (or more) variables and one group
causal-comparative studies……involve making comparisons
correlational studies……involve establishing relationships
Differences in causal-comparison Differences in causal-comparison and experimental studies…and experimental studies…
causal-comparative studies……individuals are not randomly selected but
selected because they belong to groups experimental studies…
…individuals are randomly selected and assigned to two (or more) groups
causal-comparative studies……the researcher cannot manipulate the
independent variable experimental studies…
…the researcher manipulates the independent variable
causal-comparative studies……the independent variable has already
occurred and cannot be manipulated experimental studies…
…the researcher manipulates the independent variable to determine its effects
causal-comparative studies……the random sample is selected from two
already-existing populations experimental studies…
…the random sample is selected from a single population
Conducting a causal-comparative Conducting a causal-comparative study…study…
1. select the problem2. select participants and instrument3. design and procedure4. data analysis and interpretation
1. select the problem……the researcher starts with an effect
and seeks its causes
…the independent variable cannot or should not be manipulated
2. select the participants and instrument……select samples representative of
their respective populations and similar with respect to critical variables other than the independent variable
…called “comparison groupscomparison groups”
3. design and procedure……the performance of the groups is
compared using some valid dependent variable measure (“instrumentinstrument”)
…lack of randomization, manipulation, and control are sources of weakness
controlcontrol…the process by which the researcher
attempts to ensure that the findings are as free of researcher bias and error as possible
types of controltypes of control…random assignment of participants to
groups…pair-wise matching…comparing homogeneous groups…comparing homogeneous subgroups…factorial analysis of variance…analysis of covariance
random assignment of participants to random assignment of participants to groupsgroups
…not possible in causal-comparative studies because the groups already exist and have already received the treatment
pair-wise matchingpair-wise matching…first: find a participant in the second (third,
fourth, etc.) group with the same or similar score on the control (nonmanipulated) variable as the participant in the first group
…second: if a participant in either group does not have a suitable match, the participant is eliminated from the study
comparing homogeneous groupscomparing homogeneous groups…control for extraneous variables that
are homogeneous with respect to the extraneous variables
…limitation: lowers the number of participants in the study and, of course, limits the generalizability of the findings
comparing homogeneous subgroupscomparing homogeneous subgroups…form subgroups within each group that
represent all levels of the control (nonmanipulated) variable
…controls for the variable and also permits the researcher to determine whether the independent variable affects the dependent variable differently at different levels of the control (nonmanipulated) variable
factorial analysis of variance factorial analysis of variance (“FANOVA”)(“FANOVA”)…building the control (nonmanipulated)
variable into the research design …then use FANOVA to analyze the results
to determine the effect of the independent and control (nonmanipulated) variable on the dependent variable, both separately and in combination
…FANOVA allows the researcher to determine if there is an interaction between the independent variable and the dependent variable such that the independent variable operates differently at different levels of the independent variable building it into the research design
analysis of covariance (“ANCOVA”)analysis of covariance (“ANCOVA”)…statistically adjusts initial group
differences on a dependent variable for initial differences on some other variable related to performance on the dependent variable
…removes initial differences so that the results can be fairly compared as if the two groups started equally
symbolic representation of the basic symbolic representation of the basic causal-comparative designcausal-comparative design
Independent DependentGroup Variable Variable
(E) (X) O (C) O
Where: E (experimental group); C (control group); X (independent variable); O (dependent variable)
Independent DependentGroup Variable Variable
(E) (X1) O (C) (X2) O
Where: E (experimental group); C (control group); X (independent variable); O (dependent variable)
4. Data analysis and interpretation……researcher uses a variety of
descriptive and inferential statistics:
meanmeanstandardstandard deviationdeviation
t-testt-testanalysis of analysis of variancevariancechi squaredchi squared
meanmean…the descriptive statistic indicating the
average performance of an individual or group on a measure of some variable
standard deviationstandard deviation…the descriptive statistic indicating the
spread of a set of scores around the mean
t-testt-test…the inferential statistic indicating
whether the means of two groups are significantly different from one another
analysis of variance (“ANOVA”)analysis of variance (“ANOVA”)…the inferential statistic indicating the
presence of a significant difference among the means of three or more groups
chi squared (chi squared (ΧΧ22))…the inferential statistic indicating that
there is a greater than expected difference among group frequencies
Mini-Quiz…Mini-Quiz…
True and false…
…causal-comparative studies attempt to identify the cause-effect relationships; correlational studies do not
True
…causal-comparative studies typically involve two (or more) groups and one independent variable, whereas correlational studies typically involve two (or more) variables and one group
True
…causal-comparative studies involve relation, whereas correlational studies involve cause
False
…oftentimes, causal-comparative research is undertaken because the independent variable could be manipulated but should not
True
…one of the most important reasons for conducting causal-comparative research is to identify variables worthy of experimental investigation
True
…“lack of control” means that the researcher can and should manipulate the independent variable
False
…each group in a causal-comparative study represents a different population
True
…the more similar two groups are on all relevant variables except the independent variable, the stronger the study is
True
…there is random assignment to treatment groups from a single population in causal-comparative studies
False
…lack of randomization, manipulation of the independent variable, and control are all sources of weakness in a causal-comparative design
True
…matching, comparing homogenous groups or subgroups, and covariate analysis are strategies that enable researchers to overcome problems of initial group differences on an extraneous variable
True
…interpretation of the findings in a causal-comparative study requires considerable caution because the cause may be the effect and the effect may be the cause
True
…extraneous variables or confounding factors may be the real “cause” of both the independent and dependent variables
True
Fill in the blank…
…groups selected for a causal-comparative study which differ on some independent variable and comparing them on some dependent variable
comparison groups
Fill in the blank…
…unexplained variables that influence a dependent variable
confounding factors
extraneous variables
Fill in the blank…
…a method for controlling extraneous variables by comparing groups that are homogeneous with respect to the extraneous variable
comparing homogeneous groups
Fill in the blank…
…a method for controlling extraneous variables by forming subgroups within each group that represent all levels of the control variable
comparing homogeneous subgroups
Fill in the blank…
…a statistical tool to determine the effects of the independent variable and the control variable on the dependent variable, both separately and in combination
factorial analysis of variance
Fill in the blank…
…a statistical tool to adjust initial group differences on variables
analysis of covariance
Fill in the blank…
…the descriptive statistic indicating the average performance of a group on a measure of some variable
mean
Fill in the blank…
…the descriptive statistic indicating how clustered or spread out around the mean a set of scores is
standard deviation
Fill in the blank…
…the inferential statistic determining whether there is a significant difference between the means of two groups
t-test
Fill in the blank…
…the inferential statistic determining whether there is a significant difference between the means of three or more groups
analysis of variance
Fill in the blank…
…the inferential statistic determining whether there is a greater than expected difference among group frequencies
chi squared
Fill in the blank…
…activities by which a researcher endeavors to ensure that the results of a causal-comparative study are not tainted by extraneous variables
control
This module has focused on...This module has focused on...
…which identify the cause, or reason, for existing differences in the behavior or status of groups
causal-comparative studies
The next module will focus on...The next module will focus on...
...which test hypotheses to establish cause-and-effect relationships
experimental studies