Research Design
Dec 13, 2015
research design
a methodology – to establish a sequence for
measurement,– to reduce confounds, and – to increase control
indicates the basic structure of the study
A research plan is
a detailed description of a proposed study designated to investigate a specific problem.
• (Gay, 83).
As used in research, the term “design” refers to the researcher’s plan for how to proceed.
We said previously the distinction between good and poor research, more often than any other factor, is the distinction between asking a good or poor research question.
The success, however, is often a function of the
extent to which the methods of endeavor have
been subjected to quantification through the use of measurement and
mathematics.
Measurement
systematic representation of the data by numbers
objectives– objectivity– assignment of numbers to data
according to well-established rules
Mathematics
the systematic representation of the relationships among the measurements with mathematical expressions
a universal language that any science or technology may use with power and convenience.
the vocabulary of mathematics is – unlimited– defined with rigorous accuracy
Contributors to the nature of the design
nature of the hypothesis the variables involved
– constructs constraints of the “real world”
Constructs
Traits such as intelligence, learning, anxiety, motivation, are not directly observable.
Referred to as constructs -- implying constructs of the scientist’s imagination
Constructs cannot be seen heard or felt. Can only be inferred by phenomenon such
as test scores, pulse rate, observed aggressive acts, etc.
• Best, J. & Kahn, J. (1998). Research in Education . Boston: Allyn and Bacon.
Constraints of the Real World: Problems in Conducting Research With Humans
No two persons are alike. No one person is completely
consistent from moment to moment. Humans are influenced by the
research process itself. The behavioral sciences have been
limited by the use of constructs.• Best, J. & Kahn, J. (1998). Research in Education . Boston: Allyn and Bacon.
Whether conducting research or evaluating the reporting of research findings, we want to be reasonably sure of two things:
to understand what influence, if any, the independent variable has on the dependent variable, and
to feel comfortable that the experimental environment sufficiently mirrors the real world to allow for generalizability of findings.
Internal and External
Internal– The differences are a result of the
manipulation of the independent variable.
External– The results of the study can be
expected to confirmed with similar studies of other groups
Threats to Internal Validity-- Campbell and Stanley
history maturation testing instrumentation statistical regression differential selection of subjects mortality selection-maturation interaction
Threats to External Validity
Pretest-Treatment Interactions Multiple Treatment Interference Selection-Treatment Interaction Specificity of Variables Experimenter Effects Reactive Arrangements
Methods for controlling Methods for controlling effects of variableseffects of variables
Hold the variable constant Randomize the effects of the
variable Match subjects with respect to the
variable Control the variable statistically
Must perform two related functions
must provide a logical structure that enables the researcher to pinpoint the effects of the independent variable on the dependent variable and thus answer the research question
help the researcher to rule out confounds as an alternative explanation for our findings
random selection
– selection of the entire group of participants for an experiment
– Larger samples are generally better– use of a random number generator or
random number table random assignment
– after participants have been selected, they can be assigned to experimental and control groups
Randomization
Dewey wins! All behavioral research should use the
method of randomization. It is the most powerful technique we
have for eliminating unintended participant-assignment confounds from the design of experiments.
Randomization randomizes everything.
selection
sampling– see text
• procedure• types
– simple random sample– stratified random sample– cluster sampling– two-stage cluster sampling
matching
Stratified samples Use of matched pairs Subject participates in each phase
of the experiment. (Could be source of additional threats.)
controlled statistically
If two variables are correlated, one can be used to predict the other:– GPA and SAT
A Way of Looking at Designto Maximize Internal Validity
Between Subjects design
Within Subjects design
Between Subjects Design
separate groups of subject receive different levels of the independent variable.
to control for individual differences:– randomly assigned subjects to groups,
or– match subjects characteristics in each
group
Within Subjects Design
all subjects receive all levels of the independent variable.
to try to minimize carry over effects:– randomize the order of treatments, or– counterbalance the order of
treatments.• each treatment occurs in each time period
of the experiment
Pre-experimental designs
The One Shot Case Study The One Group Pretest-Posttest
Design The Static Group Comparison
Experimental
as determined by Campbell and Stanley participants are randomly assigned to
groups there are at least two levels of the
independent variable the design can control for the major
threats to internal validity the experiment compares two alternative
theoretical positions (optional)
Experimental Designs
the Pretest-Posttest Control Group design
the Posttest Only Control Group design
the Solomon Four Group design
Quasi-Experimental designs
The nonequivalent control group design
The time series design Counterbalanced design Factorial design
Discuss
A newspaper article suggested that people who eat breakfast are healthier than those who do not. When this research was criticized, the researcher replied, “What’s the problem? All of our participants were randomly selected.”
Respond to these researchers.
Question 8aExplain the difference in
pre-experimental, experimental and quasi-experimental research
designs. Why is randomization such an
important factor?