Group Discussion Explain the difference between assignment bias and selection bias. Which one is a threat to internal validity and which is a threat to.

Group DiscussionExplain the difference between assignment bias and selection bias. Which one is a threat to internal validity and which is a threat to external validity?

A researcher wants to evaluate the effectiveness of a new teaching method. He randomly selects two 6th grade classes in Long Beach. One class is taught with the traditional method (Method A) and another class is taught using the new method (Method B). Both classes are taught by the same teacher, using the same pretest and posttest. Explain what type of research is this and what are the possible external and internal threats to validity of this research? Extra credit

Research Strategies

Chapter 6

Research steps

Introduction & Literature Review• Identify a new idea for research, • Form a hypothesis and a prediction,Methodology • Define and measure your variables, • Sampling • How to treat them ethically, • Select a research strategy

ResearchTypes, Strategies, Designs, and

Procedures

types, srategies, designs, and procedures

• The process of developing a research study can be broken down into three distinct stages: determining a research type, research strategy, determining a research design, and determining research procedures.

Definitions of Quantitative and Qualitative Research(Creswell)

Quantitative Research A type of research in which the

researcher decides what to study; asks specific, narrow questions; collects quantifiable data from participants; analyzes these numbers using statistics; and conducts the inquiry in an unbiased, objective manner.

Qualitative Research A type of research in which the

researcher relies on the views of participants; asks broad, general questions; collects data consisting largely of words (or text) from participants; describes and analyzes these words for themes; and conducts the inquiry in a subjective, biased manner.

Research Strategies

The 5 basic research strategies are • Experimental strategy• Quasi- experimental strategy• Non-experimental strategy • Correlational strategy • Descriptive strategyThese are the ones that you report in your research report

Research Design,It is not enough to say the design is experimental or correlational

•Group study versus individual study (case study)•Same individuals (within subjects) versus different individuals (between subjects)•The number of variables to be included

Research Procedures

• Exactly how the variables will be manipulated, regulated, and measured.

Comparing traditional and constructivist teaching

• Exactly how many individuals will be involved. Sampling method, random assignment? , how many in each group, how many at the end?

• Exactly how the individual participants or subjects will proceed through the course of the study.

How long, how intensive, any initial instructions, any extra resources, content of instructions, tools, surveys, tests…

Internal and external validity

• Internal validity of a research study is the degree to which the study accurately answers the question it was intended to answer.

• External validity refers to the extent to which we can generalize the results of a research.

Threats to external validity

Threats to external validity• There are at least three different kinds of

generalization, and each can be a concern for external validity.

• Generalization from a sample to the general population.

Your sample is not good

• Generalization from one research study to another one.

The tools or procedures you used in your research are not good

• Generalization from a research study to a real world situation.

It worked because you controlled so many variables in real world it may not work.

Category 1: Generalizing Across Participants or Subjects

Selection bias 1. Local samples2- College students3- Volunteer bias4- Participants characteristics (gender, age)

Category 2: Generalizing Across Features of a Study

• Novelty effect (using a new method)• Multiple treatment interference (traditional then

tech)• Experimenter characteristic (gender, race,

personality, ability)

Category 3: Generalizing Across Features of the Measures

• Sensitization ( the treatment works only if subjects are pre-tested, it is the measurement that works not the

treatment)• Measurement (the results of the study may be limited to

a specific measurement)

• Time of measurement (how long after treatment it is

measured)

Threats to internal validity

A. Extraneous Variables

Any variable in a research study other than the specific variables being studied is an extraneous variable.Every research study has thousands of them.

lighting, temperature, time of the day, day of the week, time in a semester, color of the room, material used in the furniture, gender of the researcher, …… etc.

B. Confounding variables

is an extraneous variable ( usually unmonitored) that changes systematically along with the two variables being studied.

•for example, one group may be tested in the morning and the other group in the afternoon when they are tired. •Part of the study may be conducted on a dark and cloudy Monday and another part on a sunny Tuesday.

C. Time- Related Variables

1- History (what happens between pre-post)2- Maturation (Children, can change in a

relatively short time).3- Instrumentation (fix some errors in the

pretest- the researcher doing the observing may become more proficient in making the observations in the posttest)

C. Time- Related Variables

4- Testing effects ( practice, fatigue, and carry- over effects)

C. Time- Related Variables

5- Regression toward the mean: (extreme scores on one measurement tend to be less extreme on a second measurement)

D. Other Threats

• Compensatory equalization• Compensation rivalry• Resentful demoralization• Subjects mortality

Threats

Compensatory equalization; the comparison group is somehow compensated by

others in order to make up for the fact that they are not receiving a treatment

Goal: effectiveness of access to science lab

Compensation: the other group uses a simulation

Threats

Compensation rivalrywhen the comparison group knows what the program

group is getting and therefore, develops a competitive attitude with them.

John Henry effect--John Henry was a steel-driver who outperformed a machine (steam-hammer) under an experimental setting because he was aware that his performance was compared with that of a machine.

Threats

• Resentful demoralizationwhen the comparison group knows what the program

group is getting and get discouraged or angry and withdraw from the study.

• Subjects mortality

Threats to Both Internal and External Validity

• Experimenter bias (results obtained in a study may be specific to the experimenter who has the expectations)

• Demand characteristics and participant reactivity (cues given about the hypothesis & 4 types of

human subjects)• Exaggerated Variables (70 degrees vs. 120 degrees

classrooms)