Top Banner
Methodology Experiments
38

Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

Dec 28, 2015

Download

Documents

Damian Harrison
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

Methodology

Experiments

Page 2: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Methodology: methods are procedures of gathering information

• Theory: an integrated set of principles that explain and predict observed events

Characteristics of a good theory:– Simplicity– Comprehensiveness– Generativity

Page 3: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Hypothesis: a testable proposition that describes a relationship that may exist between events

• Constructs: abstract concepts in a theory (conceptual variables)

• Variables: measurable representation of construct (operational definition)

Page 4: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

Methods of selecting data

• Observation– Participant observation– Hawthorne effect

• Self-report methods– Questionnaire– Interview– Social desirability bias

• Archival research

Page 5: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Survey• Representative sample• Sampling: the process of selecting a subset of numbers of

a population with a view to describing the population from which they are taken

• Probabilistic sampling: random sampling where each number of the population has an equal chance of being selected and the selection of every possible combination of the desired number of members is equally likely

Page 6: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Non-probability sampling: quota sampling where we select a sample that reflects the basic attributes of the population

• Biasing influences in surveys:– Unrepresentative samples– Order of questions– Response options– Question wording

Page 7: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

Experimental method

• One or more factors (independent variables) are systematically changed to determine whether such variations affect one or more other factors (dependent variables)

• High degree of control over the research setting

• CAUSE AND EFFECT

Page 8: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Experimental design refers to how participants are allocated to the different conditions (or IV groups) in an experiment.

• Probably the commonest way to design an experiment in psychology is to divide the participants into two groups, the experimental group and the control group, and then introduce a change for the experimental group and not the control group.

Page 9: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Need to consider the effect of extraneous variables-variables other than the independent variables that may influence the dependent variable

• E.g. Hawthorne effect• Volunteer bias

• The key to the success of the experimental method is to keep all extraneous variables constant

Page 10: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Some extraneous variables may be controlled by the random assignment of participants to the experimental and control group-a method of assigning subjects to groups by chance to control for the effects of extraneous variables

Page 11: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• The researcher must decide how he/she will allocate their sample to these IVs.

• For example, if there are 10 participants, will all 10 participants take part in both conditions (e.g. repeated measures) or will the participants be split in half and take part in only one condition each?

Page 12: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

1. Independent Measures

• Different participants are used in each condition of the independent variable. This means that each condition of the experiment includes a different group of participants. This should be done by random allocation, which ensures that each participant has an equal chance of being assigned to one group or the other.

• Independent measures involves using two separate groups of participants; one in each condition.

Page 13: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.
Page 14: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

2. Repeated measures

• The same participants take part in each condition of the independent variable. This means that each condition of the experiment includes the same group of participants.

Page 15: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.
Page 16: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

3. Matched Pairs

• One pair must be randomly assigned to the experimental group and the other to the control group.

Page 17: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.
Page 18: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• So, three experimental designs are commonly used:• Independent groups: Testing separate groups of people,

each group is tested in a different condition.• Repeated measures: Testing the same group of people in

different conditions, the same people are used repeatedly.• Matched pairs: Testing separate groups of people - each

member of one group is same age, sex, or social background as a member of the other group.

• In each case, there are one or more experimental groups, where the independent variable has changed and a control group where the independent variable has not changed.

Page 19: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

Variables

• Independent variable (IV): Variable the experimenter manipulates - assumed to have a direct effect on the dependent variable.

• Dependent variable (DV): Variable the experimenter measures, after making changes to the IV which are assumed to affect the DV.

• Extraneous variables (Ex Vs): Other variables, apart from the IV, that might affect the DV. They might be important enough to provide alternative explanations for the effects, for example, confounding variables.

Page 20: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

Types of experiments

• Laboratory experiment: Artificial environment with tight controls over variables.

• Field experiment: Natural environment with independent variable manipulated by researchers.

• Natural experiment: Natural changes in independent variable are used - it is not manipulated.

Page 21: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• By definition, all experiments involve manipulation of one or more independent variables, and observing the effect on some outcome (dependent variable). Another factor that distinguishes the type of experiment is the degree to which the experimenter can assign subjects to conditions. Three types are described here: True, Quasi- and Single-subject experiments.

Page 22: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

True experiments

• In a true experiment, subjects are randomly assigned to the treatment conditions (levels of the independent variable). The only differences in the groups would be due to chance.

• True experiments are excellent for showing a cause-and-effect relationship. Random assignment (or random assignment within matched groups) controls for extraneous variables.

• They tend to be high on internal validity. It is clear what is being measured. There still might be bias in the overall research design, but at least variables associated with individuals are not a source of constant error

Page 23: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

Quasi-experiments

• Quasi-experiments are sometimes called natural experiments because membership in the treatment level is determined by conditions beyond the control of the experimenter (subjects are already in the box). An experiment may seem to be a true experiment, but if the subjects have NOT been randomly assigned to the treatment condition, the experiment is a quasi- experiment (quasi = seeming, resembles).

Page 24: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Experiments that take advantage of natural occurrences are quasi-experiments, for example, comparing achievement level of first-born children with that of later-born children; or comparing student performance at two schools, one of which has a lower student-teacher ratio. The experimenter is unable to assign subjects to treatment level - the subjects are already in pre-existing groups.

Page 25: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

Single subject experiments

• Instead of comparing behavior or performance of groups of people at a single point in time, a single-subject experiment involves a single case studied over a longer period of time. One individual or situation is exposed to the varying levels of the independent variable.

Page 26: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• The most simple single-subject research design is termed ABA, where A is the baseline (non-treatment or control) condition or phase. B refers to the introduction of the treatment factor. Behavior is recorded in both stages. Then there is a return to A to see if in fact it was B that brought about the change.

• An example might be treating a hyperactive child with a drug. Stage A involves recording the child's behavior before any treatment, e.g., how many disruptive events in the classroom within a specified period of time. Stage B would involve the same measurement after the child has been treated. If B (the treatment) makes a difference, returning to level A (no treatment) should result in a return of the disruptive behavior. The basic research design can include a second treatment phase -- ABAB, thereby increasing the reliability and internal validity of the results.

Page 27: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Requirements for experimental designs:• Control over procedures (confounding variables)• Random assignment• Validity requirements

Page 28: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Internal Validity: the degree to which there can be reasonable certainty that the independent variables in an experiment caused the effects obtained on the dependent variables Threat is confounding results

• Internal validity refers to whether the effects observed in a study are due to the manipulation of the independent variable and not some other factor. In-other-words there is a causal relationship between the independent and dependent variable.

• Internal validity can be improved by controlling extraneous variables, using standardized instructions, counter balancing, and eliminating demand characteristics and investigator effects.

Page 29: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Construct validity: variables adequately capture the constructs they are supposed to represent. Threats are demand characteristics (cues in the setting that give out what we are looking for) and experimenter effects

Page 30: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• External validity: the degree to which there can be reasonable confidence that the results of a study would be obtained for other people and in other situations. Threat is volunteer bias

• External validity refers to the extent to which the results of a study can be generalized to other settings (ecological validity), other people (population validity) and over time (historical validity).

• External validity can be improved by setting experiments in a more natural setting and using random sampling to select participants.

Page 31: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Mundane realism: the degree to which the experimental situation resembles places and events in the real world

• Experimental realism: degree to which experimental procedures are involving to participants and lead them to behave naturally and spontaneously

Page 32: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

Validity of a test

Page 33: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Face Validity• This is the least sophisticated measure of

validity. Face validity is simply whether the test appears (at face value) to measure what it claims to. Having face validity does not mean that a test really measures what the researcher intends to measure, but only in the judgment of raters that it appears to do so. Consequently it is a crude and basic measure of validity.

Page 34: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Construct Validity• This type of validity refers to the extent to which a test

captures a specific theoretical construct or trait, and it overlaps with some of the other aspects of validity. To test for construct validity it must be demonstrated that the phenomenon being measured actually exists. So, the construct validity of a test for intelligence, for example, is dependent on a model or theory of intelligence. Construct validity entails demonstrating the power of such a construct to explain a network of research findings and to predict further relationships.

Page 35: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• In concurrent validity, we assess the operationalization's ability to distinguish between groups that it should theoretically be able to distinguish between. For example, if we come up with a way of assessing manic-depression, our measure should be able to distinguish between people who are diagnosed manic-depression and those diagnosed paranoid schizophrenic

Page 36: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Convergent validity• we examine the degree to which the operationalization is

similar to (converges on) other operationalizations that it theoretically should be similar to

• This is the degree to which a test corresponds to an external criterion that is known concurrently (i.e. occurring at the same time). If the new test is validated by a comparison with a currently existing criterion, we have concurrent validity. Very often, a new IQ or personality test might be compared with an older but similar test known to have good validity already.

Page 37: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• In discriminant validity, we examine the degree to which the operationalization is not similar to (diverges from) other operationalizations that it theoretically should be not be similar to.

Page 38: Methodology Experiments. Methodology: methods are procedures of gathering information Theory: an integrated set of principles that explain and predict.

• Predictive validity• This is the degree to which a test accurately

predicts a criterion that will occur in the future. For example, a prediction may be made on the basis of a new intelligence test, that high scorers at age 12 will be more likely to obtain university degrees several years later. If the prediction is born out then the test has predictive validity.