Research Design and Methodology, Dr. W.A. Kritsonis

RESEARCH DESIGN AND METHODS

Research Design

William Allan Kritsonis, PhD

Educational research is the application of the scientific and disciplined

inquiry approach to the study of educational problems and the primary goal

of educational research is to explain and help understand educational issues,

questions, and processes. Secondary goals of educational research are to

help others understand, predict future outcomes, improve future research and

practice, and raise new questions to research.

Scientific and disciplined inquiry approach is distinguished by four

crucial steps. These are: Recognize and identify a question or a problem to

be studied, Describe and execute procedures to collect information about the

questions and problems being studied, Analyze the collected information,

and state the results or implications based on the analysis of the information.

Basic research is conducted to develop or refine theory, not to solve

immediate practical problems. Applied research is conducted to find

solutions to current practical problems. Most of the educational research

would be conducted for and classified towards the applied end of the basic-

applied research continuum. The purpose of evaluation research is to aid

decision making about educational programs and practices.

The general purpose of Quantitative research is to generalize about or

control phenomena. Quantitative research has been conducted since the

early 1920s and is the primary way educational researchers have sought to

bring about an understanding of educational issues. Quantitative methods

involve the collection and analysis of numerical data that is obtained from

test, questionnaires, checklist, and surveys. Important features of

quantitative research are: defining the problem or questions to be studied and

developing a hypotheses that predicts the results of the research before the

study begins; controlling contextual factors that might influence the results

of the study; collecting data from samples of participants; and using

numerical, statistical approaches to analyze the collected data. An important

assumption that underlines the quantitative approach is that the world that

we live in and carry out our research in is relatively stable, uniform, and

coherent; therefore, it can be measured, understood, and classified.

Qualitative research methods involve the collection and analysis of

primarily nonnumerical data obtained from observation, interviews, taped

information, documents. The important points of qualitative research are the

following: defining the problem or question to be studied; the

aforementioned point does not have to be stated at the start of the study. The

researcher also takes into account contextual factors of the research

participants; data is collected from a small number of purposely selected

research participants; and using nonnumerical, interpretive approaches to

provide narrative descriptions of the participants and their contexts. The

underlining belief of qualitative research is that the world is neither stable,

coherent, nor uniform, and therefore, "truth" as sought by quantitative

researchers cannot be obtained because perspectives and understandings

differ from group to group.

Descriptive research and survey research is synonymous in that they

both collect numerical data to answer questions. Most descriptive studies

obtain information about the preferences, attitudes, practices, concerns, or

interest of some group. The data are collected by a self-administered

instruments and or telephone polls. The difficulty in conducting descriptive

research is constructing clear, consistent descriptive instruments and trying

to acquire instruments from interviewees along with cooperation in

telephone interviews.

Correlational research examines the degree of relationship that exist

between two or more variables. A correlation is a quantitative measure of

the degree of correspondence between two or more variables. The degree of

relationship is measured by a correlation coefficient of .00 indicating no

relationship between the variables. The researcher must keep in mind that if

there is a high relationship between variables, it does not mean that one is a

cause of the other.

Causal-Comparative research seeks to investigate cause and effect

relationships between two or more programs, methods, or groups. The

activity thought to make a difference; program, method, or group is called

the causal factor, treatment, or independent variable. The "effect" is called

the dependent variable. In most casual-comparative research studies the

researcher does not have control over the causal factor because it has already

occurred or cannot be manipulated. This makes cause-effect conclusions

tenuous and tentative. Causal-comparative research is useful in those

circumstances when it is impossible or unethical to manipulate the causal

factor.

Experimental research investigates the cause-effect relationship by

selecting participants from a single pool and assigning them at random to

different causal treatments. The researcher controls contextual variables that

might interfere with the study and because of the randomness of selecting

and assigning participants into different treatments cause-effect statements

are considered to be true statements.

Historical research is a form of qualitative research that involves

interpreting past events of individuals, important social issues, links between

the old and the new, and reinterpretations of prior historical works. The

primary sources of data are first-person eyewitnesses or authors. Secondary

sources are non-first person accounts. Historians use external criticism to

assess the authenticity of their data and use internal criticism to assess the

truthfulness of their data.

Qualitative research may include a number of specific methods such

as ethnology, ethnomethodology, case study phenomenology, and symbolic

interaction. These methods focus on the deep descriptions of aspects of

people's everyday perspectives and context. Qualitative research provides

field-focused, interpretive, detailed descriptions and interpretations of

participants and their settings. Long-term immersion into the research

setting is also very common. Common methods of data collection include

observation, interviewing, tape and video recording, examining artifacts, and

participant observation. Data analysis is based on categorizing and

interpreting the observations, conversations with participants, documents,

tape recordings, and interviews collected to provide a description and

explanation of the participants and their experiences. The qualitative

researcher writes from the perspective of the participants.

There are guidelines that a researcher can follow to determine the

research method needed to be used for a particular study. The same general

problem can be investigated using many types of research. The more

information available about the nature and procedures of a study, the easier

to classify it. A structured way to classify a study is to first determine if it is

a quantitative or qualitative study. If a study is quantitative, try to identify

the purpose of the study to determine whether it is descriptive, correlational,

causal-comparative, or experimental. If the study is qualitative, determine

whether it is historical or qualitative. Look for key words in the title of the

study: survey, description, relationship, historical, culture, and the like.

Identifying a research topic may come from three main sources:

theory may be composed of generalizations and concepts, personal

experience, and replication of an existing study. A basic characteristic of a

good research problem is that it is researchable.

The stating of a research topic for a quantitative study generally

indicates variables and a well-written topic that defines relevant variables,

either directly or operationally. The topic statement is the first component in

the introductory section and will provide the direction for the study.

The review of literature involves the systematic identification,

location, and analysis of documents containing information related to the

research topic. The major reasons for the review of literature are to put your

study in context with what information already exist and it also points out

research strategies, procedures, and instruments that may be used.

Identifying your sources of information is the first step in a

comprehensive research project. Most libraries use a computer catalog

system and computer databases such as: ERIC, Educational Index,

Psychological Abstract, and Dissertation Abstracts. The Internet is a

computer link to the world -wide web that can be utilized through search

engines such as Yahoo or Lycos.

Abstracting involves locating, reviewing, summarizing, and

classifying your references. For each reference write a complete

bibliographic record including authors name, date of publication, title,

journal name or book title, volume number, issue number, page number, and

library call number.

Formulation of a statement of a hypothesis is based on the theory or

review of literature. A characteristic of a good hypothesis is that it is based

on sound rational, a reasoned prediction. A good hypothesis states as clearly

and concisely as possible the expected relationship (or difference) between

two variables and defines those variables in measurable terms. A well-stated

and well-defined hypothesis must be testable.

The six types of hypothesis discussed in the Educational Research

book are as follows: An inductive hypothesis is a generalization made from

a number of observations, a deductive hypothesis is derived from theory and

is aimed at providing evidence that supports, expands, or contradicts aspects

of a given theory, A research hypothesis states the expected relationship (or

difference) between two variables, a nondirectional hypothesis indicates that

a relationship or difference exists but does not indicate the difference, a

directional hypothesis indicates that a relationship or difference exist and

indicates the direction of the difference, and a null hypothesis states that

there will be no significant relationship (or difference) between variables.

A research plan is a detailed description of a proposed study. It

includes justification for the study, description of the steps that will be

followed in the study, and information about the analysis of the collected

data. They are always ethical considerations that must be thought of and

there are several sources that a researcher can go to for information. The

National Research Act of 1974 and The Ethical Principles of Psychologists

are two documents that have guidelines as to the rules and regulations that

one should follow while conducting research.

A design is a general strategy for conducting a research study, steps

that will take place and the order that they will take place.

Sampling is the process of selecting a number of individuals for a

study in such a way that the individuals represent the larger group from

which they were selected.

Random sampling is the process by which all individuals in the

defined population have an equal and independent chance of being selected

for the sample. Stratified Sampling is the process of selecting a sample in

such a way that the sample represents an identified subgroup in the same

proportion as they exist in the general population. Cluster Sampling is a

process in which groups, not individuals, are randomly selected. Systematic

Sampling is a process of sampling by which individuals are selected from a

list by taking every Kth name. That is K= number of individuals on the list

divided by the number of participant's need for the sample.

Instruments of measurement are as follows: Constructs are mental

abstractions such as personality, creativity and intelligence that in most

instances cannot be measured directly. Constructs become variables when

different levels or scores can be used to measure the construct.

Measurement scales describe four different levels of measurement: nominal,

which categorize, ordinal which rank, interval variables have equal intervals,

and ratio variables have a defined zero point.

Cognitive Test are as follows: achievement test that measure the

current status of an individual on a particular subject matter, Aptitude test

are used to predict how well an individual is likely to do in future test or

occupations, Affective test measure characteristics such as interest, values,

attitude, and personality. The five basic types of scales used to measure

attitudes are Likert, Semantic, Differential, Rating, Thurstone, and Guttman.

Projective test takes into account an ambiguous situation and is preformed

through the use of association.

Validity is the most important quality of a test. It refers to the degree

to which a test measures what it is suppose to measure. Validation is also a

matter of degree: highly valid, moderately valid, or generally valid. Content

validity is the degree to which a test measures an intended content area.

There is also Criterion -Related Validity that is divided into two forms:

Concurrent validity is the degree to which scores on a test are related to

scores on another test administered at the same time and Predictive validity

which refers to the degree to which scores on a test are related to scores on

another test to be administered in the future.

Reliability refers to the degree to which a test consistently measures

whatever it is suppose to measure. The five general approaches to reliability

are stability, equivalence, equivalence and stability, internal consistency, and

score/rater.

The Standard Error of Measurement is an estimate of how often one

can expect test score errors of a given size.

Qualitative research has characteristics that are common features in

this type of research. There is reliance on interpretation involving intensive

field participation, rich data collected in the field, and the researcher as the

primary synthesizing and interpretive agent of data analysis. Data analysis

typically involves a six-step process. They data managing;

reading/memoing; describing; classifying interpreting; and representing the

results.

Descriptive research involves collecting data in order to test

hypotheses or answer questions about opinions. This can be done through

self-report or through observation and is categorized in terms of cross-

sectional or longitudinal. Categories to consider are: statement of the

problem, selection of participants, construction of the questionnaire,

preparation of the cover letter, pre-testing the questionnaire, follow-up to the

questionnaire, dealing with non-response, and analysis of the results.

Correctional research involves collecting data to determine whether

and to what degree a relationship exists between two or more variables. The

degree of the relationship is expressed as a correlation coefficient. The

process is composed of the following: problem selection, participant and

instrument selection, design and procedure, data analysis and interpretation

Causal-comparative research or ex post facto research is used to

determine the cause or reason for existing differences in the behavior in a

group. The basic design involves selecting two groups differing on some

independent variable and comparing them on some dependent variable.

Data analysis involves a variety of descriptive and inferential statistics.

Experimental research utilizes the manipulation of at least one

independent variable, controls the other independent variables, and observes

the effect on one or more independent variables. The steps in the

experimental research are basically the same as the steps for other types of

research: selection and definition of a problem, selection of participants and

measuring instruments, selection of a design, execution of a procedure,

analysis of data, and formulations of conclusions.

Inferential statistics deals with the inferences about populations based

on the results of samples. This type of research allows the researcher to

generalize to a population of individuals based on the information obtained

from a limited number of research participants. There is an expected

standard error of chance variation among the means. The Null Hypothesis

says that there is no true difference or relationship between parameters in the

population. The levels of significance are four fold and along with this there

are Type I and Type II errors. When the null hypothesis is true and the

researcher rejects it and says there is a difference, he/she makes an incorrect

decision, this is a type I error. If the null hypothesis is false and the

researcher says it is true and does not reject it, the researcher also makes an

incorrect decision referred to as a type II error.

Probably the most foremost rule of research report writing is to relate

aspects of the study in a manner that accurately reflects what was done and

what was found.

GLOSSARY OF RESEARCH-RELATED TERMS

A-B design A single-subject design in which baseline measurements are repeatedly made until stability is presumably established, treatment is introduced, and an appropriate number of measurements are made during treatment.A-B-A design A single-subject design in which baseline measurements are repeatedly made until stability is presumably established, treatment is introduced, and an appropriate number of measurements are made, and the treatment phase is followed by a second baseline phase.A-B-A-B design A single-subject design in which baseline measurements are repeatedly made until stability is presumably established, treatment is introduced, and an appropriate number of measurements are made, and the treatment phase is followed by a second baseline phase, which is followed by a second treatment phase.abstract A summary of a study, which appears at the beginning of the report and describes the most important aspects of the study, including major results and conclusions.accessible population Refers to the population from which the researcher can realistically select participants.accidental sampling See convenience sampling.achievement test An instrument that measures the current status of individuals with respect to proficiency in given areas of knowledge or skill.action research An approach in which teachers study their own problems or concerns in their own classrooms.

additive designs Refers to variations of the AB design which involve the addition of another phase or phases in which the experimental treatment is supplemented with another treatment.alternating treatments design A variation of a multiple-baseline design which involves the relatively rapid alternation of treatments for a single participant.analysis of covariance A statistical method of equating groups on one or more variables and for increasing the power of a statistical test; adjusts scores on a dependent variable for initial differences on some variable such as pretest performance or IQ.analytic induction A method of identifying regularities in qualitative data, determining their explanation, and finding other contexts to determine whether the explanations hold up.applied research Research conducted for the purpose of applying, or testing, theory and evaluating its usefulness in solving problems.aptitude test A measure of potential used to predict how well someone is likely to perform in a future situation.artificial categories Categories which are operationally defined by the researcher.assumption Any important "fact" presumed to be true but not actually verified; assumptions should be described in the procedures section of a research plan or report.

attenuation Refers to the principle that correlation coefficients tend to be lowered because less-than-perfectly reliable measures are used.basic research Research conducted for the purpose of theory development or refinement.case study The in-depth investigation of one "unit," e.g., individual, group, institution, organization, program, or document.category The classification of ideas and concepts in qualitative data analysis.causal-comparative research Research that attempts to determine the cause, or reason, for existing differences in the behavior or status of groups of individuals; also referred to as ex post facto research.census survey Descriptive research that attempts to acquire data from each and every member of a population.changing criterion design A variation of the A-B-A design in which the baseline phase is followed by successive treatment phases, each of which has a more stringent criterion for acceptable behavior level.chi square A nonparametric test of significance appropriate when the data are in the form of frequency counts; it compares proportions actually observed in a study with proportions expected to see if they are significantly different.clinical replication Refers to the development and application of a treatment package, composed of two or more interventions which have been found to be effective individually, designed for persons with complex behavior disorders.cluster sampling Sampling in which intact groups, not individuals, are randomly selected.coefficient alpha (a) See Cronbach's alpha.common variance The variation in one variable that is attributable to its tendency to vary with another variable.concurrent validity The degree to which the scores on a test are related to the scores on another, already established test administered at the same time, or to some other valid criterion available at the same time.constant comparison A qualitative method for identifying similarities and differences by comparing new evidence to prior evidence.construct validity The degree to which a test measures an intended hypothetical construct, or nonobservable trait, which explains behavior.contamination The situation that exists when the researcher's familiarity with the participants affects the outcome of the study.content analysis The systematic, quantitative description of the composition of the object of the study.content validity The degree to which a test measures an intended content area; it is determined by expert judgment and requires both item validity and sampling validity.control Efforts on the part of the researcher to remove the influence of any variable other than the independent variable that might affect performance on a dependent variable.control group The group in a research study that either receives a different treatment than the experimental group or is treated as usual.control variable A nonmanipulated variable, usually a physical or mental characteristic of the participants (such as IQ).

convenience sampling The process of using as the sample whoever happens to be available, e.g., volunteers. (Also referred to as accidental sampling and haphazard sampling.) correlational research Research that involves collecting data in order to determine whether, and to what degree, a relationship exists between two or more quantifiable variables.correlation coefficient A decimal number between .00 and :!:1.00 that indicates the degree to which two variables are related.counterbalanced design A quasi-experimental design in which all groups receive all treatments, each group receives the treatments in a different order, the number of groups equals the number of treatments, and all groups are post tested after each treatment.credibility A term used in qualitative research to indicate that the topic was accurately identified and described.criterion In a prediction study, the variable that is predicted.criterion-related validity Validity which is determined by relating performance on a test to performance on another criterion; includes concurrent and predictive validity.Cronbach's alpha (a) The general formula for estimating internal consistency based on a determination of how all items on a test relate to all other items and to the total test. (Also referred to as coefficient alpha and Cronbach's coefficient alpha.) cross-validation Validation of a prediction equation with at least one group other than the group on which it was based; variables that are no longer found to be related to the criterion measure are removed from the equation.curvilinear relationship A relationship in which increase in one variable is associated with a corresponding increase in another variable to a point, at which point further increase in the first variable is associated with a corresponding decrease in the other variable (or vice versa).data saturation A point in qualitative research when so much data are collected that it is very unlikely that additional data will add to what is already collected.deductive hypothesis A hypothesis derived from theory which provides evidence which supports, expands, or contradicts the theory.dependent variable The change or difference in behavior that occurs as a result of the independent variable; also referred to as the criterion variable, the effect, the outcome, or the post test.descriptive statistics Data analysis techniques enabling the researcher to meaningfully describe many scores with a small number of numerical indices.developmental studies Studies concerned with behavior variables that differentiate children at different levels of age, growth, or maturation.diagnostic test A type of achievement test yielding multiple scores for each area of achievement measured that facilitate identification of specific areas of deficiency.differential selection of participants Refers to the fact that groups may be different before a study even begins, and this initial difference may at least partially account for post test differences.direct replication Refers to the replication of a study by the same investigator, with the same participants or with different participants, in a specific setting.ecological validity The degree to which results can be generalized to environments outside of the experimental setting.

educational research The formal, systematic application of the scientific and disciplined inquiry approach to the study of educational problems.environmental variable A variable in the setting in which a study is conducted that might cause unwanted differences between groups (e.g., learning materials).equivalent forms Two tests identical in every way except for the actual items included.equivalent-forms reliability Indicates score variation that occurs from form to form of a test; also referred to as alternate-forms reliability.ethnographic research A qualitative approach that studies the cultural patterns and perspectives of participants in their natural setting.ethnomethodology A qualitati\Te approach that studies how participants make sense of their everyday activities to act in a social way.evaluation The systematic process of collecting and analyzing data in order to make decislons.experimental group The group in a research study that typically receives a new, or novel, treatment, a treatment under investigation.experimental research Research in which at least one independent variable is manipulated, other relevant variables are controlled, and the effect on one or more dependent variables is observed.experimenter bias A situation in which the researcher's expectations concerning the outcomes of the study actually contribute to producing various outcomes.ex post facto research See causal-comparative research.external criticism The analysis of data to determine their authenticity.external validity The degree to which results are generalizable, or applicable, to groups and environments outside of the experimental setting.factorial analysis of variance The appropriate statistical analysis if a study is based on a factorial design and investigates two or more independent variables and the interactions between them; yields a separate F ratio for each independent variable and one for each interaction.factorial design An experimental design that involves two or more dependent variables (at least one of which is manipulated) in order to studv the effects of the variables individually a~d in interaction with each other.fieldwork A qualitative research strategy that involves spending considerable time in the setting under study, immersing oneself in this setting, and collecting as much relevant information as possible as unobtrusively as possible.follow-up study A study conducted to determine the status of a group of interest after some period of time.generosity error The tendency to give an individual the benefit of the doubt whenever there is insufficient knowledge to make an objective judgment.grounded theory Theory based on data collected in real-world settings which reflect what naturally occurred over an extended period of time.halo effect The phenomenon whereby initial impressions concerning an individual (positive or negative) affect subsequent measurements.haphazard sampling See convenience sampling.hardcopy Refers to computer output that is printed out on paper.hardware Refers to the actual equipment, the computer itself and related accessories such as printers.

Hawthorne effect A type of reactive arrangement resulting from the participants' knowledge that they are involved in an experiment, or their feeling that they are in some way receiving "special" attention.historical research The systematic collection and evaluation of data related to past occurrences in order to describe causes, effects, or trends of those events which may help to explain present events and anticipate future events.history Any event which is not part of the experimental treatment but which may affect performance on the dependent variable.hypothesis A tentative, reasonable, testable explanation for the occurrence of certain behaviors, phenomena, or events.independent variable An activity or characteristic believed to make a difference with respect to some behavior; also referred to as the experimental variable, the cause, and the treatment.inductive hypothesis A generalization based on observation.inferential statistics Data analysis techniques for determining how likely it is that results based on a sample or samples are the same results that would have been obtained for an entire population.instrumentation Unreliability in measuring instruments that may result in invalid assessment of participants' performance.interaction Refers to the situation in which different values of the independent variable are differentially effective depending upon the level of the control variable.interiudge reliability The consistency of two (or more) independent scorers, raters, or observers.internal criticism The analysis of data to determine their accuracy which takes into consideration the knowledge and competence of the author, the time delay between the occurrence and recording of events, biased motives of the author, and consistency of the data.internal validity The degree to \\,hich obse!\'ed differences on the dependent variable are a direct result of manipulation of the independent variable, not some other variable.interval scale A measurement scale that classifies and ranks participants, is based upon predetermined equal intervals, but does not have a true zero point.intervening variable A variable which intervenes between, or alters the relationship between, an independent variable and a dependent variable, which cannot be directly observed or controlled (e.g., anxiety) but which can be controlled for.intrajudge reliability The consistency of the scoring, rating, or observing of an individual.item validity The degree to which test items represent measurement in the intended content area.John Henry effect The phenomenon whereby if for any reason members of a control groupfeel threatened or by being in competition with an experimental group, they may out do themselves and perform way beyond what would normally be expected.judgment sampling The process of selecting a sample which is believed to be representative of a given population. (Also referred to as purposive sampling.) Likert scale An instrument that asks an individual to respond to a series of statements by indicating whether she or he strongly agrees (SA), agrees (A), is undecided (U), disagrees (D), or strongly disagrees (SD) with each statement.

limitation An aspect of a study which the researcher knows may negatively affect the results or generalizability of the results, but over which he or she has no control.linear relationship The situation in which an increase (or decrease) in one variable is associated with a corresponding increase (or decrease) in another variable.logical validity Validity which is determined primarily through judgment; includes content validity.matching A technique for equating groups on one or more variables, resulting in each member of one group having a direct counterpart in another group.maturation Physical or mental changes which occur within participants over a period of time and which may affect their performance on a measure of the dependent variable.mean The arithmetic average of a set of scores.measures of Indices representing the average or typical score attained by a group of participants.measures of variability Indices indicating how spread out the scores are in a distribution.median That point in a distribution above and below which are 50% of the scores.menu-driven Refers to computer programs which allow the user to select desired analyses from a list, or menu, of options.meta-analysis A statistical approach to summarizing the results of many studies which have investigated basically the same problem.mode The score that is attained by more pa;..ticipants in a group than any other score.modem A device which permits telephone communication between two computers by converting computer language to audiotones.mortality Refers to the fact that participants who drop out of a study may share a characteristic such that their absence has a significant effect on the results of the study. multiple-baseline design A single-subject design in which baseline data are collected on several behaviors for one participant or one behavior for several participants and treatment is applied systematically over a period of time to each behavior (or each participant) one at a time until all behaviors (or participants) are under treatment.multiple comparisons Procedures used following application of analysis of variance to deterrnine which means are significantly different from which other means.multiple regression equation A prediction equation using two or more variables that individually predict a criterion to make a more accurate prediction.multiple time-series design A variation of the time-series design that involves the addition of a control group to the basic design.multiple-treatment interference Refers to the carry-over effects from an earlier treatment that make it difficult to assess the effectiveness of a later treatment.naturalistic observation Observation in which the observer purposely controls or manipulates nothing, and in fact works very hard at not affecting the observed situation in any way.negatively skewed distribution A distribution in which there are more extreme scores at the lower end than at the upper, or higher, end.nominal scale The lowest level of measurement which classifies persons or objects into two or more categories; a person can only be in one category, and members of a category have a common set of characteristics.

nonequivalent control group design A quasiexperimental design involving at least two groups, both of which are pretested; one group receives the experimental treatment, and both groups are posttested.nonparametric test A test of significance appropriate when the data represent an ordinalor nominal scale, when a parametric assumption has been greatly violated, or when the nature of the distribution is not known.nonparticipant observation Observation in which the observer is not directly involved in the situation to be observed, i.e., the observer does not intentionally interact with or affect the object of the observation.nonprobability sampling The process of selecting a sample using a technique which does not permit the researcher to specify the probability, or chance, that each member of a population has of being selected for the sample.novelty effect A type of reactive arrangement resulting from increased interest, motivation, or participation on the part of participants simply because they are doing something different.null hypothesis States that there is no relationship (or difference) between variables and that any relationship found will be a chance relationship, the result of sampling error, not a true one.observational research Descriptive research in which the desired data is obtained not by asking individuals for it but through such means as direct observation.observer bias The phenomenon whereby an observer does not observe objectively and accurately, thus producing invalid observations.observer effeds The phenomenon whereby persons being observed behave atypically simply because they are being observed, thus producing invalid observations.one-group pretest-posttest design A pre-experimental design involving one group which is pretested, exposed to a treatment, and post tested.one-shot case study A pre-experimental design involving one group which is exposed to a treatment and then post-tested.operational definition One which defines concepts in terms of processes, or operations.ordinal scale A measurement scale that classifies participants and ranks them in terms of the degree to which they possess a characteristic of interest.organismic variable A characteristic of a participant, or organism (e.g., sex), which can-not be directly controlled but which can be controlled for.parameter A numerical index describing the behavior of a population.parametric test A test of significance appropriate when the data represent an interval or ratio scale of measurement and other assumptions have been met.participant A person who provided data for a research study.participant observation Observation in which the observer actually becomes a part of, a participant in, the situation to be observed.participant variable A variable on which participants in different groups in a study might differ, e.g., intelligence.pattern The connection of categories in qualitative analysis.Pearson r A measure of correlation appropriate when the data represent either inter\'al or ratio scales; it takes into account each and every score and produces a coefficient between .00 and ::::1.00.

percentile rank A measure of relative position indicating the percentage of scores that fall at or below a given score.phenomenology The experience of an activity or concept from participants' perspectives pilot study A small-scale study conducted prior to the conducting of the actual study;the entire study is conducted, every procedure is followed, and the resulting data are analyzed-all according to the research plan.placebo effect Refers to the discovery in medical research that any "medication" could make participants feel better, even sugar and water.population The group to which the researcher would like the results of a study to be generalizable.positively skewed distribution A distribution in which there are more extreme scores at the upper, or higher, end than at the lower end.posttest-only control group design A true experimental design involving at least two randomly formed groups; one group receives a new, or unusual, treatment and both groups are post tested.power The ability of a significance test to avoid making a Type II error.prediction study An attempt to determine which of a number of variables are most highly related to a criterion variable, a complex variable to be predicted.predictive validity The degree to which a test is able to predict how well an individual will do in a future situation.predictor In a prediction study, the variable upon which the prediction is based.pretest-postest control group design A true experimental design which involves at least two randomly formed groups; both groups are pretested, one group receives a new, or unusual treatment, and both groups are post tested.pretest sensitization See testing.pretest-treatment interaction Refers to the fact that participants may respond or react differently to a treatment because they have been pretested.primary source Firsthand information such as the testimony of an eyewitness, an original document, a relic, or a description of a study written by the person who conducted it.probability sampling The process of selecting a sample using a sampling technique which permits the researcher to specify the probability, or chance, that each member of a defined population has of being selected for the sample.problem statement A statement which indicates the variables of interest to the researcher and the specific relationship behveen those variables which is to be, or was, investigated.prospective causal-comparative research A variation of the basic approach to causalcomparative research which involves starting with the causes and investigating effects.purposive sampling See judgment sampling.qualitative approach The collection of extensive narrative data in order to gain insights into phenomena of interest.qualitative research The collection of extensive narrative data on many variables over an extended period of time, in a naturalistic setting, in order to gain insights not possible using other types of research.

quantitative research The collection of numerical data in order to explain, predict and/ or control phenomena of interest.quartile deviation One-half of the difference between the upper quartile (the 75th percentile) and the lower quartile (the 25th percentile) in a distribution.quota sampling The process of selecting a sample based on required, exact numbers, or quotas, of persons of varying characteristics.random sampling The process of selecting a sample in such a way that all individuals in the defined population have an equal and independent chance of being selected for the sample.range The difference between the highest and lowest score in a distribution.rationale equivalence reliability An estimate of internal consistency based on a determination of how all items on a test relate to all other items and to the total test.ratio scale The highest level of measurement that classifies participants, ranks participants, is based upon predetermined equal intervals, and has a true zero point.reactive arrangements Threats to the external validity of a study associated with the way in which a study is conducted and the feelings and attitudes of the participants involved.readiness test A test administered prior to instruction or training in a specific area in order to determine whether and to what degree a student is ready for, or will profit from, instruction.relationship study An attempt to gain insight into the variables, or factors, that are related to a complex variable such as academic achievement, motivation, and self-concept.reliability The degree to which a test consistently measures whatever it measures.replication Refers to when a study is done again; the second study may be a repetition of the original study, using different participants, or it may represent an alternative approach to testing the same hypothesis.research The formal, systematic application of the scientific and disciplined inquiry approach to the study of problems.research hypothesis A statement of the expected relationship (or difference) between two variables.research plan A detailed description of a proposed study designed to investigate a given problem.response set The tendency of an observer to rate the majority of observees the same regardless of the observees' actual behavior. retrospective causal-c:omparative research The basic approach to causal-<:omparative research which involves starting with effects and investigating causes.review of literature The systematic identification, location, and analysis of documents containing information related to a research problem.sample A number of individuals selected from a population for a study, preferably in such a way that they represent the larger group from which they were selected.sample survey Research in which information about a population is inferred based on the responses of a sample selected from that population.sampling The process of selecting a number of individuals (a sample) from a population, preferably in such a way that the individuals selected represent the larger group from which they were selected.

sampling bias Systematic sampling error; two major sources of sampling bias are the use of volunteers and the use of available groups.sampling error Expected, chance variation in variables that occurs when a sample is selected from a population.sampling validity The degree to which a test samples the total intended content area.Scheffe test A conservative multiple comparison technique appropriate for making any and all possible comparisons involving a set of means.secondary source Secondhand information, such as a brief description of a study written by someone other than the person who conducted it.selection-maturation interaction Refers to the fact that if already-formed groups are used in a study, one group may profit more (or less) from treatment or have an initial advantage (or disadvantage) because of maturation factors; selection may also interact with factors such as history and testing.selection-treatment interaction Refers to the fact that if nonrepresentative groups are used in a study the results of the study may hold only for the groups involved and maynot be representative of the treatment effect in the population.self-report research Descriptive research in which information is solicited from individuals using, for example, questionnaires or interviews.semantic differential scale An instrument that asks an individual to give a quantitative rating to the participant of the attitude scale on a number of bipolar adjectives such as goodbad, friendly-unfriendly, positive-negative. shrinkage Refers to the tendency of a prediction equation to become less accurate when used with a different group, a group other than the one on which the equation was originally formulated.simple analysis of variance (ANOVA) A parametric test of significance used to determine whether there is significant difference between or among two or more means at a selected probability level.simulation observation Observation in which the researcher creates the situation to be observed and tells the participant what activities they are to engage in.simultaneous replication Refers to when replication is done on a number of participants with the same problem, at the same location, at the same time.single-subject experimental designs Designs applied when the sample size is one; used to study the behavior change which an individual exhibits as a result of some intervention, or treatment.single-variable designs A class of experimental designs involving only one independent variable (which is manipulated).single variable rule An important principle of single-subject research which states that only one variable should be manipulated at a time.skewed distribution A nonsymmetrical distribution in which there are more extreme scores at one end of the distribution than the other.sociometric study A study that assesses and analyzes the interpersonal relationships within a group of individuals.software Refers to the programs which give instructions to the computer concerning desired operationsSolomon four-group design A true experimental design that involves random assignment of participants to one of four groups; two groups are pretested, two are not;

one of the pretested groups and one of the unpretested groups receive the experimental treatment, and all four groups are post tested.Spearman rho A measure of correlation appropriate when the data for at least one of the variables are expressed as ranks; it produces a coefficient between .00 and :!:1.00.specificity of variables Refers to the fact that a given study is conducted with a specific kind of participant, using specific measuring instruments, at a specific time, under a specific set of circumstances, factors that affect the generalizability of the results.split-half reliability A type of reliability that is based on the internal consistency of a test and is estimated by dividing a test into two equivalent halves and correlating the scores on the two halves.standard deviation The most stable measure of variability which takes into account each and every score in a distribution.standard error of the mean The standard deviation of sample means which indicates by how much the sample means can be expected to differ if other samples from the same population are used.standard error of measurement An estimate of how often one can expect errors of a given size in an individual's test score.standard score A derived score that expresses how far a given raw score is from some reference point, typically the mean, in terms of standard deviation units.stanines Standard scores that divide a distribution into nine parts.static group comparison A pre-experimental design that involves at least two nonrandomly formed groups; one receives a new, or unusual, treatment and both are posttested.statistic A numerical index describing the behavior of a sample or samples.statistical regression The tendency of participants who score highest on a pretest to score lower on a post test, and of participants who score lowest on a pretest to score higher on a post test.statistical significance The conclusion that results are unlikely to have occurred by chance; the observed relationship or difference is probably a real one.stratified sampling The process of selecting a sample in such a way that identified subgroups in the population are represented in the sample in the same proportion that they exist in the population or in equal proportion.structured interview Interview questions that provide options for participants to select from.structured item A question and a list of alternative responses from which the responder selects; also referred to as a closed-form item.survey An attempt to collect data from members of a population in order to determine the current status of that population \\'ith respect to one or more variables.systematic replication Refers to replication which follows direct replication, and which involves different investigators, behaviors, or settings.systematic sampling Sampling in which individuals are selected from a list by taking every Kth name, where K equals the number of individuals on the list divided by the number of participants desired for the sample.T score A standard score derived from a z score by multiplying the z score by 10 and adding 50.

t test for independent samples A parametric test of significance used to determine whether there is a significant difference between the means of two independent samples at a selected probability level.t test for nonindependent samples A parametric test of significance used to determine whether there is a significant difference between the means of two matched, or nonindependent, samples at a selected probability level.target population Refers to the population to which the researcher would ideally like to generalize results.terminal A device for communicating with a computer which consists of a display screen and a keyboardtest A means of measuring the knowledge, skill, feelings, intelligence, or aptitude of an individual or group.testing A threat to experimental validity which refers to improved scores on a post test which are a result of participants having taken a pretest. (Also referred to as pretest sensitization.) test objectivity Refers to a situation in which an individual's score is the same, or essentially the same, regardless of who is doing the scoring.test of significance A statistical test used to determine whether or not there is a significant difference between or among two or more means at a selected probability level.test-retest reliability The degree to which scores on a test are consistent, or stable, over time.time-series design A quasi-experimental design involving one group which is repeatedly pretested, exposed to an experimental treatment, and repeatedly post tested.triangulation The use of multiple methods, data collection strategies, and/ or data sources, in order to get a more complete picture and to cross-check information.true categories Categories into which persons or objects naturally fall, independently of the research study.Type I error The rejection by the researcher of a null hypothesis which is actually true.Type II error The failure of a researcher to reject a null hypothesis which is really false.unobtrusive measures Inanimate objects (such as school suspension lists) which can be observed in order to obtain desired information.munstructured ite A question giving the responder complete freedom of response.validity The degree to which a test measures what it is intended to measure; a test is valid for a particular purpose for a particular group.variable A concept that can assume anyone of a range of values, e.g., intelligence, height, aptitude.z score The most basic standard score that expresses how far a score is from a mean m terms of standard deviation.Z score See T score.

Link: http://www.artsci-ccwin.Concordia.ca/edtech/et664.html http://www.ed.gov/offices/oeri/nerppb/ http://www.oeri/research.html

http://www./resCtr.htmlhttp://www.crede.ucsc.edu/