Research Design

RESEARCH DESIGNAK Dhamija

RESEARCH DESIGN

Definition Classification Exploratory Research Descriptive Research Causal Research Relationships Among Exploratory, Descriptive,

and Causal Research Threats to Internal & External Validity Overcoming Threats

Definition Classification Exploratory Research Descriptive Research Causal Research Relationships Among Exploratory, Descriptive,

and Causal Research Threats to Internal & External Validity Overcoming Threats

DEFINITION

A research design is a blueprint for conductingthe research project.

It details the procedures for obtaining theinformation needed to solve the researchproblems.

A research design is a blueprint for conductingthe research project.

It details the procedures for obtaining theinformation needed to solve the researchproblems.

COMPONENTS OF RESEARCH DESIGN

Information needed Design the exploratory, descriptive, and/or causal

phases of the research Specify the measurement and scaling procedures Construct and pretest a questionnaire Specify the sampling process and sample size Develop a plan of data analysis

Information needed Design the exploratory, descriptive, and/or causal

phases of the research Specify the measurement and scaling procedures Construct and pretest a questionnaire Specify the sampling process and sample size Develop a plan of data analysis

CLASSIFICATION OF RESEARCH DESIGN

Research Design

ConclusiveResearch Design

ExploratoryResearch Design

Single Cross-Sectional Design

Multiple Cross-Sectional Design

DescriptiveResearch

CausalResearch

Cross-SectionalDesign

LongitudinalDesign

BASIC RESEARCH DESIGNS

Objective:

Characteristics:

Methods:

Discovery of ideasand insights

Flexible, versatile

Often the front endof total researchdesign

Expert surveysPilot surveysSecondary data:qualitativeanalysisQualitativeresearch

Describe marketcharacteristics orfunctions

Marked by the priorformulation of specifichypotheses

Preplanned andstructured design

Secondary data:quantitative analysisSurveysPanelsObservation and otherdata

Determine causeand effectrelationships

Manipulation ofone or moreindependentvariables

Control of othermediatingvariables

Experiments

Exploratory Descriptive CausalObjective:

Characteristics:

Methods:

Discovery of ideasand insights

Flexible, versatile

Often the front endof total researchdesign

Expert surveysPilot surveysSecondary data:qualitativeanalysisQualitativeresearch

Describe marketcharacteristics orfunctions

Marked by the priorformulation of specifichypotheses

Preplanned andstructured design

Secondary data:quantitative analysisSurveysPanelsObservation and otherdata

Determine causeand effectrelationships

Manipulation ofone or moreindependentvariables

Control of othermediatingvariables

Experiments

ALTERNATIVE RESEARCH DESIGNS

Exploratory Research• Secondary DataAnalysis

• Focus Groups

Conclusive Research•Descriptive/Causal

Conclusive Research•Descriptive/Causal

(a)

(b) Conclusive Research•Descriptive/Causal

Exploratory Research• Secondary DataAnalysis

• Focus Groups

Conclusive Research•Descriptive/Causal

(b)

(c)

POTENTIAL SOURCES OF ERROR INRESEARCH DESIGNS

Total Error

Non-samplingError

RandomSampling Error

Non-responseError

ResponseError

Surrogate Information ErrorMeasurement ErrorPopulation Definition ErrorSampling Frame ErrorData Analysis Error

Respondent Selection ErrorQuestioning ErrorRecording ErrorCheating Error

Inability ErrorUnwillingness Error

Non-responseError

ResponseError

InterviewerError

RespondentError

ResearcherError

PRIMARY AND SECONDARY DATA

Primary Data Secondary Data

Collection purpose For the problem at hand For other problemsCollection process Very involved Rapid & easyCollection cost High Relatively lowCollection time Long Short

USE OF SECONDARY DATA

Identify the problem Better define the problem Develop an approach to the problem Formulate an appropriate research design (for

example, by identifying the key variables) Answer certain research questions and test some

hypotheses Interpret primary data more insightfully

Identify the problem Better define the problem Develop an approach to the problem Formulate an appropriate research design (for

example, by identifying the key variables) Answer certain research questions and test some

hypotheses Interpret primary data more insightfully

CRITERIA FOR EVALUATING SECONDARYDATA

Criteria Issues Remarks

Specifications& Methodology

Error &Accuracy

Currency

Objective

Nature

Dependability

Data collection method, responserate, quality & analysis of data,sampling technique & size,questionnaire design, fieldwork.Examine errors in approach,research design, sampling, datacollection & analysis, & reporting.

Time lag between collection &publication, frequency of updates.Why were the data collected?

Definition of key variables, units ofmeasurement, categories used,relationships examined.Expertise, credibility, reputation,and trustworthiness of the source.

Data should be reliable,valid, & generalizable tothe problem.

Assess accuracy bycomparing data fromdifferent sources.

Census data are updatedby syndicated firms.The objectivedetermines therelevance of data.Reconfigure the data toincrease theirusefulness.

Data should be obtainedfrom an original source.

Specifications& Methodology

Error &Accuracy

Currency

Objective

Nature

Dependability

Data collection method, responserate, quality & analysis of data,sampling technique & size,questionnaire design, fieldwork.Examine errors in approach,research design, sampling, datacollection & analysis, & reporting.

Time lag between collection &publication, frequency of updates.Why were the data collected?

Definition of key variables, units ofmeasurement, categories used,relationships examined.Expertise, credibility, reputation,and trustworthiness of the source.

Data should be reliable,valid, & generalizable tothe problem.

Assess accuracy bycomparing data fromdifferent sources.

Census data are updatedby syndicated firms.The objectivedetermines therelevance of data.Reconfigure the data toincrease theirusefulness.

Data should be obtainedfrom an original source.

CONDITIONS FOR CAUSALITY

Concomitant variation Time order of occurrence Absence of other possible causal factors

A set of procedures specifying:

the test units and how these units are to be dividedinto homogeneous subsamples,

what independent variables or treatments are to bemanipulated,

what dependent variables are to be measured; and

how the extraneous variables are to be controlled.

EXPERIMENTAL DESIGN

the test units and how these units are to be dividedinto homogeneous subsamples,

what independent variables or treatments are to bemanipulated,

what dependent variables are to be measured; and

how the extraneous variables are to be controlled.

VALIDITY IN EXPERIMENTATION

Internal validity

Establishing Cause and Effect relationship

Control of Extraneous Variables

External validity

Generalizability

To what populations, settings, times, independentvariables and dependent variables

Internal validity

Establishing Cause and Effect relationship

Control of Extraneous Variables

External validity

Generalizability

To what populations, settings, times, independentvariables and dependent variables

EXTRANEOUS VARIABLES History (H) : specific external events that occur at the same time as

the experiment. Maturation (MA) : changes in the test units themselves that occur

with the passage of time. Testing effects (T) : are caused by the process of experimentation. Main testing effect (MT) : occurs when a prior observation affects a

latter observation. Interactive testing effect (IT) : A prior measurement affects the

test unit's response to the independent variable. Instrumentation (I) : changes in the measuring instrument, in the

observers or in the scores themselves. Statistical regression effects (SR) : occur when test units with

extreme scores move closer to the average score during the course ofthe experiment.

Selection bias (SB) refers to the improper assignment of test unitsto treatment conditions.

Mortality (MO) refers to the loss of test units while the experimentis in progress.

History (H) : specific external events that occur at the same time asthe experiment.

Maturation (MA) : changes in the test units themselves that occurwith the passage of time.

Testing effects (T) : are caused by the process of experimentation. Main testing effect (MT) : occurs when a prior observation affects a

latter observation. Interactive testing effect (IT) : A prior measurement affects the

test unit's response to the independent variable. Instrumentation (I) : changes in the measuring instrument, in the

observers or in the scores themselves. Statistical regression effects (SR) : occur when test units with

extreme scores move closer to the average score during the course ofthe experiment.

Selection bias (SB) refers to the improper assignment of test unitsto treatment conditions.

Mortality (MO) refers to the loss of test units while the experimentis in progress.

CONTROLLING EXTRANEOUS VARIABLES

Randomization : The random assignment of testunits and treatment conditions to experimental groups

Matching involves comparing test units on a set ofkey background variables before assigning them to thetreatment conditions.

Statistical control involves measuring theextraneous variables and adjusting for their effectsthrough statistical analysis.

Design control involves the use of experimentsdesigned to control specific extraneous variables.

Randomization : The random assignment of testunits and treatment conditions to experimental groups

Matching involves comparing test units on a set ofkey background variables before assigning them to thetreatment conditions.

Statistical control involves measuring theextraneous variables and adjusting for their effectsthrough statistical analysis.

Design control involves the use of experimentsdesigned to control specific extraneous variables.

CLASSIFICATION OF EXPERIMENTALDESIGNS

Pre-experimental TrueExperimental

QuasiExperimental Statistical

RandomizedBlocks

Latin Square

FactorialDesign

Experimental Designs

One-Shot CaseStudy

One GroupPretest-Posttest

Static Group

Pretest-PosttestControl Group

Posttest: OnlyControl Group

Solomon Four-Group

Time Series

Multiple TimeSeries

RandomizedBlocks

Latin Square

FactorialDesign

VARIOUS PRE EXPERIMENTAL DESIGNS

One-Shot Case Study : X 01

More appropriate for exploratory study

One Group Pre-Post Test : 01 X 02

Treatment effect is computed as 02 – 01

Static Group Design:EG: X 01

CG: 02

Treatment effect is computed as 02 – 01

One-Shot Case Study : X 01

More appropriate for exploratory study

One Group Pre-Post Test : 01 X 02

Treatment effect is computed as 02 – 01

Static Group Design:EG: X 01

CG: 02

Treatment effect is computed as 02 – 01

VARIOUS TRUE EXPERIMENTAL DESIGNS

Pretest Posttest Control Group Design:

EG: R 01 X 02CG: R 03 04

02 – 01= TE + H + MA + MT + IT + I + SR + MO

04 – 03= H + MA + MT + I + SR + MO

Treatment effect(TE) = (02 - 01) - (04 - 03) = TE + IT

Selection Bias is eliminated by Randomization

Pretest Posttest Control Group Design:

EG: R 01 X 02CG: R 03 04

02 – 01= TE + H + MA + MT + IT + I + SR + MO

04 – 03= H + MA + MT + I + SR + MO

Treatment effect(TE) = (02 - 01) - (04 - 03) = TE + IT

Selection Bias is eliminated by Randomization

VARIOUS TRUE EXPERIMENTAL DESIGNSPosttest Only Control Group Design:

EG: R X 01CG: R 02Treatment effect(TE) = (01 – 02)

Solomon‘s Four Group Six Study Design:

EG1: R 01 X 02 :: 02 - 01CG1: R 03 04 :: 04 - 03EG2: R X 05 :: 05 – 0.5(01 + 03)CG2: R 06 :: 06 – 0.5(01 + 03)02 – 01 = TE + H + MA + MT + IT + I + SR + MO04 – 03 = H + MA + MT + I + SR + MO

Highest on Internal & External validity

[05 -0.5(01 + 03)] - [06 -0.5(01 + 03)] = TE : EG2 vs CG2

[(02 - 01)] - [05 -0.5(01 + 03)] = IT : EG2 vs EG1

[(04 - 03)] - [06 -0.5(01 + 03)] = Total uncontrolled Factors : : CG1 vs CG2

Posttest Only Control Group Design:EG: R X 01CG: R 02Treatment effect(TE) = (01 – 02)

Solomon‘s Four Group Six Study Design:

EG1: R 01 X 02 :: 02 - 01CG1: R 03 04 :: 04 - 03EG2: R X 05 :: 05 – 0.5(01 + 03)CG2: R 06 :: 06 – 0.5(01 + 03)02 – 01 = TE + H + MA + MT + IT + I + SR + MO04 – 03 = H + MA + MT + I + SR + MO

Highest on Internal & External validity

[05 -0.5(01 + 03)] - [06 -0.5(01 + 03)] = TE : EG2 vs CG2

[(02 - 01)] - [05 -0.5(01 + 03)] = IT : EG2 vs EG1

[(04 - 03)] - [06 -0.5(01 + 03)] = Total uncontrolled Factors : : CG1 vs CG2

QUASI EXPERIMENTAL DESIGNS

Time Series Design01 02 03 04 05 X 06 07 08 09 010

No randomization of test units to treatments.

The timing of treatment presentation, as well aswhich test units are exposed to the treatment, maynot be within the researcher's control.

Time Series Design01 02 03 04 05 X 06 07 08 09 010

No randomization of test units to treatments.

The timing of treatment presentation, as well aswhich test units are exposed to the treatment, maynot be within the researcher's control.

STATISTICAL DESIGNS

The effects of more than one independentvariable can be measured.

Specific extraneous variables can be statisticallycontrolled.

Economical designs can be formulated when eachtest unit is measured more than once.

The effects of more than one independentvariable can be measured.

Specific extraneous variables can be statisticallycontrolled.

Economical designs can be formulated when eachtest unit is measured more than once.

STATISTICAL DESIGNS - RCBRandomized Block Design(RCB) : Blocking ensuresclose matching of groups on external variable

Treatment GroupsBlock Store Commercial Commercial CommercialNumber Patronage A B C

1 Heavy A B C2 Medium A B C3 Low A B C4 None A B C

Treatment GroupsBlock Store Commercial Commercial CommercialNumber Patronage A B C

1 Heavy A B C2 Medium A B C3 Low A B C4 None A B C

STATISTICAL DESIGNS - LSDLatin Square Design: Statistically control two noninteracting external variables as well as tomanipulate the independent variable.Assignment rule is that each level of theindependent variable should appear only once ineach row and each column

Latin Square Design: Statistically control two noninteracting external variables as well as tomanipulate the independent variable.Assignment rule is that each level of theindependent variable should appear only once ineach row and each column

Interest in the StoreStore Patronage High Medium Low

Heavy B A CMedium C B ALow and none A C B

STATISTICAL DESIGNS - FDFactorial Design: To measure the effects of two or

more independent variables at various levels.

All the factors of all the levels are crossedSo many interactions : unwieldy but moreinformative

Amount of HumorAmount of Store No Medium HighInformation Humor Humor HumorLow A B C

Medium D E F

High G H I

Amount of HumorAmount of Store No Medium HighInformation Humor Humor HumorLow A B C

Medium D E F

High G H I

LABORATORY VS FIELD EXPERIMENTS

Factor Laboratory FieldEnvironment Artificial RealisticControl High LowReactive Error High LowDemand Artifacts High LowInternal Validity High LowExternal Validity Low HighTime Short LongNumber of Units Small LargeEase of Implementation High LowCost Low High

Factor Laboratory FieldEnvironment Artificial RealisticControl High LowReactive Error High LowDemand Artifacts High LowInternal Validity High LowExternal Validity Low HighTime Short LongNumber of Units Small LargeEase of Implementation High LowCost Low High

REFERENCES

Marketing Research by Naresh Malhotra Marketing Research by Churchill & Iacobucci