Market Intelligence Session 7 Experimental Research.

Market Intelligence Session 7

Experimental Research

Experiments

• Only way to test causal hypotheses

• Independent Variable = hypothesized cause– Usually manipulated by the researcher/manager – Example: Send a color or black and white brochure

• Dependent Variable = effect– Measured (observed) by researcher/manager– Example: New accounts secured

2

3

3 key features of true experiments

1. Manipulation of a variable2. Control/comparison group3. Random assignment to groups

4

Can’t always run true experiment

• Sometimes can’t manipulate (or ethically manipulate) variable of interest (smoking)

• Sometimes can’t get a comparison group beforehand (all people affected by event)

• Can’t always randomly assign people to groups (which class people take)

5

Can’t always run true experiment

• Sometimes can’t manipulate (or ethically manipulate) variable of interest

• Sometimes can’t get a comparison group beforehand

• Can’t always randomly assign people to groups

• Solution: Correlational or Quasi-experimental designs– Goal: get as close to true experiment as possible

6

Types of designs

• Correlational (cross-lag panel)• Quasi-experimental

– One group posttest only– One group pretest-posttest– Nonequivalent control posttest only– Nonequivalent control pretest-posttest– Time series– Non-equivalent control time series

• True experiments– Posttest equivalent groups– Pretest-posttest equivalent groups

Notation (used across social sciences)

• X = Manipulation – No X means that group did not receive manipulation

• R = Random assignment to different experimental groups or conditions

• On = Observation of DV at Time N

– O1 = before manipulation

– O2 = after manipulation

7

8

Types of designs

• Correlational (cross-lag panel)• Quasi-experimental

– One group posttest only– One group pretest-posttest– Nonequivalent control posttest only– Nonequivalent control pretest-posttest– Interrupted time series– Non-equivalent control time series

• True experiments– Posttest equivalent groups– Pretest-posttest equivalent groups

9

Correlational designs

• Examine correlations between 2 variables• Most common: Cross lag panels

– Examine correlations between 2 variables at 2 time points

– Purpose: to see if evidence supports 1 causal direction more than the other

– Notation: O1 O2

10

Key: Look at diagonal correlations

11

Example: Lefkowitz et al. (1972)

.21

.01

.38

.05

.31

.-.05

10 years

TV Violence TV Violence

Aggression Aggression

12

Example: Lefkowitz et al. (1972)

.21

.38

.05

-.05

10 years

TV Violence TV Violence

Aggression Aggression

13

Example: Lefkowitz et al. (1972)

.21

.01

.38

.05

.31

-.05

10 years

TV Violence TV Violence

Aggression Aggression

14

Types of designs

• Correlational (cross-lag panel)• Quasi-experimental

– One group posttest only– One group pretest-posttest– Nonequivalent control posttest only– Nonequivalent control pretest-posttest– Interrupted time series– Non-equivalent control time series

• True experiments– Posttest equivalent groups– Pretest-posttest equivalent groups

15

One group posttest onlyX O2

16

Types of designs

• Correlational (cross-lag panel)• Quasi-experimental

– One group posttest only– One group pretest-posttest– Nonequivalent control posttest only– Nonequivalent control pretest-posttest– Interrupted time series– Non-equivalent control time series

• True experiments– Posttest equivalent groups– Pretest-posttest equivalent groups

17

One group pretest-posttest O1 X O2

18

Types of designs

• Correlational (cross-lag panel)• Quasi-experimental

– One group posttest only– One group pretest-posttest– Nonequivalent control posttest only– Nonequivalent control pretest-posttest– Interrupted time series– Non-equivalent control time series

• True experiments– Posttest equivalent groups– Pretest-posttest equivalent groups

19

Nonequivalent control posttest onlyX O2

O2

20

Types of designs

• Correlational (cross-lag panel)• Quasi-experimental

– One group posttest only– One group pretest-posttest– Nonequivalent control posttest only– Nonequivalent control pretest-posttest– Interrupted time series– Non-equivalent control time series

• True experiments– Posttest equivalent groups– Pretest-posttest equivalent groups

21

Nonequivalent control pretest-posttestO1 X O2

O1 O2

22

Types of designs

• Correlational (cross-lag panel)• Quasi-experimental

– One group posttest only– One group pretest-posttest– Nonequivalent control posttest only– Nonequivalent control pretest-posttest– Interrupted time series– Non-equivalent control time series

• True experiments– Posttest equivalent groups– Pretest-posttest equivalent groups

23

Interrupted time seriesO1 O1 O1 X O2 O2 O2

(no)

24

25

Removal of treatment

26

Nonequivalent control time seriesO1 O1 O1 X O2 O2 O2

O1 O1 O1 O2 O2 O2

27

Adding control condition

28

Threats to internal validity

• Selection bias: people in different groups/conditions may be different (because groups occurred naturally)

• History: an event occurring around same time as manipulation that has nothing to do with manipulation

• Maturation: people change over time

• Testing: repeatedly testing can change responses

• Differential attrition: when attrition is related to condition

• No control/baseline: nothing to compare it to 29

30

Types of designs

• Correlational (cross-lag panel)• Quasi-experimental

– One group posttest only– One group pretest-posttest– Nonequivalent control posttest only– Nonequivalent control pretest-posttest– Interrupted time series– Non-equivalent control time series

• True experiments– Posttest equivalent groups– Pretest-posttest equivalent groups

31

True Experimental Designs

Posttest equivalent groupsR X O2

R O2

Pretest posttest equivalent groupsR O1 X O2

R O1 O2

Breckenridge Brewery Ad

• Breckenridge Brewery wants to assess the efficacy of TV ad spots for its new Amber Ale.

X: Two weeks of ads for Breckenridge Ale. O: Give survey on beer brands purchased over past week.

32

33

Match up the 8 designs

• Quasi-experimental– One group posttest only– One group pretest-posttest– Nonequivalent control posttest only– Nonequivalent control pretest-posttest– Time series– Non-equivalent control time series

• True experiments– Posttest equivalent groups– Pretest-posttest equivalent groups

Breckenridge Amber Ale

O2

Mean = 1.3 Packs per Week

XTwo Weeks of Ads for

Breckenridge Amber Ale

O2

Mean = 0.5 Packs per WeekDifference b/w cities = 0.8

Durham

Chapel Hill

Design? ______________

Breckenridge Amber Ale

O2

Mean = 1.3 Packs per Week

XTwo Weeks of Ads for

Breckenridge Amber Ale

O2

Mean = 0.5 Packs per WeekDifference b/w cities = 0.8

Durham

Chapel Hill

Design? Nonequivalent control posttest only

36

Mean Breckenridge consumption(packs per week)

O1 O2 O3 O4 O5 O60

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

Durham (ad shown)

Design? ______________

37

Mean Breckenridge consumption(packs per week)

O1 O2 O3 O4 O5 O60

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

Durham (ad shown)

Design? Interrupted Time series

Breckenridge Amber Ale

O1 O2

Mean = 0.2 Packs per Week Mean = 1.3 Packs per Week

XTwo Weeks of Ads for

Breckenridge Amber Ale

O1

Mean = 0.3 Packs per Week

O2

Mean = 0.5 Packs per Week

Δ=1.1

Δ=0.2

ΔΔ=0.9

Experimental Group (Randomly Assigned)

Control Group (Randomly Assigned)

Design? ______________

Breckenridge Amber Ale

O1 O2

Mean = 0.2 Packs per Week Mean = 1.3 Packs per Week

XTwo Weeks of Ads for

Breckenridge Amber Ale

O1

Mean = 0.3 Packs per Week

O2

Mean = 0.5 Packs per Week

Δ=1.1

Δ=0.2

ΔΔ=0.9

Experimental Group (Randomly Assigned)

Control Group (Randomly Assigned)

Design? Pretest-posttest equivalent groups

Breckenridge Amber Ale

O2

Mean = 0.16 Packs per Week

X Two Weeks of Ads for

Breckenridge Amber Ale

Design? ______________

Breckenridge Amber Ale

O2

Mean = 0.16 Packs per Week

X Two Weeks of Ads for

Breckenridge Amber Ale

Design? One group posttest only

Breckenridge Amber Ale

O1 O2

Mean = 0.2 Packs per Week Mean = 1.3 Packs per Week

XTwo Weeks of Ads for

Breckenridge Amber Ale

O1

Mean = 0.3 Packs per Week

O2

Mean = 0.5 Packs per Week

Δ=1.1

Δ=0.2

ΔΔ=0.9

Durham

Chapel Hill

Design? ______________

Breckenridge Amber Ale

O1 O2

Mean = 0.2 Packs per Week Mean = 1.3 Packs per Week

XTwo Weeks of Ads for

Breckenridge Amber Ale

O1

Mean = 0.3 Packs per Week

O2

Mean = 0.5 Packs per Week

Δ=1.1

Δ=0.2

ΔΔ=0.9

Durham

Chapel Hill

Design? Nonequivalent control pretest-posttest

Breckenridge Amber Ale

O2

Mean = 1.3 Packs per Week

XTwo Weeks of Ads for

Breckenridge Amber Ale

O2

Mean = 0.5 Packs per Week

Difference between groups=0.8

Experimental Group (Randomly Assigned)

Control Group (Randomly Assigned)

Design? ______________

Breckenridge Amber Ale

O2

Mean = 1.3 Packs per Week

XTwo Weeks of Ads for

Breckenridge Amber Ale

O2

Mean = 0.5 Packs per Week

Difference between groups=0.8

Experimental Group (Randomly Assigned)

Control Group (Randomly Assigned)

Design? Posttest equivalent groups

46

Mean Breckenridge consumption(packs per week)

O1 O2 O3 O4 O5 O60

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

Durham (ad shown)Chapel Hill (ad not shown)

Design? ______________

47

Mean Breckenridge consumption(packs per week)

O1 O2 O3 O4 O5 O60

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

Durham (ad shown)Chapel Hill (ad not shown)

Design? Nonequivalent control time series

Breckenridge Amber Ale

O1 O2

Mean = 0.2 Packs per Week Mean = 1.3 Packs per Week

X Two Weeks of Ads for

Breckenridge Amber Ale

Δ=1.1

Design? ______________

Breckenridge Amber Ale

O1 O2

Mean = 0.2 Packs per Week Mean = 1.3 Packs per Week

X Two Weeks of Ads for

Breckenridge Amber Ale

Δ=1.1

Design? One group pretest-posttest

Experiments - Factorial Designs

• 2 or more independent variables (manipulated and/or measured), each with two or more levels. – Type 1: 2 marketing mix variables

• Both variables manipulated• Important for determining whether you need to coordinate

marketing actions

– Type 2: “tactical segmentation” (1 segment responds differently to a marketing mix variable than another segment)

• Segmenting variable is measured, marketing action is manipulated

• Important for determining whether you should segment for that particular marketing action 50

Experiments - Factorial Designs

• What to look for in factorial designs– Is there a main effect of A?– Is there a main effect of B?– Key: Is there an interaction between A and B?

(interaction: effect of one IV on DV depends on level of another IV)

• Analysis – Eye-ball method– Analysis of Variance (ANOVA)

51

Type 1: 2 marketing mix variables

• Assume two of you manage the Oreo account at Kroger. – One manages advertising, one

manages in store promotions like end-of-aisle display

• You have been asked to evaluate whether ads and/or end-of-aisle display would increase sales …

AdvertisingEnd of Aisle

Oreo Promotion Experiment

Kroger: Supporting a discount on Oreo cookies

Factor A: Ads in local papera1 = no adsa2 = ad in Thursday local paper

Factor B: Display locationb1 = regular shelfb2 = end aisle

53

OREO PROMOTION EXPERIMENTScenario 1

(EXPENDITURES/CUSTOMER/2 WKS)

54

OREO PROMOTION EXPERIMENTScenario 1

(EXPENDITURES/CUSTOMER/2 WKS)

55

Main effect of A?

Main effect of B?

OREO PROMOTION EXPERIMENTScenario 1

(EXPENDITURES/CUSTOMER/2 WKS)

56Interaction?

this diffvs.

this diff

OREO PROMOTION EXPERIMENTScenario 1

(EXPENDITURES/CUSTOMER/2 WKS)

57Interaction?

this diffvs.

this diff

SALES OF OREOS

58

SALES OF OREOS

59

2 main effects, no interaction

SALES OF OREOS

60

Do they need to coordinate to make their decisions?

Oreo Promotion ExperimentScenario 2

(Expenditures/customer/2 wks)

61

0.95 0.70

0.75

0.8250.800.77

5

Oreo Promotion ExperimentScenario 2

(Expenditures/customer/2 wks)

62

0.95 0.70

0.75

0.8250.800.77

5

Main effect of A?

Main effect of B?

Oreo Promotion ExperimentScenario 2

(Expenditures/customer/2 wks)

63

0.95 0.70

0.75

0.8250.800.77

5

Interaction?

64

A1=ads A2=no ads0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

B1-reg shelfB2-end aisle

SALES OF OREOS(Expenditures/customer/2 wks)

65

A1=ads A2=no ads0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

B1-reg shelfB2-end aisle

“Cross-over” interaction

SALES OF OREOS(Expenditures/customer/2 wks)

66

A1=ads A2=no ads0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1

B1-reg shelfB2-end aisle

SALES OF OREOS(Expenditures/customer/2 wks)

Do they need to coordinate to make their decisions?

Oreo Promotion ExperimentScenario 3

(Expenditures/customer/2 wks)

67

1.30

Oreo Promotion ExperimentScenario 3

(Expenditures/customer/2 wks)

68

1.30

Main effect of A?

Main effect of B?

Oreo Promotion ExperimentScenario 3

(Expenditures/customer/2 wks)

69

Interaction?

1.30

70

SALES OF OREOS(Expenditures/customer/2 wks)

a1=no ads a2=ads0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

1.4

1.5

b1=regular shelfb2=end aisle

71

SALES OF OREOS(Expenditures/customer/2 wks)

a1=no ads a2=ads0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

1.4

1.5

b1=regular shelfb2=end aisle

“fan effect” interaction

72

SALES OF OREOS(Expenditures/customer/2 wks)

a1=no ads a2=ads0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

1.4

1.5

b1=regular shelfb2=end aisle

Do they need to coordinate to make their decision?

Oreo Example

• No A x B interaction– Effect of changing A (Ads) is independent of level of B

(Display Location). – Implies that Ad & Display decisions can be

decoupled…they influence sales additively

• A x B interaction– Effect of changing A (ads) depends on level of B

(display location), and/or vice-versa– Fan effect: Cannot decouple variables– Cross-over: Cannot decouple variables

73

Type 2: Tactical Segmentation

• Should groups be treated same or differently with respect to specific marketing decision variable?

• A is a controllable decision variable and B is a potential segmentation variable – Interaction means that segments respond differently to

this marketing lever.– Example: coupons x urban/suburban

• Question: does marketing mix variable have bigger effect for segment A or B?

• Is coupon more effective in urban or suburban neighborhoods? 74

Interactions and segmentation

75

Interactions and segmentation

76

Coupons have a bigger effect in the suburbs

Tactical Segmentation

• Example - Dog Food• 1 potential segmentation variable (Size of Dog)• 2 decisions

– Price: Hi v. Lo – Ad Theme: “Love between dog and owner”

vs. “Dog’s Active Life”

77

Segmentation Example: Dog Food I (rated on 10 pt scale)

• Price

• Advertising

78

Segmentation Example: Dog Food I• Price

• Advertising

79

80

Implications of Contrast

• A variable that is an excellent basis for segmentation with respect to one decision about a marketing mix element may be a poor basis for segmentation with respect to another mix element

• For any given mix element decision, when evaluating alternative bases for segmentation, look for ones with big differences in sensitivity to mix variable.

81

Tactical Segmentation II

• Example - Dog Food • 1 decision: Price (hi vs. lo)• 2 potential segmentation variables

– Size of Dog – Income of owner

82

Segmentation Example: Dog Food II

83

How to compare? ANOVA

Segmentation Example: Dog Food II

84

Eye-ball method: compare difference of differences

Segmentation Example: Dog Food II

85

Price

Price

86

Garlic Chopper Prototype Breakout

• Survey: what do you want to know?• How to structure survey?• What type(s) of scales to use?• Can randomly assign to conditions

– What would you want to manipulate/measure?

87

For next time

• IBM case (team assignment due)• Guest lecture: Kevin Clark• Due: Cola conjoint assignment• Not due: “product line scenarios”• Quiz 2 next Friday

– Study guide will be on Sakai shortly