How reliable are the consumers?Comparison of sensory profiles
from consumers and expertsfrom consumers and expertsWORCH Thierry(1)
LÊ Sébastien(2)LÊ Sébastien( )
PUNTER Pieter(1)
(1) OPP Product Research(2) A C O t
mailto: [email protected](2) AgroCampus Ouest
Project 8013July 2008
Senior project manager Pieter PunterProject manager Thierry Worch
introduction
• in the sensory theory:• experts panels are used for the products’ descriptionp p p p
• consumers should only be used for the hedonic task• they lack two essentials qualities for profiling (consensus and reproducibility) • there are strong halo effects (Earthy, MacFie & Hedderley,there are strong halo effects (Earthy, MacFie & Hedderley, 1997)
• in the sensory practice:• consumers are sometimes used for both tasks• it has been proven that consumers’ description show the• it has been proven that consumers description show the required qualities (consensus and reproducibility) (Husson, Le Dien, Pagès, 2001)
8013 3
problematic
How reliable are the consumers?
8013 4
presentation of the studies
• products:• twelve luxurious women perfumestwelve luxurious women perfumes
(Gazano, Ballay, Eladan & Sieffermann, 2005)
A l L’I t tAngel (Eau de Parfum)
L’Instant(Eau de Parfum)
Cinéma J’Adore(Eau de Parfum) (Eau de Toilette)
Pleasures (Eau de Parfum)
J’Adore(Eau de Parfum)(Eau de Parfum) (Eau de Parfum)
Aromatics Elixir(Eau de Parfum)
Pure Poison(Eau de Parfum)
Lolita Lempicka(Eau de Parfum)
Shalimar(Eau de Toilette)
Chanel N°5 Coco Mademoiselle
8013 5
Chanel N 5(Eau de Parfum)
Coco Mademoiselle(Eau de Parfum)
presentation of the studies
• expert panel (Agrocampus Rennes)
• twelve persons (11 students and 1 teacher) from the Chantal Le Cozic school (esthetics and cosmetic school)
• focus group per group of six, with two animators• generation of a list of twelve attributes• generation of a list of twelve attributes
• “Vanille”, “Notes Florales”, “Agrume”, “Boisé”, “Vert”, “Epicé”, “Capiteux”, “Fruité”, “Fraîcheur Marine”, “Gourmand”, “Oriental”, “Enveloppant”
• training session for the most difficult ones
• the twelve products were tested two times in two one-hour sessions
8013 6
presentation of the studies
• consumer panel (OP&P Product Research, Utrecht)
• 103 naïve Dutch consumers living in the Utrecht area
• the same twelve perfumes were rated on 21 attributes• the same twelve perfumes were rated on 21 attributes• “odour intensity”, “freshness”, “jasmine”, “rose”, “camomile”, “fresh lemon”, “vanilla”, “mandarin/orange”, “anis”, “sweet fruit/melon”, “honey”, “caramel”, “spicy” “woody” “leather” “nutty/almond” “musk” “animal” “earthy” “incense”spicy , woody , leather , nutty/almond , musk , animal , earthy , incense , “green”
• two products (Shalimar and Pure Poison) were duplicated
• the fourteen (12+2) products were tasted in two one-hour sessions (seven products in each session, presentation order was balanced)
8013 7
presentation route map
• the consumer and expert data are compared in three different ways
1.Univariate analysis• analyses of variance • correlations
2 Multivariate comparison2.Multivariate comparison• construction of the two products’ spaces (PCA)• comparison of the products’ spaces through GPA and MFAcomparison of the products spaces through GPA and MFA
3.Confidence ellipses • graphical confidence intervals around the products averaged over the two panels• graphical confidence intervals around the products defined
8013 8
graphical confidence intervals around the products defined by the different panels
Performance of the two panels(univariate analysis)
8013 9
performance of the panels
• usually, the expert panels should have many qualities:
• discrimination: panelists should be able to detect and describe the differences existing between the products
• reproducibility: panelists should describe the products in the same way, when they are repeatedsame way, when they are repeated
• agreement: panelists should give the same description of the fproducts as the rest of the panel
• it can be measured with the correlations (usually, one panelist is compared to the mean over the rest of the pa e s s co pa ed o e ea o e e es o epanel)
8013 10
expert panel
• panel performance
• discriminate on 11 out of 12 attributes (“Agrume”, pvalue=0.08)• reproducible for 11 out of 12 attributes (“Notes Florales”)
panellist performance (discrimination reproducibility)• panellist performance (discrimination, reproducibility)
• panellists 1, 3 and 12 are very goodpanellists 1, 3 and 12 are very good • panellists 8, 9 and 10 are not good in discrimination (discriminate the products on less than 6 out of 12 attributes)
lli 9 i l d i d ibili ( d ibl• panellist 9 is also not good in reproducibility (reproducible on only 3 out of 12 attributes. “Notes Florales”, “Agrume” and “Enveloppant”)
8013 11
expert panel (correlations)
• distribution of the correlations (correlation between expert i and the mean over the (n-1) others)( ) )
8013 12
consumer panel
• discrimination (on the twelve original products)
• the consumers discriminate the products on all attributes except “camomile” (pvalue = 0.62)
• NB: the consumers discriminate on “Citrus” (pvalue < 0.001)
• reproducibility (on the two duplicated products only)
• consumers are reproducible on all attributes except one (“woody”)
8013 13
consumer panel (reproducibility)
Shalimar
Shalimar 2
8013 14
consumer panel (correlations)
• distribution of the correlations (correlation between a consumer i and the mean over the (n-1) others)( ) )
8013 15
conclusions on the panel performance
• expert panel• discriminates between the productsdiscriminates between the products• are reproducible• high correlations
consumer panel• consumer panel• discriminates between the products• shows reproducibility’s qualitiesshows reproducibility s qualities• lower but still positive correlations (consumers are untrained)
Both panels show the same qualities
8013 16
Both panels show the same qualities
Products’ spaces(multivariate analysis)
8013 17
methodology
• products’ spaces• the products profiles (averaged over the panellists or consumers)the products profiles (averaged over the panellists or consumers) are computed.• Principal Components Analysis is then run on these product x attribute matricesattribute matrices
• comparison of the two products’ spaces (expert and consumer) is acomparison of the two products spaces (expert and consumer) is a “multi-table problem”
• comparison through the Procrustean analysis( )• comparison through Multiple Factor Analysis (MFA)
• comparison through the confidence ellipses technique
8013 18
expert panelg p
2 5
3,0
3,5
AromaticsElixir
1,0
1,5
2,0
2,5
%)
AromaticsElixir
Chaneln5
JAdore EP
Pleasures
Shalimar
1
-1,0
-0,5
0,0
0,5
nsio
n 2
(21.
87 %
CocoMelle
_
JAdore_ET
LInstant
PurePoison
Epicé
Boisé
-2,5
-2,0
-1,5
Dim
en
AngelCinema
02 (2
1.87
%)
CapiteuxVertNotes.florales
Agrume
Fraicheur.marine
Oriental
-4,5
-4,0
-3,5
-3,0 LolitaLempicka0
Dim
ensi
on
Fruité
Enveloppant
-4 -3 -2 -1 0 1 2 3 4
Dimension 1 (64.22 %)
Vanille
8013 19
-1 0 1
Dimension 1 (64.22 %)
-1 Gourmand
consumer panel
4,5
5,0
5,5
6,0
1 5
2,0
2,5
3,0
3,5
4,0
%)
AngelCinema
LInstant
LolitaLempickag
1vanilla
-1,0
-0,5
0,0
0,5
1,0
1,5
nsio
n 2
(17.
97 %
CocoMelle
JAdore_EP
JAdore_ETPleasures PurePoison
camomille
citrus
anis
honey
caramel
-4,0
-3,5
-3,0
-2,5
-2,0
-1,5
Dim
en Chaneln5 Shalimar
0
n 2
(17.
97 %
)
freshness
jasmin
sweet_fruit nutty
animalgreen
7 0
-6,5
-6,0
-5,5
-5,0
-4,5AromaticsElixir
Dim
ensi
oni t it
jasminrose
fresh_lemon
i
woodyleather
muskanimal
earthy
incense
-5 -2.5 0 2.5 5 7.5
Dimension 1 (68.29 %)
-7,0
1
intensity spicy
8013 20
-1 0 1
Dimension 1 (68.29 %)
-1
Multivariate comparison of the two panels (GPA and MFA)
8013 21
expert vs consumer: Procrustes analysis
GPA consensus space
(coefficient of similarity: 0.93)
0.4
(coefficient of similarity: 0.93)
0.2
0.3
A l
LolitaLempicka
0.1
0
m 2
AngelCinema
LInstant
0.0
Dim
CocoMelleJAdore_EPJAdore_ET
PleasuresPurePoison
-0.2
-0.1
AromaticsElixir
Chaneln5 Shalimar
0 2 0 0 0 2 0 4
-0.3
-
8013 22
-0.2 0.0 0.2 0.4
Dim 1
expert vs consumer: Multiple Factor Analysis
expertsconsommateurs
MFA partial points’ representation
2
Angel
LolitaLempicka(RV coefficient: 0.87)
1
9.35
%)
AngelCinemaLInstant
0
Dim
2 (1
Chaneln5
CocoMelleJAdore_EPJAdore_ET
PleasuresPurePoison
Shalimar-1
AromaticsElixir
-2 -1 0 1 2 3
-2
8013 23
-2 -1 0 1 2 3
Dim 1 (64.06 %)I di id l f t
expert vs consumer: Multiple Factor Analysisg
1
Vanille
Gourmand
vanilla honey
MFA variables’ representation
expert
consumer
camomille
anis
caramel(RV coefficient: 0.87)
(19.
35 %
) Fruité
Fraicheur marineEnveloppant
freshness
citrus
sweet_fruit
nutty
0
Dim
ensi
on 2
CapiteuxVertNotes.florales
Agrume
Fraicheur.marinefreshness
jasminrose
f h l
woodyleather
muskanimal
earthyincense
green
D
Epicé
Boisé
Oriental
intensity
fresh_lemon
spicy
-1
8013 24
-1 0 1
Dimension 1 (64.05 %)
Comparison through thefid lli t h iconfidence ellipses technique
(Husson, Lê & Pagès, 2005)( , g , )(Lê, Pagès & Husson, 2008)
8013 25
confidence ellipses
methodology
1.Compute the product profiles (averaged by product over the judges)2.Create the products’ space3 R l b b t t i l3.Re-sample by bootstraping new panels4.For each new panel, compute new products’ profiles5.Project as illustrative the products on the original product space6.Steps 3 to 5 are repeated many times (i.e. 500 times)7.Confidence ellipses around the products containing 95% of the data are constructedconstructed
principle• if ellipses are superimposed, the products are not significantly different• the size of the ellipses is related to the variability existing around the products
8013 26
p
confidence ellipses
2Confidence ellipses around the products
2
LolitaLempicka
1
.39%
)
Angel
CinemaLInstant
0
Dim
2 (1
9.
Chaneln5
CocoMelleJAdore_EPJAdore_ET
PleasuresPurePoison
-1
AromaticsElixir
Chaneln5Shalimar
-3 -2 -1 0 1 2 3 4
8013 27
Dim 1 (64.02%)
confidence ellipses
• as we have two different panels, we can apply this methodology to both
• creation of confidence ellipses around each product seen by each panel (24 ellipses are created here)each panel (24 ellipses are created here)
• comparison of a given product through the two panels (samecomparison of a given product through the two panels (same colour)
f ff (• comparison of the different products within a panel (same type of line)
8013 28
confidence ellipses
Confidence ellipses for the partial points2
1
9.39
%) Angel
CinemaLInstant
LolitaLempicka
0
Dim
2 (1
9
Chaneln5
CocoMelleJAdore_EPJAdore_ETPleasures
PurePoison
Shalimar
-2-1
AromaticsElixir
cons.expert
-4 -2 0 2 4
-
Dim 1 (64 02%)
expert
8013 29
Dim 1 (64.02%)
confidence ellipses
• partial points
• within a product, the ellipses related to the two panels are always superimposed (no differences between the panels)
• the sizes of the ellipses are equal• the sizes of the ellipses are equal• the higher amount of consumers compensate the higher variability due to the lack of training for consumers
8013 30
conclusions
• although consumers don’t have the habit to describe perfumesalthough consumers don t have the habit to describe perfumes (difficult task), they give the same information as the expert panel (and it’s identical to the standard description of the perfumes)
• they also have the same qualities (discrimination and reproducibility)
• a difference between consumers and experts panel exists in the variability of the results (more variability for consumers), but this is compensated by the larger size of the panel (here 103 vs 12)compensated by the larger size of the panel (here 103 vs 12)
• with consumers, not only intensity, but also ideal and hedonic co su e s, o o y e s y, bu a so dea a d edo cquestions can be asked in the same time
8013 31
references
• Earthy P., MacFie H & Hedderlay D. (1997). Effect of question order on sensory perception and preference in central locations. Journal of Sensory Studies, vol.12, p215-237237
• Gazano G., Ballay S., Eladan N. & Sieffermann J.M. (2005). Flash Profile and flagrance research: using the words of the naïve consumers to better grasp the perfume’s universeresearch: using the words of the naïve consumers to better grasp the perfume s universe. In: ESOMAR Fragrance Research Conference, 15-17 May 2005, New York, NY.
• Husson F Le Dien S & Pagès J (2001) Which value can be granted to sensoryHusson F., Le Dien S. & Pagès J. (2001). Which value can be granted to sensory profiles give by consumers? Methodology and results. Food Quality and Preference, vol.16, p291-296
• Husson F., Lê S.& Pagès J. (2005). Confidence ellipses for the sensory profile obtained by principal component analysis. Food Quality and Preference, vol.16, p245-250
• Lê S., Pagès J. & Husson F. (2008). Methodology for the comparison of sensory profiles provided by several panels: Application to a cross cultural study. Food Quality and Preference, vol.19, p179-184
8013 32
thank you
• special thanks tospecial thanks to• Melanie COUSIN• Maëlle PENVEN• Mathilde PHILIPPE• Marie TOULARHOAT
students from AgroCampus-Rennes, who took care of the whole expert panel data.
8013 33
Thank you for your attention!Thank you for your attention!