LET ME COUNT THE WORDS Quantifying Open-Ended Interactions with Guests
Dec 25, 2015
LET ME COUNT THE WORDSQuantifying Open-Ended Interactions with Guests
Pullman, Madeleine, Kelly McGuire, and Charles Cleveland (2005). Let Me Count the Words: Quantifying Open-Ended interactions with Guests. Cornell Hotel and Restaurant Administration Quarterly, 46(3):323-343.
SURVEY
Ask customers to choose from a scale (5=very satisfied, 1=very dissatisfied) We had 95% customer satisfaction last month We have improved our customer-satisfaction
rating in housekeeping by 3 percent over last year
What’s difference between 7 out of 10 and 8 out of 10?
Customer choose their own words and topics Comment cards, survey comments, phone
interactions and Internet chat rooms and bulletin boards
Rarely discussed
BENEFITS OF OPEN-ENDED INTERACTIONS
Provide his/her own view, definition and context with no limitations
No guiding bias in the comments Words can be evaluated for emotional and
other behavioural ideas related to satisfaction and repurchase intent or other loyalty behaviour
DRAWBACKS OF VERBAL INTERACTIONS
Customers lacking verbal skills may have difficulty articulating their ideas in a meaningful way
Could generate irrelevant information There could be measurement and
sampling problems from certain sources of qualitative data
use qualitative sources as a complement to traditional survey data
CURRENT INDUSTRY PRACTICES Comment card
Please provide comments or suggestions Guest surveys
Ten to thirty questions using a Likert-type scale along with one or two small areas for comments
Address business’s physical and service attributes areas and overall satisfaction and loyalty question Please rate your guest room from Excellent-5 to Poor-1 Please rate from Very Much Agree -7 to Very Much Disagree-1,
the following statements: during my stay, I was overall very satisfied; I would stay at this hotel again…
Barsky and Nash have twelve emotion-related questions Emotional reaction ratings could predict loyalty and price
elasticity for hotel rooms in different segments
In-depth interviews for capturing complex guest perception
CURRENT INDUSTRY PRACTICES
Source of open-ended comments A small section in a surveys Comment cards Transcripts of phone and emails (interactions
have been monitored for conformance to company expectations, e.g. proper language use)
few companies use this information in any systematic way
SOFTWARE PACKAGE FACILITATE THE WORD-ANALYSIS PROCESS Content analysis and data linking
Counting frequencies, sequences, or locations of words and phrases; connecting relevant data segments to each other; forming categories, clusters, or networks of information from the words
Linguistic analysis Identifying and counting key ideas, grammar use, and
behavioural context overall and relative to expected usage in a certain context
QUALITATIVE-DATA-ANALYSIS METHODS
Content analysis Label text phrases according to certain
predetermined categories (manually) Word-use-analysis package (WUAP, for large
volumes of textual data) Read some of the comments Run a count of word frequencies Categorize concepts and themes by means of a
user-defined dictionary of words and phrases major ideas and themes Has default dictionaries that can include or exclude
common words and phrases The category frequencies can be cross-referenced
with respondents demographic data to analyse which themes are important to which groups of customers
HOT-BEVERAGE VENDING MACHINE
Creating and categorizing major themes into the dictionary is the most complicated part Use default dictionary find synonyms Based on initial reading and create a new category
Cream control: whitening, cream, milk, amount, portion
Frequencies by demographic or other categorical data
Data linking (looking for combination of words or themes that respondents use in concert) Which menu items are getting negative rating and
why? Based on similarity index, matrix or tree
Presentation of the data
TREE DIAGRAM FOR IDEA SIMILARITIES
CaffeineChoice
LocationTaste
Customization
UsabilityCups
MachineCostTimeNeg
StrengthValue
PosTemperatur
eHealthNeeds
UncertaintyPayment
SweetDispensing
Trade
LIMITATIONS OF WUAP
Inability to interpret the meaning of a word in context High, positive or negative?
High ceiling vs. high school The best veal vs. not the best Very good, not very good, not good
LINGUISTIC ANALYSIS
Looks at natural language use, its meaning, and the behaviour associated with that language usage Semantics (ideas): proximity search Syntax (grammar)
Motivation: I will buy the Ritz pen pragmatics (the context of behaviour)
Rule-based behaviour, referencing standards set in the past “Ought”
Goal-based behaviour, referencing bottom-line future conditions that need to be met
“the” in business situation Emotion- or feeling-based behaviour, referencing gratification
or feelings of satisfaction with the moment Less “the” in social situation
Analytic or thinking-based behaviour, referencing the abstract, explanation and explaining behaviour or conditions
VIP HOSPITALITY TENT
Survey including Likert scale questions, open-ended questions, and loyalty-behaviour ratings Tell us about the highlights of the VIP experience
Segment comment texts using the combined score on loyalty-behaviour question (HL, ML, LL)
Intuitional-probabilities matrix
Different groups address different themes? The semantic parser used to identify, code and
count the number of people who used a key idea HL guests take a journey to a mystical place and
feel like they are entering another world
Value/Cost Judgement
I will not come again
Immersion in the whole experience,
emotional, intellectual, physical and
social engagement
I will tell others
I will come again
Worth
Being waited on 27%
Magical/
mystical 24%Involve
ment/ participation 21%
LINGUISTIC ANALYSIS OF CUSTOMER-SERVICE PHONE INTERACTIONS
Transcripts of customer phone interactions Resolve problems Complaint letters per month concerning
customer hone interactions Goal: improving customer satisfaction Initial benchmark
1,500 minutes of transcribed random telephone conversations (app. 150,000 to 200,000 words)
Successful conversation Trained to improve relationship-
building language and use language more similar to customers
When customer-service representatives used personal language with the customers, customers were more positive with the services
Angry customers became less angry and used more goal-oriented language when the representatives used their name at least three times in the conversation and used the pronouns “I” with “you”.
Representatives language use the customers’ language use moved closer Complaint letters decrease Second calls fell Customer-satisfaction rating increased Representative turnover diminished
IMPROVING CURRENT ANALYSIS
Conversation simulators, e.g. Socrates Web-based programs that analyse conversations
stemming from probing questions The food was bad You said the food was bad; can you tell me more about
that?
Monitoring chat rooms, forums
CONCLUSION
Analysing feedback surveys, in-depth guest interviews for new service-design improvements, performance monitoring, and strategy formulation