L ET ME COUNT THE WORDS Quantifying Open-Ended Interactions with Guests.

LET ME COUNT THE WORDSQuantifying Open-Ended Interactions with Guests

Pullman, Madeleine, Kelly McGuire, and Charles Cleveland (2005). Let Me Count the Words: Quantifying Open-Ended interactions with Guests. Cornell Hotel and Restaurant Administration Quarterly, 46(3):323-343.

SURVEY

Ask customers to choose from a scale (5=very satisfied, 1=very dissatisfied) We had 95% customer satisfaction last month We have improved our customer-satisfaction

rating in housekeeping by 3 percent over last year

What’s difference between 7 out of 10 and 8 out of 10?

Customer choose their own words and topics Comment cards, survey comments, phone

interactions and Internet chat rooms and bulletin boards

Rarely discussed

BENEFITS OF OPEN-ENDED INTERACTIONS

Provide his/her own view, definition and context with no limitations

No guiding bias in the comments Words can be evaluated for emotional and

other behavioural ideas related to satisfaction and repurchase intent or other loyalty behaviour

DRAWBACKS OF VERBAL INTERACTIONS

Customers lacking verbal skills may have difficulty articulating their ideas in a meaningful way

Could generate irrelevant information There could be measurement and

sampling problems from certain sources of qualitative data

use qualitative sources as a complement to traditional survey data

CURRENT INDUSTRY PRACTICES Comment card

Please provide comments or suggestions Guest surveys

Ten to thirty questions using a Likert-type scale along with one or two small areas for comments

Address business’s physical and service attributes areas and overall satisfaction and loyalty question Please rate your guest room from Excellent-5 to Poor-1 Please rate from Very Much Agree -7 to Very Much Disagree-1,

the following statements: during my stay, I was overall very satisfied; I would stay at this hotel again…

Barsky and Nash have twelve emotion-related questions Emotional reaction ratings could predict loyalty and price

elasticity for hotel rooms in different segments

In-depth interviews for capturing complex guest perception

CURRENT INDUSTRY PRACTICES

Source of open-ended comments A small section in a surveys Comment cards Transcripts of phone and emails (interactions

have been monitored for conformance to company expectations, e.g. proper language use)

few companies use this information in any systematic way

SOFTWARE PACKAGE FACILITATE THE WORD-ANALYSIS PROCESS Content analysis and data linking

Counting frequencies, sequences, or locations of words and phrases; connecting relevant data segments to each other; forming categories, clusters, or networks of information from the words

Linguistic analysis Identifying and counting key ideas, grammar use, and

behavioural context overall and relative to expected usage in a certain context

QUALITATIVE-DATA-ANALYSIS METHODS

Content analysis Label text phrases according to certain

predetermined categories (manually) Word-use-analysis package (WUAP, for large

volumes of textual data) Read some of the comments Run a count of word frequencies Categorize concepts and themes by means of a

user-defined dictionary of words and phrases major ideas and themes Has default dictionaries that can include or exclude

common words and phrases The category frequencies can be cross-referenced

with respondents demographic data to analyse which themes are important to which groups of customers

HOT-BEVERAGE VENDING MACHINE

Creating and categorizing major themes into the dictionary is the most complicated part Use default dictionary find synonyms Based on initial reading and create a new category

Cream control: whitening, cream, milk, amount, portion

Frequencies by demographic or other categorical data

Data linking (looking for combination of words or themes that respondents use in concert) Which menu items are getting negative rating and

why? Based on similarity index, matrix or tree

Presentation of the data

TREE DIAGRAM FOR IDEA SIMILARITIES

CaffeineChoice

LocationTaste

Customization

UsabilityCups

MachineCostTimeNeg

StrengthValue

PosTemperatur

eHealthNeeds

UncertaintyPayment

SweetDispensing

Trade

LIMITATIONS OF WUAP

Inability to interpret the meaning of a word in context High, positive or negative?

High ceiling vs. high school The best veal vs. not the best Very good, not very good, not good

LINGUISTIC ANALYSIS

Looks at natural language use, its meaning, and the behaviour associated with that language usage Semantics (ideas): proximity search Syntax (grammar)

Motivation: I will buy the Ritz pen pragmatics (the context of behaviour)

Rule-based behaviour, referencing standards set in the past “Ought”

Goal-based behaviour, referencing bottom-line future conditions that need to be met

“the” in business situation Emotion- or feeling-based behaviour, referencing gratification

or feelings of satisfaction with the moment Less “the” in social situation

Analytic or thinking-based behaviour, referencing the abstract, explanation and explaining behaviour or conditions

VIP HOSPITALITY TENT

Survey including Likert scale questions, open-ended questions, and loyalty-behaviour ratings Tell us about the highlights of the VIP experience

Segment comment texts using the combined score on loyalty-behaviour question (HL, ML, LL)

Intuitional-probabilities matrix

Different groups address different themes? The semantic parser used to identify, code and

count the number of people who used a key idea HL guests take a journey to a mystical place and

feel like they are entering another world

Value/Cost Judgement

I will not come again

Immersion in the whole experience,

emotional, intellectual, physical and

social engagement

I will tell others

I will come again

Worth

Being waited on 27%

Magical/

mystical 24%Involve

ment/ participation 21%

LINGUISTIC ANALYSIS OF CUSTOMER-SERVICE PHONE INTERACTIONS

Transcripts of customer phone interactions Resolve problems Complaint letters per month concerning

customer hone interactions Goal: improving customer satisfaction Initial benchmark

1,500 minutes of transcribed random telephone conversations (app. 150,000 to 200,000 words)

Successful conversation Trained to improve relationship-

building language and use language more similar to customers

When customer-service representatives used personal language with the customers, customers were more positive with the services

Angry customers became less angry and used more goal-oriented language when the representatives used their name at least three times in the conversation and used the pronouns “I” with “you”.

Representatives language use the customers’ language use moved closer Complaint letters decrease Second calls fell Customer-satisfaction rating increased Representative turnover diminished

IMPROVING CURRENT ANALYSIS

Conversation simulators, e.g. Socrates Web-based programs that analyse conversations

stemming from probing questions The food was bad You said the food was bad; can you tell me more about

that?

Monitoring chat rooms, forums

CONCLUSION

Analysing feedback surveys, in-depth guest interviews for new service-design improvements, performance monitoring, and strategy formulation