Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
A presentation from the NewMR ‘Listening is the New Asking’ Text Analytics Event - March 8 2011
The sponsor of the ‘Listening is the New Asking’ event is Zinc Research For more information about NewMR events visit newmr.org
The copyright for this material is jointly owned by The Future Place and the presenter.
‘Online Text Analytics – Top Ten Tips: Welcome to the oldest new game in town’ Anna Tomkowicz Freshminds Research, UK
Online Text Analytics – Top Ten Tips Welcome to the oldest new game in town 8th March 2011
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
There are a lot of conversations going on out there...
> In 2010, Twitter users sent 25 billion Tweets and more than
100 million new accounts were registered.
> More than 3.5 billion pieces of content are shared each week on Facebook.
> Purpose built Facebook pages have created more than 5.3 billion fans.
> In 2009, there were 126 million blogs on the internet.
Source: e-consultancy, Twitter
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
Powered by
... and lots of automated tools to analyse them
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
It’s all really confusing
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
I will take you through Top Ten Tips for using automated tools for text analytics
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
0
1000
2000
3000
4000
5000
6000
7000
Forum
Blog
Different tools lead to different results
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
Check which areas of the internet your tool is actually monitoring 1
• You are probably missing out on data on closed communities. Most tools are unable to trawl private areas on Facebook.
• You can use youropenbook.org to source
quotes from open profiles, but there is no tool enabling robust analysis.
• There are serious ethical implications in
sourcing quotes from closed communities.
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
Check how your tool recognises location or nationality 2
• Most tools use the domain name to determine location (com vs. co.uk), which may lead to bias.
• The language used on a forum isn’t
necessarily indicative of the user nationality.
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
Check how recent your data is 3
• Some tools provide you with fresh data every day, with mentions from the most popular websites coming in first.
• Other tools may lag up to a week.
• Sometimes, speedy delivery of the data happens at the expense of accuracy, so a quick data set may not be a very good one.
• Make a decision – do you need insight from great quality data, or the ability to quickly react at the first indication of a crisis?
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
Automated sentiment analysis is difficult 4
20%
70%
10%
• Tools use complicated algorithms to detect sentiment.
• There are certain limitations, e.g. challenges
around irony.
• Most tools are between 60%-85% accurate, but if you look at the usual sentiment metrics observed in consumer goods, you will see that if you mark all mentions as ‘neutral’ you will be on average 70% accurate.
• Accuracy goes down once you compare automated coding with manual coding on a one by one basis as opposed to on aggregate.
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
Emerging topics are sensitive to the search terms used
Topic Alterian Biz360 Brandwatch Nielsen Radian6 Scoutlabs Sysomos Baristas Yes
Burger King Yes
Cannabis Yes
Caribou Yes
Celebrity Yes Yes
Costa Yes
Credit Cards Yes
Fair Trade Yes
Falling in love Yes
Flat White Yes Yes
Freebies Yes
Gold Card Yes
Goo Yes
Guns Yes Yes Yes Yes Yes Yes Yes
Keep going Yes
Proximity Yes
Rewards Yes
Service Yes
Sundays Yes
Survey Yes
Wifi Yes
5
• Imposing a search term can have an impact on all your results and terms you are detecting
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
Which leads us to our over-arching tips
6 • Make sure you have clear business objectives before you commit to online text analytics.
7 • Don’t try to distill the analysis to a single metric.
8 • Use multiple data sources and robust sampling techniques.
9 • Tools change every week, so stay abreast of the developments and changes in methodology.
10 • Challenge the findings!
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
Q & A
Anna Tomkowicz FreshMinds
Ray Poynter The Future Place
Anna Tomkowicz, FreshMinds, UK NewMR “Listening is the new asking” – Text Analytics and Market Research, March 8, 2011
FreshMinds Research
FreshMinds Research is an award-winning research-led consultancy specialising in qualitative & quantitative market research and business analysis. We bring together the brightest minds* and the freshest thinking to create effective, commercial solutions to every client challenge. Clients include HSBC, The Cabinet Office, innocent and eBay; from tech start-ups to global FMCG companies. And no two projects are the same. Our approach is always fresh and energetic, our focus always commercial. We aim to inspire, offer new perspectives and help you move your business forward. * (ours, yours, consumers’ and experts’)
For more information, please contact:
Anna Tomkowicz Lead Research Manager [email protected]
020 7692 4554
Phil Burgess Head of Consumer Research
[email protected] 020 7692 4549
FreshMinds 229-231 High Holborn
London WC1V 7DA www.freshminds.co.uk
T: 020 7692 4300 F: 0870 460 1596