"Findability and usability lessons learnt from text analytics" By: Anna Divoli At: CS4HS 2013 Unitec

Findability and Usability Lessons Learnt from Text Analytics

Anna Divoli@annadivoli

Pingar Research

CS4HS 2013 @ Unitec

twitter, July 2006Google, September 1998

http://www.laurencecaro.co.uk/other/what-the-worlds-biggest-websites-looked-like-at-launch/

http://www.laurencecaro.co.uk/other/what-the-worlds-biggest-websites-looked-like-at-launch/

Yahoo!, March 1995

Facebook, February 2004

http://www.whoisandrewwee.com/facebook/the-new-facebook-hot-or-not/

Often there is resistance!

Anna Divoli CS4HS 2013 @ Unitec

http://imgur.com/gallery/U8C1r

Users adapt & evolve as well!


15

• Familiar Interfaces• User Centered Design & Usability• Who am I & who is Pingar• Text analytics in a nutshell• Our usability studies & findings• 1. Search interface features: Biomedical Scientists • 2. Study existing popular systems • 3. Mock ups of specific features

• Take away message

Outline




Outline


http://blog.lib.umn.edu/torre107/si/pics/superficialintelligence14.jpg

Usability testing is important! Needs to be done right!


User Centered Design

Design

Prototype

User Feedback

Analysis


http://headrush.typepad.com/photos/uncategorized/focusonusersnotcompetitors.jpghttp://headrush.typepad.com/creating_passionate_users/2006/07/

User Centered Design – What should companies focus on!


FOCUS

Users Competitors

• Eye tracking• Mouse clicks• Tasks (goal & time)• Questionnaires• Observation• Talk aloud• Brain scanners• ….

Usability testing tools

Measurements & Preferences on:

• Aesthetics• Craftmanship• Technical performance• Ergonomics• Usability


http://gazehawk.com/blog/

Eye tracking




Outline


Who am I?


UKBSc (1999) Biomedical Sciences.

MSc (2001) Biosystems & Informatics.

PhD (2006) Biomedical Text Mining:- Sentence extraction- Automatic protein family database annotation- Document clustering and visualization

USA (2006-2011)Postdoctoral Academic Research:- Text analytics - Information retrieval- User search interfaces- Usability research- Knowledge acquisition- Expert opinions analysis

NZ (2011- )Pingar Research:- Entity Extraction- Automatic Taxonomy Generation- Taxonomy Editor (UI)- Friendly UIs- Usability & UX

Who is Pingar?


A small global company that uses state of the art text analytics to offer solutions!

We analyse unstructured text!

Entity ExtractionPeople, Organizations, LocationsDates, Money amounts, Email addresses…+ Custom Entities

Keyword Extraction

Taxonomy Term ExtractionDomain specific terminology (can plug in any taxonomy/ontology)

SummarizationSelect length and focus

Redaction & SanitizationEnsure anonymization of sensitive information.

In English, German, French, SpanishDutch, Chinese, Japanese



Outline


research A brief Text Analytics Overview – just Entity Recognition basics…


S. Arlington initiated partnership discussions during his visit to Eureka’s Ltd offices last month.

John Smith went to Washington to see the Smithsonian and also met up with Virginia for a coffee.

Find and classify names…

Information Extraction – Named Entity Recognition




Find and classify names…





Find and classify names… People Locations Organizations

Methods: lexicon-based (gazeteers)grammar-based (rule-based)statistical models (machine learning)hybrids






Methods: lexicon-based (gazeteers)grammar-based (rule-based)

✓ statistical models (machine learning) ✓ hybrids






Who? Where?

When?

Dates





Word Sense Disambiguation: identifying which sense/meaning of a word is used in a sentence, when the word has multiple meanings.

Text normalization: transforming text into a single canonical form that it might not have had before.

Disambiguation & Normalization:Word Sense Disambiguation & Text Normalization



Sam Arlington initiated partnership discussions during his visit to Eureka offices in September.


J. Smith went to Washington DC to see the Smithsonian Institute and also met up with Virginia Peterson for a coffee.

Word Sense Disambiguation & Text Normalization



Sam Arlington initiated partnership discussions during his visit to Eureka offices in September.


J. Smith went to Washington DC to see the Smithsonian Institute and also met up with Virginia Peterson for a coffee.

Word Sense Disambiguation & Text Normalization


What?



Fact & Relationship extraction


What?



Fact & Relationship extraction


How? Why? How do we feel about it?



Deeper knowledge & Sentiment


How? Why? How do we feel about it?



S. Arlington visited the Eureka’s Ltd offices last month to initiate partnership discussions.

John Smith was delighted to go to Washington to see the Smithsonian and also met up with Virginia for a coffee.

Deeper knowledge & Sentiment


S. Arlington did not initiate partnership discussions during his visit to Eureka’s Ltd offices last month.

John Smith went to Washington to see the Smithsonian and almost also met up with Virginia for a coffee.

Negation… Hedging… Conditional phrases…

Also:

Name boundaries… Disambiguation… Normalization…

Common problems


Eye drops off shelf.

Include your children when baking cookies.

Turn right here.

John saw the man on the mountain with a telescope.

He gave her cat food.

They are hunting dogs.

Human language!


Looking for: interactions between SAF and viral LTR elements

(SAF is a transcription factor, LTR stands for ‘long terminal repeat’)

Gene names:tinman, lilliputian, dreadlocks, lush, cheap date, methuselah, Van Gogh, maggie, brainiac, grim, reaper, cleopatra, swiss cheese, ken and barbie, kenny, out cold, lava lamp, hamlet, sonic hedgehog, werewolf, half pint, fucK, drop dead, chardonnay, agnostic, I’m not dead yet…

Example: Biology…


Knowledge management - Extracting metadata

Search engines

Question Answering

Text mining / knowledge discovery

Analytics & trend analysis

Applications




Outline


Specialized domain study:

1. Search interface features: Biomedical Scientists (HCIR)

Anna Divoli and Alyona Medelyan Search interface feature evaluation in biosciences, HCIR 2011, Google, Mountain View, CA

UI preferred features studies:

2. Study existing popular systems (EuroHCIR)

Matthew Pike, Max L. Wilson, Anna Divoli and Alyona MedelyanCUES: Cognitive Usability Evaluation System, EuroHCIR 2012, Nijmegen, Netherlands

3. Mock ups of specific features (survey)

Our User Studies


1. Search interface features: Biomedical Scientists


Anna Divoli and Alyona Medelyan, Search interface feature evaluation in biosciences, HCIR 2011, Google, Mountain View, CA, USA

Facets – favorite feature for search systems


animal models huntington disease

Facets (in search systems)


Most liked Least liked

animal models huntington disease

Bio-Facets


• Faceted search is the most important stand alone feature in a search interface for bioscientists.

• Few, query-oriented facets presented as checkboxes work best.

• Overly simple aesthetics, although not desirable, do not hurt overall UI score.

• Complex aesthetics turn users away from the systems.

• Bioscientists prefer tools that help them narrow their search, not expand it.

• For generic search: doc-based facets. For domain-specific search: query-based facets.

Facets as search features for biomedical scientists: Findings


• Useful categories• Simple• Vertical list

• Too complex/busy• Too many colors• Poor design• Limited functionality • Too many symbols• Not special/ Colorless

Facets as search feature: likes & dislikes




Outline


Exploring UI features - Systems Tested: Yippy, Carrot, MeSH, ESD








Boston Oct 2012

A

A B C D E F

B

C

DE

F

A B C D E F A B C D E F A B C D E F A B C D E F


• Menu highlighting• Hierarchical folder layout• Expand hierarchy with “+” and “–”• Dual view (tree on left, results on right)• Ability to change visualisations of taxonomy• Search function is important• Familiar interface with folders

• Too simple or too much writing - would be nice to have color• Lots of scrolling • Dots in carrot circle – confusing• Double click on foam tree is unintuitive• Too broad taxonomies

Exploring UI features (Yippy, Carrot, MeSH, ESD): likes & dislikes




Outline


60.0%26-4012.7%41-600%61 or older

27.3%25 or youngerAge:

52.7%College/University43.6%Graduate School

3.6%High School

Highest level of education:

47.3%Yes, but very little21.8%Yes

30.9%No

Do you have experience using taxonomies?

47.3%Very47.3%Second nature

5.5%Somewhat

How comfortable you are with computers?

Taxonomy UI preferences: The 51 participants


44.2%popularity (A)42.3%alphabetically (B)13.5%no preference

Concept sorting


42.3%A51.9%B5.8%no preference

Displaying Counts


72.5%in frames (A)23.5%with labels (B)3.9%no preference

Using Labels


47.1%A37.3%B15.7%no preference

Plus/minus signs or arrows


11.8%B70.6%C3.9%no preference

13.7%A

Search Results Display


74.5%partial64.7%hidden2.0%no preference

Search Functionality


Outcome: User-friendly Taxonomy Editor


Outcome: User-friendly Taxonomy Editor


Lessons Learnt


Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away. Antoine de Saint-Exupery, French writer (1900 - 1944)

• Follow basic design principles: alignments, right use of colors, etc

• Avoid clutter! Go for clean and simple always!

• Aim for intuitive User Interfaces. Do not make users think!

• You will never please everybody! Choose the users and use cases you plan to support.

Ignas Kukenys

PhD (2010) in Computer Vision, University of Otago

Previously: NextWindow

Expert in: computer vision, machine learning, classification methods

Alyona Medelyan

PhD (2009) in Natural Language Processing, University of Waikato

Previously: Google, European Media Lab

Expert in: keyword and entity extraction, Wikipedia mining

Ralf Heese.

Diploma in Computer Science, PhD candidate, Humboldt-Universität Berlin.Previously: Humboldt-Universität Berlin, Freie Universität Berlin

Expert in: semantic databases, semantic technologies

Andy Chao

BSc (2011) in Computer and Information Sciences, AUT

Hayden Curran

Computer Science, University of Waikato (2008-2010)

Collaborators

Annika Hinze Ian Witten Max Wilson Jeen Broekstra

Alumni

Matthew Pike Steve Manion David Milne Jose Chen Anna Huang

Anna Divoli.

PhD (2006) in Biomedical Text Mining, University of Manchester.Previously: University of Chicago, UC Berkeley, Inpharmatica

Expert in: biomedical text mining, text analytics, user search interfaces, usability

Waikato Uni Waikato Uni Nottingham Uni Rivuli Development

Nottingham Uni PhDc Canterbury Uni PhDc CSIRO postdoc NIH fellow Yangtze Uni Lecturer

Acknowledgements: Pingar Research Team

And all 65+ anonymous

studies participants!

http://farm5.static.flickr.com/4087/5019944289_5e2f0637c9.jpg

Take away message: Involve the user!


"Findability and usability lessons learnt from text analytics" By: Anna Divoli At: CS4HS 2013 Unitec

Technology

text analytics anna

pingar text analytics

usability lessons

usability studies findings

unitec uk bsc

art text analytics

unitec focus users competitors

search interface features