FINDING PATTERNS IN TEMPORAL DATA KRIST WONGSUPHASAWAT TAOWEI DAVID WANG CATHERINE PLAISANT BEN SHNEIDERMAN HUMAN-COMPUTER INTERACTION LAB UNIVERSITY OF.

FINDING PATTERNS IN TEMPORAL DATAKRIST WONGSUPHASAWATTAOWEI DAVID WANGCATHERINE PLAISANTBEN SHNEIDERMAN

HUMAN-COMPUTER INTERACTION LAB UNIVERSITY OF MARYLAND

27th HCIL Symposium

May 27, 2010

FINDING PATTERNS IN TEMPORAL DATAKRIST WONGSUPHASAWATTAOWEI DAVID WANGCATHERINE PLAISANTBEN SHNEIDERMAN

HUMAN-COMPUTER INTERACTION LAB UNIVERSITY OF MARYLAND

27th HCIL Symposium

May 27, 2010

Patient ID: 45851737

12/02/2008 14:26 Arrival12/02/2008 14:36 Emergency12/02/2008 22:44 ICU12/05/2008 05:07 Floor12/14/2008 06:19 Exit

Time

EmergencyICU

FloorExit

TEMPORAL CATEGORICAL DATA• A type of time series

04/26/2010 10:00 31.0304/26/2010 10:15 31.0104/26/2010 10:30 31.0204/26/2010 10:45 31.0804/26/2010 11:00 31.16

EventCategory

Stock: Microsoft

Numerical

Arrival

Event

TEMPORAL CATEGORICAL DATA

Electronic Health Records: symptoms, treatment, lab test

Traffic incident logs: arrival/departure time of each unit

Student records: course, paper, proposal, defense, etc.

Others: web logs, usability study logs, etc.

10+ years work on temporal visualization(mostly on Electronic Health Records)

LIFELINES

SINGLE

RECORD

[Plaisant et al. 1998]http://www.cs.umd.edu/hcil/lifelines

LifeLines – Single Patient

working with physicians at WASHINGTON HOSPITAL CENTER

EXAMPLE DATA• Patient transfers

ARRIVAL Arrive the hospital

EMERGENCY Emergency room

ICU Intensive Care Unit

INTERMEDIATE Intermediate Medical Care

FLOOR Normal room

EXIT-ALIVE Leave the hospital alive

EXIT-DEAD Leave the hospital dead

TASKS

within 2 days

ICU Floor ICU

• Example: Finding “Bounce backs”

LIFELINES 2

RECORDRECORD

RECORD

RECORD

RECORD

[Wang et al. 2008, 2009]http://www.cs.umd.edu/hcil/lifelines2

LifeLines2 – Search and Visualize

ARF (Align-Rank-Filter)Framework

Temporal Summary

Multiple Records

ALIGNMENT• Sentinel events as reference points

Time

Patient #45851737 ArrivalEmergency

ICUFloor

Exit

Patient #43244997 ArrivalEmergency

ICUFloor

Exit

June July August

ALIGNMENT (2)• Time shifting

Time

Patient #45851737 AdmitEmergency

ICUFloor

Exit

Patient #43244997 AdmitEmergency

ICUFloor

Exit

0 1 M 2 M

SIMILAN

RECORDRECORD

RECORD

RECORD

RECORD

[Wongsuphasawat & Shneiderman 2009]http://www.cs.umd.edu/hcil/similan

Similan – Search by Similarity

Similan – Search by Similarity

FINDING “BOUNCE BACKS”Before After

• Much faster to specify new query• Visualizing the results gives better

understanding

USER STUDIES: SEARCH

ExactMUST have A, B, C

Record#1

Record#2

Record#3

moresimilar

Similarity-basedSHOULD have A, B, C

SimilanLifeLines2

Query

Record#2

Record#1

Record#3

Query

USER STUDIES: SEARCH

ExactMUST have A, B, C

Similarity-basedSHOULD have A, B, C

Query

Record#1

Record#2

Record#3

moresimilar

Query

Record#2

Record#1

Record#3

SimilanLifeLines2

1

NEW STUFFNeeds for an overview -> LifeFlow!

TASKS

within 2 days

ICU Floor ICU

• Example: Finding “Bounce backs”

• Other questionsArrival

ICU

?

??

LIFEFLOW

RECORD

RECORD

RECORD

RECORD

RECORD

RECORD

RECORD

RECORD

AGGREGATEMerge multiple records into

tree

VISUALIZEDisplay the aggregation

AGGREGATE• Aggregate by prefix

#1

#2

#3

#4

Example with 4 records

AGGREGATE• Aggregate by prefix

#1

#2

#3

#4

VISUALIZE• Inspired by the Icicle tree [Fekete 2004]

Number of files

VISUALIZE (2)• Use horizontal axis to represent time• Video

DEMO – LIFEFLOW When the lines are combined into flow

FUTURE WORK• Comparison

Jan-Mar 2008 April-June 2008

IntermediateICU

IntermediateICU

Floor

TAKE-AWAY MESSAGE

Information visualization is a powerful way to explore temporal patterns.

You can work with us on new case studies.

TEMPORAL CATEGORICAL DATA

Electronic Health Records: symptoms, treatment, lab test

Traffic incident logs: arrival/departure time of each unit

Student records: course, paper, proposal, defense, etc.

Others: web logs, usability study logs, etc.

EXAMPLE – TRAFFIC INCIDENTS

ACKNOWLEDGEMENTDR. PHUONG HO, DR. MARK SMITH, DAVID

ROSEMANWASHINGTON HOSPITAL CENTER

http://www.whcenter.org

NATIONAL INSTITUTES OF HEALTH (NIH) - GRANT CA147489 http://www.nih.gov

MICHAEL PACK, MICHAEL VANDANIKERCENTER FOR ADVANCED TRANPORTATION

TECHNOLOGY LAB(CATT LAB)

http://www.cattlab.umd.edu

http://www.whcenter.org/

http://www.nih.gov/

http://www.cattlab.umd.edu/

TAKE-AWAY MESSAGE

Information visualization is a powerful way to explore temporal patterns.

You can work with us on new case studies.

More demos this afternoon{kristw, tw7, plaisant, ben}@cs.umd.eduhttp://www.cs.umd.edu/hcil/temporalviz

Q&AQuestions?

{kristw, tw7, plaisant, ben}@cs.umd.edu

http://www.cs.umd.edu/hcil/temporalviz

THANK YOUThank you

BACKUP SLIDESJunkyard...

LIFELINES2• 8 case studies

– Bounce backs– Step ups– BIPAP– Etc.

DR. P

Does not help exploring sequential patternsNeeds a new overview

LifeLines2’s Temporal Summary [Wang et al. 2009]

Continuum’s Histogram [Andre 2007]

USER STUDIES• 8 Extensive case studies• Compared LifeLines2 with Similan

– Learn advantages & disadvantages• Drawing is preferred• Clear cut off points is needed

• Working on improvements– Flexible temporal search

SIMILAN• Compared with LifeLines2 in an

experiment– Learn advantages & disadvantages– Drawing is preferred– No clear cut off points

• Working on improvements– Flexible temporal search

LIFEFLOW

AGGREGATE

VISUALIZE

Merge multiple records into tree

Display the tree

APPROACHESExact SearchMUST have A, B, C

Similarity-based SearchSHOULD have A, B, C

Query

Record#1

Record#2

Record#3

moresimilar

Query

Record#1

Record#2

Record#3

MOTIVATION

RESEARCH QUESTIONS

RESEARCHQUESTION#1

PRELIM. + PROPOSED WORK CONCLUSION

RESEARCHQUESTION#2

PRELIM. + PROPOSED WORK

EXPECTED CONTRIBUTIONS

1. Design of visual representations, user interfaces and interaction techniques

2. Algorithms for flexible temporal search

3. Evaluation results

4. Open new directions for exploring temporal categorical data

NEEDS FOR AN OVERVIEW• We learn

NEEDS

Visualize overviewor show summary

Where should I start?

TEMPORAL VISUALIZATIONSBackground and related work

RELATED WORK• Single record

Patient ID: 4585173712/02/2008 14:26 Arrival12/02/2008 14:26 Emergency12/02/2008 22:44 ICU12/05/2008 05:07 Floor12/08/2008 10:02 Floor12/14/2008 06:19 Exit

Visualization

• E.g. LifeLines, MIDGAARD, etc.

RELATED WORK (2)• Multiple records

VisualizationVisualization

VisualizationVisualization

• E.g. LifeLines2, Continuum, ActiviTree, etc.





More space please....

INFORMATION VISUALIZATION MANTRAOVERVIEW FIRST, ZOOM AND FILTER, THEN DETAILS ON DEMAND

RELATED WORK (3)• Multiple records

Visualization

• E.g. LifeLines2, Continuum





SEQUENTIAL PATTERNS

within 2 days

ICU Floor ICU

• Examples: “Bounce backs”

Patient #1

Patient #2

Patient #3

Patient #4

DESIGN AN OVERVIEW• Sequential patterns• Scalability vs. Loss of information

FINDING PATTERNS IN TEMPORAL DATA KRIST WONGSUPHASAWAT TAOWEI DAVID WANG CATHERINE PLAISANT BEN SHNEIDERMAN HUMAN-COMPUTER INTERACTION LAB UNIVERSITY OF.

Documents

query slide

similarity slide

aggregation slide

bounce backs slide

hospital dead slide

c record

lifelines single patient

record wang