Synchronous Collaboration and Awareness Systems Bo Begole Ubiquitous Computing Area Manager Computer Science Lab UC Berkeley, Sep 25, 2006.

Synchronous Collaboration and Synchronous Collaboration and Awareness SystemsAwareness Systems

Bo Begole

Ubiquitous Computing Area ManagerComputer Science Lab

UC Berkeley, Sep 25, 2006

Synchronous Collaboration and Synchronous Collaboration and Awareness SystemsAwareness Systems

Degrees of Interdependence Replicated Application Sharing: Flexible JAMM Multi-user UNIX terminal: SharedShell Awareness:

– ConNexus– Awarenex

Presence and Availability Forecasting:– Rhythm Awareness– Lilsys

Why CSCW Research is ImportantWhy CSCW Research is Important

Inter-Personal Computing Most of what we do with computers is communicated

to others– Documents– Information Analysis– Even calculating Ballistic Missile trajectories are a form of

communication (“I hate you”) CSCW combines systems and social research

Grudin’s Time/Space Model

CoLocated Remote

Synchronous Meeting RoomsVideo

conferencing

AsynchronousArgumentation

ToolsEmail

Degrees of InterdependenceDegrees of InterdependenceModerately

Interdependent work

Asynch

Highly interactive Interdependent

work

WeaklyInterdependent

work

Semi,Peri,Psuedo-synch Synchronous

ProductivityTools

CommunicationTools

ProposalModule-level programming

ReviewsSystem design & prgrmng

NegotiationSystem integration & debugging

TwoExamples

Research Funding:Technology Development:

IMSMS

Push-to-talkaudio

Full-duplexaudio

Face-to-facemeeting

emailBlogs

Usenet

Webpages

PaperMemos Text chat

Shared Apps

WikisShared FileSystems

Editors

MeetingSupportSystems

MultiplayerGames

DecisionSupportSystems

DistributedPresentation

Browsers &other Viewers

TimeSpent

TaskCriticality

Collaboration Transparency

• Sharing single-user legacy applications• Application source code is not modified• Runtime environment is modified• sharing mechanism is “transparent” to application.

• Synchronous “Application sharing” for• Pair programming• Debugging/integration• Collaborative document editing

• Examples: • NetMeeting, WebEx, GoToMyPC, SharedX, SharedApp, XTV,

SunForum, Timbuktu, etc.

What You See Is What I See

• Collaboration is grounded in shared view, but• prevents independent work

• Collaboration is grounded in shared view• Are there downsides to WYSIWIS?

Maintaining Awareness: What You See Is What I Think You See

Conventional Collaboration Transparency Architecture

NetworkTraffic

DisplayDisplay UserInput

UserInput

DisplayBroadcaster

ConferenceAgent Merged

Input

ApplicationConference Agent Host

User A Host User B Host

• Centralized architecture

• One copy of application

• Remote inputs merged

• Graphics output sent to each remote participant

• Used by all collaboration-transparent systems

Does this model human

collaboration?

• Input events can interleave and conflict

• Solution: take turns using “floor control”• How well does turn-taking model human

collaboration?

Concurrency Control

(a) intended result of two users drawing curves simultaneously

(b) unintended result due to conflicting mouse movement events

Limitations of Collaboration-Transparency Systems

• Strict What You See Is What I See

• Slower application responsiveness

• No concurrent work

• Limited group awareness information

• Higher network bandwidth requirement than collaboration-aware applications

Collaboration-Aware Applications

• Applications designed for collaborative use• Examples:

• Editors – SASSE, Calliope, SubEthaEdit, Writely • Whiteboards – innumerable • Chat – ICQ, AIM• Visualization – CAPI, Shastra, Sieve• Work flow – TeamRooms, Groove• Learning – LiNC, CoVis• Games – Diablo, Doom, WorldOfWarcraft, There• Toolkits – Habanero, Tango, GroupKit

Collaboration-Aware Applications

Shared editors

Shared whiteboards

SubEthaEdit

Workspace Awareness

• Information about participants:• identity• location• activity• access

Telepointers to represent remote mouse cursors

“radar” views to indicate remote scroll positions

• Copy on each host

• Remote inputs merged

• Inputs distributed

• Enables:• Lower network bandwidth

• Independent views

• Concurrent work

Replicated Architecture

NetworkTraffic

DisplayDisplay

User B HostUser A Host

UserInput

UserInput

EventBroadcaster

MergedInput

ConferenceAgent

Conference Agent Host

Appli-cation

Appli-cation

Can We Achieve Spontaneous Collaboration?

• Co-workers “encounter” each other• Accessing shared content, docs, code, etc.• Within shared events, web sites, meetings, etc.

• Applications morph into collaborative versions on-the-fly

• Research prototypes• Flexible JAMM [Begole et al. 99]• Zipper [IBM]• Co-Word/Co-PowerPoint [Griffith U.]

Flexible JAMM: Replication + Object Replacement

Single- to Multi-user Object Replacement

(a) Original application

Applet

ScrollPane Panel

ButtonButton

Buttonobjectreference

(b) Shared application

Applet

Multi-userScrollPane

Panel

Radar ViewButton

ButtonButton

Replacement Process

• Java Object Serialization• Table of replaceable classes and equivalents

• JScrollPane => JRadarPane• PlainDocument => SharedDocument• also Image, FontMetrics, etc.

• Multi-user replacement class must • be subclass of single-user original• have constructor that takes single-user object and initializes to same state

New-comer HostInitiating Host

Replacement Filter

Concurrent Editing using Operational Transformation

“Thanks your, ...”

“Thanks your, ...”

0

1delete(10, 1)

“Thanks you, ...”

“Thanks to ou, ...”

“Thanks to you, ...”

“Thanks to your, ...”

insert(7, “to ”)

insert(7, “to ”)

delete(10, 1)delete(13, 1)

“Thanks to you, ...”

Goal: “Thanks to you, ...”

What About System Resources?

• E.g., File, system time, network sockets?• “Externalities” cannot be replicated

completely• Some services are unique per host: e.g., time,

hostname, etc.• Different file systems, paths, etc.

• Replicating files is partial solution, but• Paid network services may limit to one connection• Redundant output

• Don’t want to send multiple email

Flexible JAMM’s Proxied System Resources

OriginalApplet DIS FIS

MasterCopy

FISServer FIS

FISProxyDIS

RMI

JoinerCopy DIS

FISProxy

Does this still qualify as a “replicated” architecture?

Does it Work? Could it Introduce Problems?

• Flexible JAMM versus NetMeeting• Two tasks

• Loosely-coupled collaboration - Text Entry• Two authors simultaneously enter text

• Tightly-coupled collaboration - Copy Edit• “Editor” leads “author” to correct errors in existing

text

• 8 pairs, with 35 wpm typing proficiency

JAMM versus NetMeetingseconds to complete task

223.75247.50

353.50

266.00

0

50

100

150

200

250

300

350

400

Text Entry Copy Edit

Task Type

se

co

nd

s

JAMM NetMeeting

Performance Time

• Text Entry: less time using JAMM than NetMeeting • 223.75 sec vs.

353.50 (p<.001)

• Copy Edit: no difference• (p = 0.7905)

User Perceptions - Text Entry

Q1: satisfaction with the softwareQ2: ability to work simultaneouslyQ3: ease of controlling the shared

application

Q4: ease of indicating text locations Q5: ease of simultaneous editingQ6: ability to have independent viewsQ7: ease of knowing partner's view

User Perceptions - Copy Edit

Q1: satisfaction with the softwareQ2: ability to work simultaneouslyQ3: ease of controlling the shared

application

Q4: ease of indicating text locations Q5: ease of simultaneous editingQ6: ability to have independent viewsQ7: ease of knowing partner's view

Evaluation - other results

• No difference in accuracy• JAMM preferred for Text Entry• Neither preferred for Copy Edit• JAMM preferred overall• NetMeeting floor control mechanism is

difficult for people to use

Sun SharedShell

Designed for Sun Customer Care Center

Support Engineers “can’t see through the telephone”

SE and Customer have knowledge to jointly solve the problem

SE teaches customer how to solve the problem, reducing future support calls

SharedShell Video

Yankelovich, N., “Bo” Begole, J., and Tang, J. C. 2000. Sun SharedShell tool. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work (Philadelphia, Pennsylvania, United States). CSCW '00. ACM Press, New York, NY, 351. DOI= http://doi.acm.org/10.1145/358916.361981

Awareness through the ages

[Saddler & Hill, Apple, ‘97]

Montage, ‘94

Peepholes, ‘96

WatchMe, ‘03

AIM, ‘97

ICQ, ‘96

More to come?

# isthere <userid># campon <userid># talk <userid>

UNIX, ‘80sPortholes, ‘92

athena% zlocate username athena% zwrite username

Zephyr, ‘87

Intellisync, ‘05

Awarenex, ‘01

Hubbub, ‘02

Awarenex

Awareness Finding a good time to make contact Non-disruptive approach and leave-taking mechanisms

Integrated communication Making contact easy

Ubiquity Multiple clients Shows location

Awareness nexus

Presence & Awarenss Service

Collect, aggregate and

propagate data

“Presence” = can be reached

● No info when recipient is not present● Presence ≠ Availability

Buddy list is nearly useless when someone is not Present

● Inactive duration– Longer implies more

likely not present, but– Could be reading,

talking, etc.● Don’t really care how

long inactive, what I want to know is when/where/how can I reach them in the future?

Rhythms in Group Coordination

Temporal patterns of behavior: Rhythms Start/end of day, commutes, lunch, regular breaks, recurring

events, typical durations, … Zerubavel [1981] found that workers in a medical operations

can infer the time of day from surrounding activities Rhythms used to plan communications

“How long will Jane be away?” “Where might she be next?” “When will she be in her office for 15 minutes?” “What time does she leave for the day?” “When will she read my email?” “When should I worry if there’s no response?”

Difficult for distributed groups to form

E. Zerubavel, Hidden Rhythms: Schedules and Calendars in Social Life, Chicago: The University of ChicagoPress, 1981.

Computer Activity

Time of Day

Date

Computer Activity over 6 weeks

Key Factors Affecting RhythmsDay of Week

Location

Office

Home

Home

Office

Predictability varies between individuals

Mean and std dev of minutes active in 15 min window

programmer

manager

Human-observable patterns in Presence History

Probability of computer activity in office during Mondays

Startof day

Lunch Endof day

Lower presence probability

Mondays – Office80%

60%

40%

20%

0%

Lower presence probability

Modeling Approaches

Bayesian Network Decision Tree

[Horvitz, et al. 2002]

Difficult for end-users (and developers!) to interpretTemporal periodicity and patterns are not apparent to users

Goal: Descriptive model of temporal patterns to augment user’s mental model of rhythms

Identify and describe “Transitions” Significant changes in probability of presence Start, end of day, commute, recurring inactive periods

When, how long, how frequent Types of transition

Recurring transitions between locations Start- and end-of-day transitions Recurring mid-day transitions

EM approach to find mid-day transitions1. Estimate possible transitions2. Expectation Maximization

2.a Cluster instances using transition estimates2.b Refine estimates and iterate until converge

Step 1: Estimate Transition Periods

Determine threshold levels Upper and lower thresholds to filter noise

Threshold crossing: potential transition Initial estimate of transition properties

Transition TransitionTransition

Mondays – Office

Initial upper and lower thresholds determined

heuristically

Step 2: Cluster observed inactivity periods by distance from estimated periods

Euclidean distance

An observed inactive period is a member of cluster when distance < 3 An inactive period may differ by 2 stddev in two properties, but not all three

Sqrt ofStart End Duration

2 2 1 < 31 2 2 < 32 > 2 2 > 3

Distances from Transition time

Estimated Transition

Observed Inactive Periods

f p1, p2

start

start

2end

end

2duration

duration

2

start end

Not in the transition’s cluster

Example Rhythm Model

Mgr

Mondays – Office

Transition frequency and probability distributions of start, duration and end times

Example with Location Transition

Mondays – All Locations

Commute time: 45 – 80 mins

Starts work from home very early Monday mornings, then commutes to office

End-user Visualizations – Which are Easier to Interpret?

A.

B.

C.

D.

E.

Integrating Rhythms and IM

John office (10m)Lunch (66%)

Probability that Inactivity is a Transition

CurrentInactive Period

HistoricInactivePeriods

12:15 12:25

Lunch Period

Not Lunch Period

Current Inactive Period

8 of 12 past periods that were at least aslong as current were instances of lunch.

12:15 ±start of lunch

Out to lunch?

=> 66% probability gone to lunch

Suppose:

John office (20m)Lunch (80%)

Probability that Inactivity is a Transition

CurrentInactive Period

HistoricInactivePeriods

12:15 12:35

Lunch Period

Not Lunch Period

Current Inactive Period

8 of 10 past periods that were at least aslong as current were instances of lunch.

12:15 ±start of lunch

Out to lunch?

=> 80% probability gone to lunch

Suppose:

OutlineOutline

Presence and Non-Presence Rhythm Awareness

– Modeling– Applications

Availability and Unavailability Lilsys

– Technical details– User reactions

Future Directions

Presence Presence ≠ Availability≠ Availability

Interruptions are necessary– and welcome when related to

current task [Sproull ’84, J. Hudson, et al. ’02]

But carry costs– 15-20% of time spent on interrupts [Solingen ‘98, Czerwinski ‘04, Gonzalez & Mark ‘04] – £139bn annual loss in UK (14.6% GDP) [Brother Industries Ltd. 2005]

» roughly $1.75tn US

Willingness to be interrupted depends on – Situation, activity, task, relation, topic, time, etc.

Inferencing based on situation can approach human accuracy [Fogarty 2004]

Two philosophies on infering Two philosophies on infering availabilityavailability Human interpretation

Awarenex[Saddler & Hill, 97]

Machine interpretation

Salient Factors in Detecting Salient Factors in Detecting Unavailability in an OfficeUnavailability in an Office

Predicting human interruptibility with sensors: a Wizard of Oz feasibility study

Social engagement was the most salient factor Top 3 factors

1. Speaking

2. Telephone on/off-hook

3. Keyboard/mouse activity

Minor factors: sitting/standing, eye gaze on monitor, drinking water, etc.

Door was not a factor

[Hudson, Fogarty et al. 2003]

Lilsys

Motion Detector

Indicator Lights

Sound Detector

Online/Offline Toggle

Override Timer

Client Interface

Sensor-enabled

ProbablyUnavailable

PossiblyUnavailable

Usage ObservationsUsage Observations

“Online/offline” switch not used– but its presence was reassuring to some

“Override: Not Available” not used Individuals weigh indicators differently

– Users desire more control over inference Observers reverse-infer unavailability

– Users guess reasons why remote party is considered unavailable

– Diminishes value of hiding sensor data

Did it work?Did it work?

Can there ever be a good time to interrupt?– Users perceive same amount of interruption

» Also found in MyVine by Fogarty, Lai, Christensen 2004– But the interrupter “approached” more politely

Interruptions in face-to-face– Approach is shaped by availability assessment– Both parties aware of degree of intrusion– Recipient gives feedback on appropriateness by being gracious

or annoyed You can’t hold a caller accountable if they can’t be

expected to know your interruptibility Availability inference helps contact negotiation

What’s next?What’s next?

The genie is out of the bottle– Scott McNealy: 'You already have zero privacy anyway, get over

it.'

[Reality Mining, MIT ’05]

[IMWatching.net, 2005]

Impression ManagementImpression Management

Awareness complicates Impression Management [Goffman]

– Harder to “give” intended impressions– Harder to know what is “given off” (i.e., others see)– Harder to repair undesired impressions

Support impression management– Show what others can and have seen– Give user control of inference rules– Modify personal data– Give users “reverse Digital Rights Mgmt”

» E.g., if rhythm model shows Janet takes long lunches

» Show that Janet attends meetings during lunch Janet reads email from home in evenings Janet had the most sales last quarter, etc.

How?

When is Dagwood Available?When is Dagwood Available?

Rhythm and Unavailability InferencingRhythm and Unavailability Inferencing

Dagwood office {Lunch, ETA < 12:47}

Dagwood office {5:30 < ETD}

Dagwood office {ETA < 3m}

Rhythm Inferencing

Dagwood office

Dagwood office

Unavailability Inferencing

Dagwood office {ETA ~9:10 am}

Synchronous Collaboration and Synchronous Collaboration and Awareness SystemsAwareness Systems

Bo Begole

Ubiquitous Computing Area ManagerComputer Science Lab

References "Incorporating Human and Machine Interpretation of Unavailability and Rhythm Awareness into the

Design of Collaborative Applications", James "Bo" Begole and John C. Tang, to appear in HCI Journal. “If Not Now, When?:The Effects of Interruption at Different Moments in Task Execution”, P.

Adamczyk & B. Bailey, CHI 2004 "Beyond Instant Messaging", J. Tang & J. Begole, ACM Queue, (1)8, Nov 2003, pp. 28-37 “Lilsys: Sensing Unavialability”, J. Begole, N. Matsakis and J. Tang, Technical Note in CSCW 2004, to appear “Rhythm Modeling, Visualizations and Applications”, J. Begole, J. Tang, and R. Hill, UIST 2003 “Work Rhythms: Analyzing Visualizations of Awareness Histories”, Begole, Tang, Smith & Yankelovich,

CSCW 2002 “Coordinate: Probabilistic Forecasting of Presence and Availability”, E. Horvitz, P. Koch, C. Kadie, A.

Jacobs, UAI 2002 “'I’d Be Overwhelmed, But It’s Just One More Thing to Do:' Availability and Interruption in Research

Management”, J. Hudson, J. Christensen, W. Kellogg, and T. Erickson, CHI 2002 “Predicting Human Interruptibility with Sensors”, S. Hudson, J. Fogarty, C.Atkeson, J.Forlizzi, S.Kiesler,

J.Lee, J.Yang, CHI 2003 “The time famine: Toward a sociology of work time”, L. Perlow, Administrative Science Quarterly, 44,

(1999), 57-81. “When Can I Expect an Email Response? A Study of Rhythms in Email Usage”, J. Tyler and J. Tang,

ECSCW 2003 Hidden Rhythms: Schedules and Calendars in Social Life, E. Zerubavel, 1981