Video in CSCW Part I—Michael Boyle Part II—Michael Rounding
Dec 20, 2015
Part I—Video Mediated Communication
Role of visual channel in collaboration Video-mediated communication
technologies Opportunities and applications of VMC
Role of Visual Channel
Significance Function and operation of visual
behaviours Technological requirements Consequences of inadequate support
Significance of Visual Channel
Principle means for navigation and orientation in our 3D physical world Hand-eye coordination and motor skills
Principle means for experiencing our 3D physical world Other senses are limited Greatly inconvenienced if disabled
Characteristics of Visual Channel
Humans can only focus on small area Large peripheral visual cone
Humans are adept at handling vast quantities of visual information Scanning, searching, reacting to changes Recognizing, classifying
Predominantly diurnal creatures Large eyes; eyebrows; stereoscopic vision
Uses of Visual Channel
Infer much about people from visual information Gender, age, status Identity, background, personality
Bulk of human knowledge encoded visually
Arts, aesthetics, entertainment Communication
Gaze—Function and Operation
Indicates locus of interest, activity One generally looks at something when one
talks about it or is about to use it Uses: grounding, workspace awareness
Signals affective state Interest, disposition, attitude, emotion Uses: pre-interaction informal awareness,
conversational awareness
Gaze—Function and Operation
Conveys extremely subtle yet significant content Infer big meaning from even small physical
changes Although important, not always observed
Observed <7% of conversation time Mutual gaze (eye contact) <5% of time
Gaze—Technological Requirements
Careful camera position Eyes must be fully visible, aperture at eye-level
Large field of view See point of interest
High image resolution Clearly see orientation, percieve minor variations
High frame rate, low latency Smoothly convey rapid changes in real-time
Gaze—Consequences of Inadequate Support
Systemic miscommunication, misinterpretation if undiagnosed
Requires extra effort to explicitly communicate signals, or disambiguate/repair miscommunication
De-referencing focal points is tough!
Facial Expressions—Function and Operation
Involves whole of the face Eyes, eyebrows, forehead, mouth (especial
lips and corners), cheeks, ears, nose Conveys affective information
Blinks, winks, smiles, frowns, furrowed brows, wrinkled up nose, etc.
Uses: conversational awareness, disambiguate meaning of sensitive or easily-misinterpreted content
Facial Expressions—Technological Requirements
Tight audio/video synchronization Face must be in full view with high
resolution Fast frame rate to capture winks, etc.
Facial Expressions—Consequences of Inadequate Support
If latencies are too high, then signals may come at inappropriate times
If resolution too low, may not be able to clearly discern expression, adding to ambiguity
Could lead to break-down of conversation if important expression miscommunicated
Gestures—Function and Operation
Uses mostly hands, fingers, and arms Position, orientation, movement Position relative to other parts of the body, or
other objects in environment Rapid, generally small movements
Context crucial for interpretation Silent, subtle, implicit long-distance
communication
Gestures—Classes
Pointing gestures Grounding: de-reference deictic references
Emblematic or iconic Convey content Useful when words don’t come to mind Sometimes redundant
“Beats” Convey emphasis
Gestures—Technological Requirements
Large field of view Full body must be visible
High image fidelity Resolution, frame rate high enough to capture
small, rapid motions Likelihood that there will be obstructions
Fixed, mono-camera configuration insufficient
Gestures—Consequences of Inadequate Support
Miscommunication, resulting in explicit communication
Misinterpretation of gestures could lead to conversation breakdown
Missed deictic references impairs workspace awareness
Loss of personable content
Posture—Function and Operation
Consequence of work Involves whole body
Particularly head, neck back, hips Position, relative orientation are cues to
look for Communicates affective information
Particularly availability: informal awareness
Posture—Technological Requirements
Field of view Must capture whole body yet provide
background for reference Fidelity doesn’t need to be high
Posture—Consequences of Inadequate Support
Incorrect assessment of availability during pre-interaction periods Unnecessary interruptions, missed
opportunities Erroneous assessment of affective
information Impact on smoothness, but likely not so
terrible
Visible Behaviours—Recap
Subtle yet significant, sometimes redundant
Signal attitude, disambiguate meaning Valuable personable aspect Conflicting technological constraints People can get by without them, but
interaction dynamics are fundamentally altered
Part I—Video Mediated Communication
Role of visual channel in collaboration VMC technologies Opportunities and applications of VMC
A/V Infrastructure
Cameras and displays Digital vs. analog
Microphones and speakers Digital vs. analog Half-duplex vs. full-duplex Microphone arrays, environmental audio Noise, feedback, attenuation
Network Infrastructure
Medium Digital vs. analog Dial-up, ADSL, T1, wireless
Quality of service (QoS) factors: Bandwidth, latency, jitter, delivery guarantees
Packet-switched vs. circuit-switched Broadcast/multicast vs. unicast
Audio Software
Properties: sample size, rate, number of channels
Compression: PCM, µ-Law, ADPCM, GSM/CELP, MP3
Buffer size: transmission overhead vs. latency
Filtering, mixing
Video Software
Lossy vs. lossless compression RLE (run-length encoding): lossless
replace runs of same-coloured pixels with count JPEG: lossy
uses signal-processing techniques to change representation of image to bring out redudant aspects
quantization throws away resolution in chroma channel not visible to human eye
entropy (Huffman) coding on result
Video Software
Intraframe vs. interframe compression “M-JPEG:” intraframe MPEG: intraframe/interframe hybrid
JPEG compress key frames Encode back and forward differences of non-key
frames Decreases bandwidth, increases latency
Quality: bits-per-pixel
General Impact of Inadequacies
Frame size, resolution, fidelity Subtle facial expressions, gaze not
discernable Lost of critical eye contact
Field of view Locus of attention not visible Parts of face, body used in expressions not
visible
General Impact of Inadequacies
Frame rate Subtle changes lost
Latency Cues come at inappropriate time
A/V synchronization Reduces value of redundant encoding
VMC Technology—Recap
Technology never perfect Bandwidth/quality trade-off Results:
Content communicated subtly and implicitly via visual channel must be communicated explicitly via some other channel, or lost altogether
People adapt, but personability of FTF is lost
Part I—Video Mediated Communication
Role of visual channel in collaboration VMC technologies Opportunities and applications of
VMC
Opportunities and Applications
Tele-presentation/distance learning Augmenting shared workspaces Supporting informal interaction Tele-presence and mediating contact
Function, operation, example systems, and lessons learned
Tele-presentation
Presenter/audience with presentation matter
Numerous configurations: with/without local audience multiple participants at each remote site multiple presenters at same/different sites
Formal Information dissemination applications
Tele-presentation—Lessons Learned
Audience likes it Enables participation Integrates well with other work tasks
Presenters not so favourable Value of back-channels Reduced local attendance
Sub-conversations
Augmented Shared Workspaces
Coordinating activities in shared workspace requires communication channels E.g., text-chat or telephone
Integrate shared workspace and shared communication channels to create a shared tasks environment
Traditional meeting tasks
Augmented Shared Workspaces—Lessons Learned
Diminished priority of video relative to audio
Call setup: minimize time, provide sense of approach
Requirements vary with task Critical mass Know your users
Casual Interaction Support
Promote social interactions of serendipitous nature Based on value of social interactions to
productivity “Seamless” integration of distant sites into
a signal interaction space Hope to overcome effect of distance on
social interactions and collaboration
Casual Interaction Support—Lessons Learned
Public reaction mixed Not 100% transparent: technology still a
barrier to forging new relationships Difficult to diagnose failures
Requires extended installation/observation periods
Tele-presence and mediating contact
Video makes for a subtle, unobtrusive means of delivering tele-presence for accruing distributed informal awareness
Easy, natural for people to capitalize on visual information to coordinate interactions
Summary
Role of visual channel in collaboration Technological foundations Opportunities for video in CSCW Challenges
Summary—Role of Video
Communication medium Coordination medium
Regulation and mediation of interactivity Subtle, implicit Important, but not in the obvious ways
Importance varies with task
Summary—Video Technologies
Present technology not very robust Bandwidth/fidelity trade-off Issues: latency, field of view Consequences of inadequate technology
Vary from minor inconvenience to conversation breakdown
Must understand when video is crucial Design should reflect this
Summary—Opportunities
Tele-presentation Meeting support Communication channel for shared work
activities Casual interaction Mediating interaction