1
ImaginariumMerging science and practice
David Kirsh, UCSD Cognitive Science, Assoc. Director ACCHI
Erik Viirre M.D. Ph.D., UCSD School of Medicine
Sheldon Brown, UCSD Visual Arts, Director of ACCHI
Plus collaborators…
Objectives1. Observation: Study the nature of distributed creativity
in the natural setting of:
1. Brain storming
2. Design
3. Creative Expression
2. Theory: Build a more comprehensive theory of activity
and sensory involvement during brainstorming
3. Design: Explore impact of changes in environment
design on imagination, creativity and problem solving
Imaginarium Technology
• Binocular eye tracking among groups
• Video cameras coordinated with eye tracking
• Kinects for gesture detection
• Portable ERP caps
• Precise sound localization
• Whiteboard and other surface capture
• Automated Annotation of design session
• Remote interaction with other sites
Analytic Techniques
• Ethnographic coding, activity analysis,
• Datamining for patterns among different capture records
• Machine learning using VATIC and crowd sourcing to classify events defined via activity analysis
• Analyses using theories of distributed, situated and embodied cognition
Collaborators Relevant Research Basis
• Kirsh: NSF on Distributed Creative Cognition– In choreography – being expanded to music
• Makeig, Saygin: ERP (Event Related Potentials) during activity
• Trivedi: Video and Auditory awareness systems.• Viirre: Eye Tracking for wayfinding, medical diagnosis
and therapy• Brown: User analytics of virtual worlds
Observation Set-Up for Distributed Creative Cognition in Choreography - Kirsh
Mandeville Auditorium
Cal-IT2
Video Engineering
• Wide Angle Camera to identify head• High resolution camera slaved to get
face/eye activity• Analysis of video image (visible/IR)
• Tool for Collaboration• Centered around whiteboard, display
table, workbench– Annotation– Command interaction (Kinect)
Medical Evaluation• Monitoring of Movement control
– Fall Risk– Movement control– ADL
• Evaluation of Disease progression– Vestibular– Parkinsons
• Diagnosis?– Cerebrovascular vs. Neurodegenerative
Research Questions
• Are binocular indices sensitive indicators of neural disruption
• What do “natural” eye movements look like?– Combined Eye-Head– Head Free
• Scan paths
Embedded Cognition Theory
• Interaction with physical objects– Mental planning, practice
• Learning theory of tasks
Principals
• Erik Viirre, Medicine• David Kirsch, Cog Sci• Mohan Trivedi, Engineering• Sheldon Brown, Visual Arts