From icons to mixed realities: how external representations help us learn Yvonne Rogers SLIS/Informatics QuickTime™ QuickTime™ QuickTime™ QuickTime™ and QuickTime™ and QuickTime™ and aTIFF (
Jan 02, 2016
From icons to mixed realities: how external representations help us
learn
Yvonne Rogers
SLIS/Informatics
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
Overview
• How can we design graphical representations to facilitate learning and understanding?
• How do they work?– Public symbols– Icons– Diagrams– Animations– Multimedia– Mixed realities
Claims about their added value
• Symbols can be universally understood without ever having been seen before
• Icons are easy to infer meaning and remember• Diagrams can make relationships explicit• Animations can convey dynamic changes• Multimedia can ‘enrich’ learning• Mixed realities provoke reflection
My research agenda
• What are the main cognitive benefits of different forms of external representations?
• Can we develop a generalisable theoretical account of how external representations work?
• Can this provide design principles and guidance?
• Can we show whether the theory and principles are valid and useful?
Sometimes not…
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
1/5 people thought it signified a kite flying area
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
3/5 people though it signified foot-and-mouth disease
Public symbols
• Can be easily understood
• Can be amusing
• Can be ambiguous
• Context is important
• May need to learn a ‘picture’ grammar
Icons - why use instead of text?
• Many objects and operations can be succinctly expressed in iconic form– e.g., folders, files, trash can, to save, to delete
• Many functions are poorly expressed in an iconic form– e.g., format, preferences, to quit
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Icons
• Can be easily understood and meaning remembered
• Can be easy to scan and locate in a set• Can be confusing if look alike - difficult
to discriminate• Direct mappings best, followed by
symbolic and then abstract
Diagrams
• Help us understand difficult and complex subjects (e.g. an ecosystem)
• Make relationships and interdependencies between elements explicit
• Make it easier to ‘read off’ information and see connections
Animated diagrams
• What is the added value?
• Same benefits as static ones but also…
• Can show invisible and dynamic processes
A static and animated diagram of how the heart works
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
Is a moving picture worth 1000 static ones?
+ Animations can be effective at displaying dynamic changes
+ They make explicit what is implicit in diagrams
- Difficult to keep track and ‘digest’
- Learners simply watch and assume they have understood
Is Multimedia any better?
• Multimedia typically consists of graphics, text, speech and links
• Provides more representations and more information
Multimedia
• Early educational CD-ROMs were designed as electronic encyclopedias
• Knowledge represented in multiple media based on
a resource/book model • Full potential of multimedia not being realized
• How could it be designed to facilitate the learning process?
Designing it to teach the ‘difficult stuff’
An example of ‘difficult stuff’:reasoning about a complex system using an abstraction
Example: What would happen if the mice were wiped out?
To answer need to:
• know factual knowledge (who eats what)• understand canonical links (arrows)• reason about knock on-effects (using representations in temporal and spatial order)
Pondworld multimedia environment
• Goal was to enable children to reason more effectively about ecosystems using food web diagrams
• Get the children to relate explicitly their concrete knowledge with the abstract formalism
• To achieve this by enabling them to manipulate and construct dynamic food web diagrams and to get feedback on their actions
Multimedia can be much more…
• Different elements can be conveyed simultaneously
• Can be interacted with in novel ways
• Studies of children using PondWorld showed significant improvement in ability to reason using diagram
External cognition approach (Scaife and Rogers, 1996)
• explain how we interact with external representations
• Identify the cognitive benefits and costs of using different representations
• provide principles to inform their design
Computational offloading
• the extent to which different external representations (5, five, |||||, V) reduce the amount of cognitive effort required to solve equivalent problems
45 x Forty-five times twelve IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII XXXXV12 IIIIIIIIIIII XII
• the amount of cognitive work the learner is expected to do when using the external representation
“Dynalinking”
• linking of multiple representations so that making changes in one makes changes in the other (e.g., Pondworld)
• Dynalinking allows people to 'read' and connect information explicitly, helping them to understand the relationship between the two
Module Computatio
nal
offloading
Form of MM
interactivity
Problem
Solving
Activity
Learning
Process
1. PondWorld
Simulation
High Click and tell
animation
learning of
feeding
relationships
in ecosystem
Factual
knowledge:
feeding
relationships
2. IntroWeb Medium dynalinking
between
animation and
formalism
Make links
between
animation and
formalism
Canonical
forms in
foodweb &
mapping
between
abstractions
3. LinkWeb Medium dynalinking
between
animation and
formalism
complete
partially
constructed
formalism
Canonical
forms in
formalism &
mapping
between
abstractions
4. EraserWeb Low dynalinking
with facility for
annotating
diagram
use formalism
to reason
about
ecosystem
behaviour
Temporal
effects of
extinction of
species
Mixed realities
• What is Mixed Reality? - physical world is combined with virtual world
• Coupling the familiar with the unexpected
• Offers new opportunities for learning and understanding, e.g., reflection
Mixing colors
• Very familiar activity using wet paints and brushes
• What happens when you provide new ways of mixing colors?– e.g. combining digital paints to cause a physical
object to change color
• What do young children do?• How do they reflect upon the ‘old-new’
experiences?
The Chromarium project
(i) Wet paints
(ii) Combine digital colors
(iii) Combine physical cubes with colored sides to mix digital paints on a wall
(iv) Combine digital colors to cause a physical windmill’s sails to rotate and change color
(ii) Move digital colored discs on top of each other, resulting in new digital
colors
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
(iii) Place two physical cubes next to each other on a tabletop resulting in new digital color appearing on wall
QuickTime™ and aYUV420 codec decompressorare needed to see this picture.
(iv) Move digital windmill sails causing physical windmill sails to
move and change color
QuickTime™ and aYUV420 codec decompressorare needed to see this picture.
Findings from Chromarium study
• Children were most inventive and creative when mixing colors using the physical cubes
• Encouraged more collaboration and explaining of the ‘what’ and ‘why’
Summary: From icons to mixed realities
• Many cognitive benefits of using external representations in learning– understanding– reasoning – problem-solving– reflecting
• But can’t just assume a picture is worth a 1000 words or an animation a 1000 pictures or a mixed reality is better than a physical reality…
• Understand the properties of each and apply principles of design