Direct Perception Action Coupling

Designing Technology that’s Second Nature

The Direct Perception-Action Coupling (DPAC) Approach

Moin Rahman & Ganesh BalakrishnanDesign-Integration | Motorola

Technology, Second Nature

When I See Three Oranges, I JuggleWhen I See Two Towers, I Walk

-PHILIPPE PETIT, 1974Man on Wire

• Gibson’s Direct Perception• Neo-Gibsonian Approach (DPAC)• Human-Machine Interfaces (A Retrospective)• Designing “technology that’s second nature.”

Topics

Gibson’s Direct Perception

James Gibson

Sensation vs. Perception

Stimulus: something that incites or excites the senses.

Sensation is passive: occurs when an organism’s senses are stimulated by energy (light, sound energy, etc.). - Detects the presence/absence of stimulus.

Perception is active: it occurs when the organism interrogates an object or stimulus with its senses.- Apprehends the meaning contained within the stimulus.

Indirect Perception (classical)Matching the event or stimulus with what has been stored in memory (comparison): Meaning is Deduced.

Direct Perception (Gibsonian) The richness of the stimulation – or higher order pattern or property – is sufficient with which meaning is Derived.

Indirect vs. Direct Perception

Indirect vs. Direct Perception

Indirect Perception

Direct Perception

Two Key Concepts of DP

1) Invariance

2) Affordance

Invariance

• An invariant is defined as a constant pattern, usually amid change in other variables.– Transformational Invariance– Structural Invariance

Transformational Invariance

Transformational invariance are styles of change that remain constant.

Structural Invariance

Structural Invariance

Structural invariants are patterns that remained constant when something else changed.

Structural Invariance (violation)

Invariance?

• Invariance are higher-order patterns that are recognized as “fixed” or “constants” by the perceptual system.

Direct Perception Invariance

Invariance in Artifacts

What is the higher order pattern? - about knee high- deep enough- flat surface- sturdy

Invariance in Artifacts

Invariance in Artifacts

Invariance in Artifacts

Invariance in Artifacts

essential collateral discretionary

linguistic

graphic

empathic

Courtesy: Del Coates ()

Ergonomically relevant Aesthetically relevant

Direct Perception

Direct Perception (information pickup):

Meaning inherent in an organism-environment system can be picked up directly without any mental calculations

4

Direct Perception (examples)

“Each thing says what it is…a fruit says ‘Eat me’; water says ‘Drink me’; thunder says ‘Fear me’; and woman says ‘Love me’- From the Principles of Gestalt Psychology (Kofka, 1935).

4

Direct Perception (action possibilities):Potential for action, without significant intermediate stages involving memory or inferences

Direct Perception

Direct perception reveals Affordance

5

An organism directly perceives an object in terms of the action it may afford

Affordance (examples)

5

Door Handles

Pull Affordance Push Affordance

Affordance (examples)

5

Door Handles

Sequential Affordance

Coke Bottle & the Bushmen

More on Affordances

5

Types of Affordances

I

IVIII

II

(negative affordance)

More on Affordances

5

Perceptible Affordance: ability to ascend or descend on stairs

More on Affordances

5

Hidden Affordance: Releasing a foot-operated emergency brake

More on Affordances

5

False Affordance: The [pull] handles on the second set of doors – however, the second set of doors don’t swing open but have to be pushed.

More on Affordances

5

Correct Rejection: the affordance of avoidance

When I See Three Oranges, I JuggleWhen I See Two Towers, I Walk

- PHILIPPE PETIT, 1974

More on Affordances

5

[correct rejection]: Negative Affordance

Affordance: the use value of an object with reference to the intrinsic physical features of an organism

Direct Perception-Action Coupling (DPAC)

Effectivities: The potential for purposive behaviors.

Direct Perception-Action Coupling (Neo-Gibsonian)

Walking on wire is determined by an animals effectivities.

Semantic vs. Direct Approach

5

S D

Function Affordance

Non-Objects Objects

S D

Affordances (action & function)

Action Affordance

Hand Operated Emergency Brake

Function Affordance

Aircraft Yoke’s A-F Affordances

Roll plane CW or CCW

Turn: CW or CCW

Nose up/downPull/Push

FunctionAction (on Yoke)

Scissors’ A-F Affordances

Action affords and nudges user towards functional affordance

Perceptual Information:1) visual (jaw

movement)2) auditory (sound of

cutting) and 3) haptic (experience of

cutting movement and force)

Faucets: A-F affordances

SW Radio’s A-F affordances

a b

Neo-Gibsonian ApproachDirect Perception Action Coupling (DPAC)

Sensorimotor pattern

DirectPerception

Environment

Object/Affordance/GoalSituated

Cognition

Perception-Action Coupling (Neo-Gibsonian)

Affordances write perception in the language of action.

“I kick the ball”

Perception-Action Coupling (Neo-Gibsonian)

Perception & motor centers are directly coupled (evidence from brain-imaging)

Just perceiving an affordance was sufficient to trigger an activation in the motor regions in the brain.

And simplify verbalizing an action phrase (“I kick the ball”) excited the motor regions responsible for leg movements.

Sensorimotor pattern

DirectPerception

Environment

Object/Affordance/Goal

Situated

Cognition

Perception-Action Coupling (Neo-Gibsonian)

Effort-driven pleasure

Sensorimotor pattern

DirectPerceptio

n

Environment

Object/Affordance/Goal

Situated

Cognition

“I kick the ball”

Direct Perception-Action Coupling (DPAC)

Effort-driven Rewards

Positive emotionsIncreased perception of controlVariety of physical movements coupled with

thought processes are rewarding

• Embodied CognitionThe body instructs the mind

It is with our bodies that we prehend the world- Merleau-Ponty

Direct Perception-Action Coupling (DPAC)

Sensorimotor pattern

DirectPerception

Environment

Object/Affordance/GoalSituated

Cognition

Direct Perception-Action Coupling (DPAC)

Cognition emerges in the interaction of an agent with an environment as a result of senosrimotor activity.

Direct Perception-Action Coupling (DPAC)

A-not-B Error

A B

Development of Embodied Cognition

Direct Perception-Action Coupling (DPAC)

Embodied Cognition Shapes thought and LanguageVerticality Schema: More = Up

“prices keep going up”“number of books published are rising”“the DOW JONES index fell 1000 points”“turn up the heat”

Direct Perception-Action Coupling (DPAC)

Embodied Cognition Shapes thought and Language

Embodiment and physical experience is pervasive and shapes language and thought:

“…face of the mountain”“…mouth of the river”

“…his argument was airtight”“…his spirits soared”

ventral

dorsal

cranial

caudal

Direct Perception-Action Coupling (DPAC)

Our embodiment drives our logic and thought

ventral

dorsal

cranial

caudal

Child: “Pour the salt into the shaker”(flowing of material)

Adult: “Fill the shaker with salt”(rise in level)

Direct Perception-Action Coupling (DPAC)

Semantic vs. Direct Approach

5

Choose the direct approach!

Semantic vs. Direct Approach: Challenges

5

Semantic approach challenges the user (consider language):1. Symbolic code (a,b,c,d,..)2. Phonetic code (pu, ba, ga…)3. Semantic code (“push,” “pull”)

Challenge of the Metaphor: Using of patterns of experience from one domain to structure the other. (Model, Control, View)

Semantic vs. Direct Approach: Challenges

5

Semantic vs. Direct Approach: Challenges

5

Direct approach challenges the designer

(examples)

1) Affordance2) Feedforward (how do we make the form

“broadcast” its function to its users?)

Semantic vs. Direct Approach (example)

5

Human-Machine Interfacesa retrospective

1. Machine-Cowboy Interface2. Analog-Professional Interface3. Digital-Hacker Interface

Three Generations of Interfaces

Model T

Machine-Cowboy Interface

1. Turn on the fuel line, a petcock on the right floor of the front compartment, allowing gasoline to gravity-flow from the ten-gallon tank mounted under the front seat to the carburetor.

2. Engage the hand brake by pulling the lever floor-mounted on the left side of the driver's seat to the rear, an arc of about 45 degrees, which also puts the planetary gears in neutral.

3. Turn on the ignition key on the dashboard. 4. Prime the carburetor to a rich startup mixture by pulling out the choke, a knob on the dashboard. 5. Move the throttle lever on the right side of the steering column down about four or five notches. 6. Raise the spark lever on the left of the column to the top to retard the spark. 7. With your right heel, press the electric starter button in the center of the floorboard close to the seat

base. 8. With the engine running smoothly, advance the spark by pulling it down two or three notches. 9. Push the brake lever halfway forward to prepare to go either forward or reverse. 10. To go forward, push the brake lever all the way toward the front of the car while pressing the left

pedal all the way in, engaging low, and advance the throttle another few notches. 11. With the car moving forward at least 10 mph in low, release the left pedal to shift into high. Adjust

throttle as required for desired road speed. 12. To back up, instead of the left pedal, push the center pedal in, engaging reverse. 13. For highway travel, which for a T means 35-40 mph - definitely not expressway traffic - advance

(pull down) both spark and throttle.

Steps Required to Drive Model T

Analog Professional Interface

Evolution of the Interfaces

Evolution of the Interfaces

Tyranny of the Push Buttons

Tyranny of the Push Buttons

Overthrowing the Tyranny

Overthrowing the Tyranny

Embodied Interactions

Tyranny of the Levers

Old Design: Steering Wheel + 15 Levers

New Design: Two Joysticks

Tyranny of the Levers

Joysticks Simplify Grader Operation

Left Joystick:Side-to-side = steering Twist = articulation Right yellow button = auto articulation return to center Top black buttons = wheel lean Trigger switch = transmission direction Top yellow buttons = gear selection Fore/Aft = left blade lift Detent = left blade float

Right Joystick:Fore/aft = right blade lift Detent = right blade float Side-to-side = blade shift Twist = circle turn Hat switch fore/aft = blade tip Hat switch left/right = drawbar shift Differential Lock/Unlock Electronic Throttle Resume/Decrement

Overthrowing the Tyranny of the Levers

Overthrowing the Tyranny of the Levers

Embodied Interactions

When an interface has a bank of similar controls, whether they be push buttons, levers, or knobs, the user is forced to use his semantic routes rather than the direct perception route.

Tyranny of Similarity

Designing Technology that’s Second Nature

“t2n”

Semantic vs. Direct Approach

5

Choose the direct approach!

Semantic vs. Direct Approach (example)

5

Affordances (action vs. function)

Action Affordance

Hand Operated Emergency Brake

Function Affordance

Perception-Action Coupling (Neo-Gibsonian)

Perception & motor centers are directly coupled (evidence from brain-imaging)

Sensorimotor pattern

DirectPerceptio

n

Environment

Object/Affordance/Goal

Situated

Cognition

“I kick the ball”

Overthrowing the Tyranny

Embodied Interactions

Direct Perception (examples)

“Each thing says what it is…a fruit says ‘Eat me’; water says ‘Drink me’; thunder says ‘Fear me’; and woman says ‘Love me’- From the Principles of Gestalt Psychology (Kofka, 1935).

4

Designing Technology that’s Second Nature

1. Direct Perception NOT SemanticInvarianceAffordance

2. Meaningfully couple Perception with Action

Action AffordanceFunctional Affordance

3. Embodied Action NOT Abstract Cognition

Design any one or more of the following critical controls for a 2-way radio’s interface on a suitable form factor*.

1) Volume control (increase/decrease loudness)

2) Channel Control (connect me with a person or group of people)

3) Talk Control (open mic/channel for transmission)

DPAC Ideation Exercise (sketching)

*The form factor – on which these controls may be placed – could be shaped to provide supporting or secondary affordances.

V

ChT

Schematic diagram of “VChT”with arbitrary shapes

Points to consider from the DPAC approach:• Invariance (identifiable & operable in almost all orientations)• Action Affordance (desired action is unambiguously revealed)• Utilize functional affordance & feedforward (make the control

communicate its stated function w/o labels or icons)• Avoid monotony among controls (same shapes or movements)• Utilize sensorimotor patterns (motions) & haptics that may

meaningfully map with control attributes such as (increase/decrease loudness, talk/open mic, connect me to xyz)

• Think about how our bodily experience shapes thought and apply it in your design (“embodied control”)

DPAC Ideation Exercise

t2n: e.interaction shapes industrial design