Imitation Mechanisms of Social Resonance for …asadeghi/pub/col10a...Cognitive Interaction Technology Center of Excellence Sociable gents Outline • Motivation • Cognitive Model

Cognitive Interaction Technology Center of Excellence

gentsSociable

Imitation Mechanisms of Social Resonance for Embodied Agents

Amir SadeghipourSociable Agents Group

1


gentsSociable

Outline

• Motivation

• Cognitive Model

• Results

• Outlook

• Conclusion

2


gentsSociable

Outline

• Motivation

• Cognitive Model

• Results

• Outlook

• Conclusion

3


gentsSociable

Motivation

4

Communication& Coordination Verbal

&Non-verbal:

Gestures

Perception - Action Link

(Motor Resonance)


gentsSociable

• Focus: Human nonverbal social interaction• Capabilities:

• Perception• Representation• Generation

• Cognitive facilities:• Fast & real-time• Concurrent processing• Different levels of abstraction: from form to meaning

• Generation, perception & recognition

• Perception-action integration• Imitation learning

• Behavior coordination5

MotivationRequirements


gentsSociable

Outline

• Motivation

• Cognitive Model

• Results

• Outlook

• Conclusion

6


gentsSociable

Cognitive Model

7

Motor Knowledge

Observation

Inverse Models

Forward Models

Recognition Learning


gentsSociable

8

Observation

Inverse Models

Forward Models


Cognitive ModelMotor Knowledge

Motor Knowledge


gentsSociable

9

Motor Knowledge

Observation

Inverse Models

Forward Models


Motor Commands

Motor Programs

Motor Schemas


Motor Programs

Motor Commands


gentsSociable

Motor Knowledge

Motor Commands(Right hand)

Motor Programs(Right hand)

Motor Schemas

MC1 MC2 MC3 MC4

MP1 MP2

Waving

MC1M

C4

MC2

MC3

MP1 MP2

Waving



gentsSociable

11

Motor Knowledge

Observation

Inverse Models

Forward Models


Motor Commands

Motor Programs

Motor Schemas

Cognitive ModelGeneration

Motor Programs

Motor Commands


gentsSociable

12

Motor Knowledge

Observation

Inverse Models

Forward Models


Motor Commands

Motor Programs

Motor Schemas

Cognitive ModelPerception

Motor Programs

Motor Commands


gentsSociable

13

Observation

Inverse Models

Forward Models


Cognitive ModelRecognition process: Forward Models

Motor Knowledge

Motor Commands

Motor Programs

Motor Schemas

Motor Programs

Motor Commands


gentsSociable

14

Observation

Inverse Models

Forward Models


Cognitive ModelForward Models

MS-FM

MP-FM

MC-FM

Motor Knowledge

Motor Commands

Motor Programs

Motor Schemas

Motor Programs

Motor Commands


gentsSociable

• Perception challenges:• Bottom-up perception process• Top-down perception process

• Bayes′ theorem: P(A|B) = α P(B|A) P(A)

15


Forward Models

MS-FM

MP-FM

MC-FM

Observation

likelihood a priori



gentsSociable

16


Forward Models

MS-FM

MP-FM

MC-FM

Observation

= Pt-1( hmc | o )

Pt( hmc | o ) = α Pt(o | hmc) Pt(hmc)


gentsSociable

17


Forward Models

MS-FM

MP-FM

MC-FM

Observation


Pt( hmc | o ) = α Pt(o | hmc) Pt(hmc)

Pt( hmp | o, Hmc ) = α Pt(o, Hmc | hmc) Pt(hmc)

Pt( hms | o, Hmc, Hmp ) = α Pt(o, Hmc, Hmp | hmc) Pt(hms)


gentsSociable


Pt( hmp | Hms ) = α Pt(Hms | hmp) Pt(hmp)

Pt( hmc | Hmp ) = α Pt(Hmp | hmc) Pt(hmc)

18


Forward Models

MS-FM

MP-FM

MC-FM

Observation


gentsSociable

• Top-DownPt( hmp | Hms ) = α Pt(Hms | hmp) Pt(hmp)Pt( hmc | Hmp ) = α Pt(Hmp | hmc) Pt(hmc)Pt( hmc | o ) = α Pt(o | hmc) Pt(hmc)

• Bottom-UpPt( hmp | o, Hmc ) = α Pt(o, Hmc | hmc) Pt(hmc)Pt( hms | o, Hmc, Hmp ) = α Pt(o, Hmc, Hmp | hmc) Pt(hms)

• Prior probability at t = Posterior probability at t-1

19

Cognitive ModelBehavior Coordination

Forward Models

MS-FM

MP-FM

MC-FM

Observation


gentsSociable

• t = 0 ⇒ Initialization from „motor activation“

• Consequently, the agent(a) tends to perform MC, MP & MS that have been perceived last (P➟A)(b) and its sensitivity is primed by self performed MC, MP & MS (A➟P)

20

Cognitive ModelBehavior Coordination


gentsSociable

Outline

• Motivation

• Cognitive Model

• Results

• Outlook

• Conclusion

21

• 3D time-of-flight camera: SwissRanger SR4000• Marker-free tracking software: IISU 2.0• Virtual Agent: Vince


gentsSociable

ResultsFramework

22


gentsSociable

Results

23


gentsSociable

Outline

• Motivation

• Cognitive Model

• Results

• Outlook

• Conclusion

24


gentsSociable

Outlook

• Main challenges:• Beyond motor schema towards meaning and social intention• Considering finger configuration• Creating motor schema automatically

25


gentsSociable

OutlookBeyond Motor Schema


26

Motor Knowledge

Observation

IM

FM

Motor Commands

Motor Programs

Motor Schemas

Motor Programs

Motor Commands

Social Intention (Semantics)

Language(labels)


gentsSociable

OutlookFinger Configuration


27

Motor Knowledge

Motor Commands

Motor Programs

Motor Schemas

Motor Programs

Motor Commands

Motor ProgramsMotor Programs

Motor CommandsMotor Commands


gentsSociable

OutlookAutomatic Motor Schema Generation


28

considering variant and invariant features:• Different performances• Left and right hand• Velocity• Position• Repetition• Size• Rotation


gentsSociable

Outline

• Motivation

• Cognitive Model

• Results

• Outlook

• Conclusion

29


gentsSociable

• Focus: Human nonverbal social interaction• Capabilities:

• Perception• Representation• Generation

• Cognitive facilities:• Fast & real-time• Concurrent processing• Different levels of abstraction: from form to meaning

• Generation, perception & recognition

• Perception-action integration• Imitation learning

• Behavior coordination30

Conclusion

Motor Knowledge

Observation

IM

FM

Motor Knowledge


gentsSociable

Conclusion

31

Thanks!

Web: www.techfak.uni-bielefeld.de/ags/soa/research/gesture-imitation/

http://www.techfak.uni-bielefeld.de/ags/soa/research/gesture-imitation/

http://www.techfak.uni-bielefeld.de/ags/soa/research/gesture-imitation/


gentsSociable

DepotForward Models: Bottom-Up Process

32

• MC:

• MP:

• MS:

Imitation Mechanisms of Social Resonance for …asadeghi/pub/col10a...Cognitive Interaction Technology Center of Excellence Sociable gents Outline • Motivation • Cognitive Model

Documents