Page 1
www.ima-zlw-ifu.rwth-aachen.de
The Digitale Transformation of Human Machine Interaction
Human-Technology-Interaction 4.0 and the future of interaction
between people and technology
5. Augsburger
Technologietransfer-Kongress
Univ.-Prof. Dr.-Ing. em. Klaus Henning
IMA/ZLW & IfU
Faculty of Mechanical Engineering
RWTH Aachen University
Page 2
2
14.04.2016
Klaus Henning
Outline
I. The emerging world of interaction
� Breakthroughs
� … and a networked world
� A short history…
� … and the consequences
II. The intelligence of systems of systems
� Rediscover the Cybernetic approach
� The paradigm shift in artificial Intelligence: from Top-Down to Bottom-Up
� Approaches : multi agent systems in real-world applications
III. The new dimension of human machine interaction
� The role of the digital shadows
� It’s all about cognitive computing …
� … Addressing problems of “human-like” complexity
� … Copying human thought processes
� … Intuitive intelligent interaction with humans…
IV. Ready to go to 4.0?
Page 3
3
14.04.2016
Klaus Henning
� Almost all people of this world have a mobile (7 Billions).
� Smartphones are already in charge of 30% of this world as the
first generation of „individual agents“. In 10 year the figure of
smart phones being in use will increase to the figure of people,
living on this world.
Just one line of development
From the mobile to the car robot
From this viewpoint just think about:
� In 30 years about 1 Billion cars will drive fully automatic –
they will behave as intelligent robots.
� In 30 years all of us will have „intelligent“ cloths.
� In 30 years iron of buildings has to be intelligent by law.
� ….
Page 4
4
14.04.2016
Klaus Henning
4.0 will be everywhere
The digital transformation of all areas of live is just in the beginning
� The digital transformation is a global und local transformation of all areas of
live. It is unavoidable. It is a huge chance to recreate living, working and
learning in a worldwide context.
� These evolutions and revolutions are historically – if at all – comparable with
the introduction of mass printing facilities 500 years ago.
� The Internet of Things will invade into all devices of daily live.
� In the long run most of all reasonable devices technologies of this world will
be intelligent and perform a live long learning process by themselves and with
humans.
� 4.0 – Cyberphysical Systems - is the roadmap:
All areas will become „smart …“ (data, factory, logistics, home, grid,…..)
� The new 4.0 products, services and process will be the future strength of our
economy
Page 5
5
14.04.2016
Klaus Henning
4.0 will be everywhere
The digital transformation of all areas of live is just in the beginning
� The smart home will be a interactive multimedia platform –
the fridge will speak, all energy consumptions will be
centrally controlled.
� The heating equipment will be intelligent (i.e. System Geniax
from WILO).
� Medical online control will be possible in the private home.
� Intelligent window glasses will absorb energy and distribute it
in a controlled way.
� The auto bumper will be intelligent and adapt it`s stiffness to
the environment.
� There will be an body embedded back up cardiovascular
pump in case of an cardiac arrest.
� The embedded computer system in my jacket will become
one of my personal agent, giving me advice and warnings.
Some Examples
Page 6
6
14.04.2016
Klaus Henning
Human-Technology-Interaction 4.0
A new kind of technological intelligence
“Everything that we do now is tactile (in terms of interacting with our
environment), however, when we communicate with intelligent beings there is
very little tactile interaction. As our world becomes more intelligent, it will
require a different interactive approach.”
Bryan Brown, Human-Machine Technologies Inc. (http://www.hmtinc.org)
From a separated and interfaced interaction to an ubiquitous and inconspicuous
interaction between technology and humans.
Page 7
7
14.04.2016
Klaus Henning
… leading to the 4th (industrial) (r)evolution...
Breakthroughs - A new era of artificial intelligence
Communication technologybandwidth and computational power
Embedded systemsminiaturization
Semantic technologiesinformation integration
Watson
2011
Google Car
2012
� Systems of “human-like” complexity
Page 8
8
14.04.2016
Klaus Henning
Communication technologybandwidth and computational power
Embedded systemsminiaturization
Semantic technologiesinformation integration
… leading to the 4th (industrial) (r)evolution...
Breakthroughs - Everybody and everything is networked
Team
Robotics
Swarm
Robotics
Smart
Grid
Car2Infra-
structure
Smart
Factory
Page 9
9
14.04.2016
Klaus Henning
The upcoming world
“Information Revolution”
Everybody and everything is networked. - Big Data & Cyber-Physical Systems
Power revolutionCentralized electric power
infrastructure; mass production
by division of labor
1st industrial revolutionMechanical production
systematically using the
power of water and steam
today
Digital revolutionDigital computing and
communication technology,
enhancing systems’ intelligence
Information revolutionEverybody and everything is
networked – networked
information as a “huge brain”
around 1750 around 1900 around 1970
Weidmüller, Vission 2020 - Industrial Revolution 4.0
Intelligently networked, self-controlling manufacturing systems)
“Internet of Things & Services, M2M or Cyber Physical Systems are much more
than just buzzwords for the outlook of connecting 50 billions devices by 2015.”Dr. Stefan Ferber, Bosch (2011)
Vision of Wireless Next Generation System (WiNGS) Lab at the
University of Texas at San Antonio, Dr. Kelley
„local“
to „global“
„local“
to „global“
Page 10
10
14.04.2016
Klaus Henning
… towards a networked world
Not restricted to industry: cyber physical systems in all areas
Back to: The earth converted into a huge “brain”… (Tesla 1926)
Integrating complex information from multiple heterogeneous sources opens multiple possibilities of optimization:
e.g. energy consumption, security services, rescue services as well as increasing the quality of life
… and more
Building
automation
Smart grid
Room
automation
Smart
environment
Smart
metering
Page 11
11
14.04.2016
Klaus Henning
Communication technologybandwidth and computational power
Embedded systemsminiaturization
Semantic technologiesinformation integration
… towards a world of systems of systems
And how do these systems work?
Power revolutionCentralized electric power
infrastructure; mass production
by division of labor
1st industrial revolutionMechanical production
systematically using the
power of water and steam
today
Digital revolutionDigital computing and
communication technology,
enhancing systems’ intelligence
Information revolutionEverybody and everything is
networked – networked
information as a “huge brain”
around 1750 around 1900 around 1970
Towards intelligent and (partly-)
autonomous systems AND systems of systems
?? Steering -
Controlling ??
Page 12
12
14.04.2016
Klaus Henning
Outline
I. The emerging world of interaction
� Breakthroughs
� … and a networked world
� A short history…
� … and the consequences
II. The intelligence of systems of systems
� Rediscover the Cybernetic approach
� The paradigm shift in artificial Intelligence: from Top-Down to Bottom-Up
� Approaches : multi agent systems in real-world applications
III. The new dimension of human machine interaction
� The role of the digital shadows
� It’s all about cognitive computing …
� … Addressing problems of “human-like” complexity
� … Copying human thought processes
� … Intuitive intelligent interaction with humans…
IV. Ready to go to 4.0?
Page 13
13
14.04.2016
Klaus Henning
�
� Term: „governance“, to navigate
� Born around 1940
� 1948: “Cybernetics or control and
communication in the Animal and in the
machine” (Norbert Wiener)
� until 1953: Macy-Conferences
FeedbackloopCircular explanations for
systems behavior, self-
regulation
(Forrester, Ashby)
FeedbackloopCircular explanations for
systems behavior, self-
regulation
(Forrester, Ashby)
AutopoiesisSystem capacity to maintain
and stabilize itself
(Maturana, Varela)
AutopoiesisSystem capacity to maintain
and stabilize itself
(Maturana, Varela)
DecentralizationDecentralized navigation,
bottom up processes
(Stafford Beer)
DecentralizationDecentralized navigation,
bottom up processes
(Stafford Beer)
The Basis of thinking in system with high complexity und dynamics (Dynaxity)
The central elements of Cybernetics
EmergenceSpontaneous new properties,
swarm behavior
(Wolfram, Gell-Mann)
EmergenceSpontaneous new properties,
swarm behavior
(Wolfram, Gell-Mann)
Complex Systems
Multi-component systems
in complex interactions
(Stafford Beer)
Complex Systems
Multi-component systems
in complex interactions
(Stafford Beer)
Page 14
14
14.04.2016
Klaus Henning
„symmetric“biological-centered
The rise of agent systems …
A paradigm shift in the definition of intelligence
‘only biological brains
are intelligent’
‘any behaviorally equivalent functional
system is intelligent’
Triple jump of
intelligent agents:
1. Sensors
2. Cognition
3. Actuators
Biological
approach!
Liberal
approach!
Page 15
15
14.04.2016
Klaus Henning
The paradigm shift in artificial Intelligence
From Top-Down to Buttom-Up
� Two competing movements?
Top-Down / symbolic AI Bottom-Up / subsymbolic AI
GOFAIGood old fashioned
Artificial Intelligence;
based on high-level
"symbolic“ knowledge
representations
Connectivisminteraction as basis
of intelligence
[Szt
ipa
no
vit
s, V
an
de
rbilt,
19
97
]
[Lin
, 2
01
0]
Knowledge
storage/
knowledge
retrieval
Knowledge
on demand /
knowledge
aquisition
Top-Down / symbolic AI
GOFAIGood old fashioned
Artificial Intelligence;
based on high-level
"symbolic“ knowledge
representations
[Szt
ipa
no
vit
s, V
an
de
rbilt,
19
97
]
Knowledge
storage/
knowledge
retrieval
[Szt
ipa
no
vit
s, V
an
de
rbilt,
19
97
]
Page 16
16
14.04.2016
Klaus Henning
From the history of autonomous vehicles (already old fashioned)
2009: Truck robot platoons – distributed intelligence
� The KONVOI project (several institutes from RWTH & industry partners)
�
� 2005-2009
� automated / partly autonomous
transportation e.g. by electronically
coupling trucks to convoys
� several successful tests with trucks:
Chauffeur, KONVOI, SARTRE (EU), Energy-
ITS (Japan), …
�
� expected improvements:
beyond safety, reduction of fuel
consumption and gained road space
!
� Adv. driver assistance system for trucks
� short distances between vehicles of
approx. 10m at a velocity of 80 km/h
� Energy-ITS: 4m ! (2013)
� KONVOI:
� Car2infrastrcuture components!
� Model of multi agent systems
Connectivity…
Page 17
17
14.04.2016
Klaus Henning
� Organization forms on demand – individualized by client - initialized by product
!Product agitates as “super-agent”:
� Plans production and transportation steps
� Requests service from agents
� Negotiates with other products for agent-resources
�
� Heterogenous player modeled as multi agent concept
� Models from biology and social sciences
� Basis on Autopoiesis & embodiment theory
transport
unit
production
unit
virtual service
providerfab
ric
atio
n
ou
tsid
e w
orld
© D
an
iel E
we
rt 20
13
Trucks or shoes – state of the art is ….
Horizontal coupling (manufacturing/logistics) – to lot size 1
�
� Konvoi 2005-2009,
RWTH with partners
� (partly) autonomous
driving via convoys
Page 18
18
14.04.2016
Klaus Henning
Outline
I. The emerging world of interaction
� Breakthroughs
� … and a networked world
� A short history…
� … and the consequences
II. The intelligence of systems of systems
� Rediscover the Cybernetic approach
� The paradigm shift in artificial Intelligence: from Top-Down to Bottom-Up
� Approaches : multi agent systems in real-world applications
III. The new dimension of human machine interaction
� The role of the digital shadows
� It’s all about cognitive computing …
� … Addressing problems of “human-like” complexity
� … Copying human thought processes
� … Intuitive intelligent interaction with humans…
IV. Ready to go to 4.0?
Page 19
19
14.04.2016
Klaus Henning
Human-Technology-Interaction 4.0
A new kind of Human-Technology-Interaction
The digital shadow (or skin) will be an dominant part of the human`s and the
technical devise`s identity. The dimensions of interaction will be:
communication between the digital shadows of humans and/or technical devices,
machine to machine communication,
human to human communication,
human to machine communication,
communication of humans / machines with its digital shadows (skins).
The ubiquitous and inconspicuous interaction between the digital shadows of
technology and of humans will dominate all aspects of communication
Page 20
20
14.04.2016
Klaus Henning
Three steps
It’s all about “Cognitive Computing”
Page 21
21
14.04.2016
Klaus Henning
Let’s ask Google
Definitions around “Cognitive Computing” (I)
“Cognitive computing (CC) makes a new class of problems
computable. It addresses complex situations that are
characterized by ambiguity and uncertainty; in other words it
handles human kinds of problems. …To do this, systems often
need to weigh conflicting evidence and suggest an answer that
is “best” rather than “right”.
Cognitive computing systems make context computable.”
Page 22
22
14.04.2016
Klaus Henning
Dealing with complexity and dynamics
Human machine interaction and cooperative robotics
�Robots are no longer locked in work-cells but cooperate
with each other and/or with humans
human-machine-machine
interaction in the X-Cell
machine-machine cooperation Robotics entering
the scene as:
Cognitive computing is
about „solving real
problems“. Real
problems are usually
part of our real,
physical world…
The enhancement of
AI is strongly
connected to the
progress in robotics,
coupled by the
embodiment theory.
Page 23
23
14.04.2016
Klaus Henning
Advantage of decentralized control structures
Intralogistics goes mobile: The Festo Logistics League
Competitions robocup:
2012: 0 points in World Cup
2013: 4th in World Cup
2014: Winner of the GermanOpen
2014: Winner of the World Cup
2015: Winner of the World Cup
�
Critical factors for success:
� Totally decentralized
� No ”hard coded components“
� Strong cooperation
� Re-planning during tasks
� Mobile transportation robots from flexible routing
!
Competencies:
� localization & navigation
� computer vision
� adaptive planning
� multi agent strategies
� sensory & hardware
Page 24
24
14.04.2016
Klaus Henning
Let’s ask Google
Definitions around “Cognitive Computing” (II)
“Cognitive computing (CC) makes a new class of problems
computable. It addresses complex situations that are
characterized by ambiguity and uncertainty; in other words it
handles human kinds of problems. …To do this, systems often
need to weigh conflicting evidence and suggest an answer that
is “best” rather than “right”.
Cognitive computing systems make context computable.”
“Cognitive computing is the simulation
of human thought processes in a
computerized model…. involves self-
learning systems that use data mining,
pattern recognition and natural
language processing to mimic the way
the human brain works.”
Page 25
25
14.04.2016
Klaus Henning
formerly: „neural
networks“
What has to be expected?
From organic … to cognitive computing
!Cognitive computing: the simulation of
biological thought processes !Organic computing: Following social systems and biological models
Ne
ura
ln
etw
ork
s
Deep learning (e.g. Watson)
�Relativization: It‘s not only about „human approaches“
but about approaches which have been developed along the evolution
Macro-scale
Automation
Division of labor
Micro-scale
Multi-Core
Ha
rd
wa
re
Service
oriented
Agent-
based
So
ftw
are
Genetic algorithmsDas Bild kann zurzeit nicht angezeigt werden.
Page 26
26
14.04.2016
Klaus Henning
Shadow Dexterous Hand
Central terms in the field of intelligent distributed systems
From embodiment … to humanoids
�
Zykov V., Mytilinaios E., Adams B., Lipson H. (2005) "Self-reproducing machines",
Nature Vol. 435 No. 7038, pp. 163-164
Bongard J., et al., Resilient Machines Through Continuous Self-Modeling, Science
314, 2006
Lipson H. (2005) "Evolutionary Design and Evolutionary Robotics", Biomimetics, CRC
Press (Bar Cohen, Ed.) pp. 129-155
Robonaut 2- NASA
The Bongard robot – learning through
embodiment [Bongard, 2006; Lipson, 2007]
Embodiment theory:
„intelligence needs a body“
The existence of a body (incl. sensors and actuators)
are basic prerequisites to build experience and
finally the development of intelligence.
�
Embodiment theory:
„different bodies = different intelligences“
… leading to humanoids / humanoid components
�
Asimo Honda
KIT, Dillmann,
SFB 588
Page 27
27
14.04.2016
Klaus Henning
Lend the robots a face
Into Service Robotics: The next step – the “Oscars”
� Transform mobile robotic experiences into the field of service robotics
!
1. Investigating “new” human machine
Interfaces and interaction schemes
� Simple, intuitive
� Schematic eyes following you
� “natural eyes behavior”: randomly
looking around, showing interest by
blinking, looking bored, …
!Performing service robot tasks
� Distribute brochures and serving drinks
� Path planning, room exploration, …
!
2. Investigating the “Uncanny Valley”: when
features look almost, but not exactly, like
natural beings, it causes a response of
revulsion among the observers (Mori 1970)
3. Investigating diversity specific reactions
(gender, age, culture) to artificial systems
and in particular robots
Page 28
28
14.04.2016
Klaus Henning
Let’s ask Google
Definitions around “Cognitive Computing” (III)
“Cognitive computing (CC) makes a new class of problems
computable. It addresses complex situations that are
characterized by ambiguity and uncertainty; in other words it
handles human kinds of problems. …To do this, systems often
need to weigh conflicting evidence and suggest an answer that
is “best” rather than “right”.
Cognitive computing systems make context computable.”
“Cognitive computing systems [are] a
category of technologies that uses
natural language processing and
machine learning to enable people
and machines to interact more
naturally […]. These systems will learn
and interact to provide expert
assistance to scientists, engineers,
lawyers, and other professionals in a
fraction of the time
it now takes.”
“Cognitive computing is the simulation
of human thought processes in a
computerized model…. involves self-
learning systems that use data mining,
pattern recognition and natural
language processing to mimic the way
the human brain works.”
Page 29
29
14.04.2016
Klaus Henning
New forms of human machine interactipn
About the role of emotion in human-machine-interaction
!However, it took a while before emotions were
considered important in computer science.
Plato (ca. 400 BC)“Human behavior flows from three
main sources: desire, emotion, and
knowledge.”
Rosalind Picard (since 1997; MIT)“Computers that will interact naturally
and intelligently with humans need
the ability to at least recognize and
express affect.”
Picard coined the term
“affectice computing”
�
KISMET - MIT (1990-2000; Cynthia Breazeal)
� Analysis and simulation of human-like
emotions
� Research on interaction between robots
and humans
� Part of the “organic development”
Page 30
30
14.04.2016
Klaus Henning
New forms of human machine interactipn
Automatic Emotion Recognition
!
1. Emotion form an important part of communication –
� not an „add-on“ on audio, …
2. Emotions are an intrinsic part of intelligence -
� not an „add-on“ on ratio, …
Project IMOTION at IMA/ZLW & IfU:
the emotional navigation system
Using emotions as basis for behavior adaption:• Shouting at it will make it change its behavior
• Praising stabilizes behavior
• Randoms enable global instead of local optima
Page 31
31
14.04.2016
Klaus Henning
New forms of human machine interaction
Automatic Emotion Recognition: Project IMOTION (1)
?Did you ever yell at your GPS,
e.g. when it told you the equivalent of “drive through that river ahead!”? ☺
Well, WE do constantly. And so the story went on like this:
! Motivation and goal: Transform the GPS into an intelligent co-driver, i.e. get it to adapt to your emotions!
� Solution: A machine-learning based system architecture. Did it work? Often! Was it fun? Hell, yeah! ☺
�
Ingredients in a nutshell
� Primary source of emotion: driver
speech
� Machine-learning algorithm:
Support Vector Machine (SVM)
� Training database: talk show
based corpus (lots of yelling)
� Test-bed: a driving simulator
� Test persons: old and young
Page 32
32
14.04.2016
Klaus Henning
New forms of human machine interaction
Automatic Emotion Recognition: Project IMOTION (2)
� Impressions, findings and lessons learned during the course of the project and the trail runs
�
Trial runs and setup
� 38 test persons, from young to old. Subjects had to
pass a simulated test track, under time pressure and
had to obey the instructions of the GPS
� The GPS was intentionally mean, i.e. it gave incorrect
orders (subjects had to drive 5 times through a
roundabout)
� GPS adapted itself, e.g. it apologized when yelled at
and “updated” its map material or pacified the driver
�
Findings and lesson learned
� Speech based emotion detection worked well
� But: Some people where mute during driving (one
theoretical physicist didn't take at all, though you
could pick up his annoyance on the video recording!)
� Conclusion: We have to go multimodal in emotion
recognition, by combining audio, video with
physiological sensors (e.g. EEG, which was
researched during IMOTION)
Page 33
33
14.04.2016
Klaus Henning
Outline
I. The emerging world of interaction
� Breakthroughs
� … and a networked world
� A short history…
� … and the consequences
II. The intelligence of systems of systems
� Rediscover the Cybernetic approach
� The paradigm shift in artificial Intelligence: from Top-Down to Bottom-Up
� Approaches : multi agent systems in real-world applications
III. The new dimension of human machine interaction
� The role of the digital shadows
� It’s all about cognitive computing …
� … Addressing problems of “human-like” complexity
� … Copying human thought processes
� … Intuitive intelligent interaction with humans…
IV. Ready to go to 4.0?
Page 34
34
14.04.2016
Klaus Henning
Summary
… in four steps!
4.0: The Revolution of a
distributed artificial
intelligence
4.0 is an era of highly
interdisciplinary
research
Rediscovery of
Cybernetics
4th Industrial
Revolution
The rise of cognitive
computing
Page 35
35
14.04.2016
Klaus Henning
Human-Technology-Interaction 4.0
Summary
Nowadays, technology is often passive. Future HTI will see a much more active
role of everyday technology: i.e. in administration, medicine or mobility.
Is it comfortable, …
Necessary, …
Or even scary?
Page 36
www.ima-zlw-ifu.rwth-aachen.de
Thank you!
Univ.-Prof. Dr.-Ing. em. Klaus Henning
Senior Advisor of Institute Cluster IMA/ZLW & IfU
phone: +49 171 860 7575
[email protected]
Co-authored by:
Univ.-Prof. Dr. rer. nat. Sabina Jeschke
Head of Institute Cluster IMA/ZLW & IfU
phone: +49 241-80-91110
[email protected]
Prof. Dr.-Ing. Tobias Meisen
Institute Cluster IMA/ZLW & IfU
[email protected]
Dipl.-Inform. Christian Kohlschein
Institute Cluster IMA/ZLW & IfU
[email protected]
aachen.de
Dipl.-Wirt.-Ing. Sebastian Reuter
Institute Cluster IMA/ZLW & IfU
[email protected]
Page 37
37
14.04.2016
Klaus Henning
Univ.-Prof. em. Dr.-Ing. Klaus Henning …
… studierte Elektrotechnik und Politische Wissenschaften, promovierte über
Mensch-Maschine-Systeme und habilitierte über Entropie in der Systemtheorie. Er hat über 40 Jahre Berufserfahrung.
Er war 25 Jahre lang Leiter des größten Institutsclusters für Kybernetik
(IMA/ZLW & IfU der RWTH Aachen University). Jeweils einige Jahre war er
Mitglied im Präsidium des VDI, Prorektor für Finanzen der RWTH
und Dekan der Fakultät für Maschinenwesen der RWTH.
Heute ist er Senior-Berater und Mitgesellschafter der
OSTO Systemberatung GmbH, ein Unternehmen der P3 group.
Er ist Mitglied des Vorstands des Instituts für Unternehmenskybernetik (IfU e.V) an der RWTH Aachen und als
Senior Advisor Mitglied des Board of Management des Institutsclusters IMA/ZLW & IfU an der RWTH Aachen.
Er ist Vorsitzender des Aufsichtsrats der Xenium A.G., München, Mitglied des Universitätsrats der Universität des
Saarlands und Mitglied des Zukunftskommission des Wirtschaftsrats der CDU.
Derzeit kommen die meisten von ihm betreuten Kunden – meist auf Vorstands- und Abteilungsleiterebene aus der IT-
Branche, universitären Krankenhäusern und der Zulieferindustrie des Maschinen- und Anlagenbau, der Automotive
Branche, der Luft- und Raumfahrtindustrie sowie der Logistik.
Er hat – zusammen mit zwei Kollegen – 2011 bis 2012 den Zukunftsdialog der Bundeskanzlerin „Wie wollen wir
leben?“ wissenschaftlich koordiniert.
Seine Erfahrungen hat er in einem Buch „Die Kunst der kleinen Lösung – wie Menschen und Unternehmen die
Komplexität meistern“ zusammengefasst, das 2014 im Murrmann- Verlag erschienen ist.