-
Machine Consciousness in CiceRobot, a Museum Guide Robot
Irene Macaluso and Antonio ChellaDipartimento di Ingegneria
Informatica, Università di Palermo
Viale delle Scienze, 90128, Palermo, Italy
Abstract
The paper discusses a model of robot perceptual aware-ness based
on a comparison process between the effec-tive and the expected
robot input sensory data generatedby a 3D robot/environment
simulator. The paper con-tributes to the machine consciousness
research field bytesting the added value of robot perceptual
awareness onan effective robot architecture implemented on an
oper-ating autonomous robot RWI B21 offering guided toursat the
Archaeological Museum of Agrigento, Italy.
IntroductionAn autonomous robot operating in real and
unstructured en-vironments interacts with a dynamic world populated
withobjects, people, and in general, other agents: people andagents
may change their position and identity during time,while objects
may be moved or dropped. In order to workproperly, the robot should
be able to pay attention to rele-vant entities in the environment,
to choose its own goals andmotivations, and to decide how to reach
them. We claim thatthe robot, in order to properly move and act in
complex anddynamic environment should have some form of
perceptualawareness of the surrounding environment.
Taking into account several results from neuroscience,psychology
and philosophy, summarized in the next Sect.,we hypothesize that at
the basis of the robot perceptualawareness there is a continuous
comparison process betweenthe expectation of the perceived scene
obtained by a projec-tion of the 3D reconstruction of the scene,
and the effectivescene coming from the sensory input. The paper
contributesto the machine consciousness research field (Chella
& Man-zotti 2007) by implementing and testing the proposed
robotperceptual awareness model on an effective robot architec-ture
implemented on an operating autonomous robot RWIB21 offering guided
tours at the Archaeological Museum ofAgrigento (Fig. 1).
Theoretical remarksAnalyzing the perceptual awareness from an
evolutionarypoint of view, (Humphrey 1992) makes a distinction
be-tween sensations and perceptions. Sensations are active re-
Copyright c© 2007, Association for the Advancement of
ArtificialIntelligence (www.aaai.org). All rights reserved.
Figure 1: CiceRobot at the Archaeological Museum of
Agri-gento.
sponses generated by the body in reaction to external stim-uli.
They refers to the subject, they are about what is hap-pening to
me. Perceptions are mental representations relatedto something
outside the subject. They are about what ishappening out there.
Sensations and perceptions are twoseparate channels; a possible
interaction between the twochannels is that the perception channel
may be recoded interms of sensations and compared with the
effective stimulifrom the outside, in order to catch and avoid
perceptual er-rors. This process is similar to the echoing back to
sourcestrategy for error detection and correction.
(Gärdenfors 2004b) discusses the role of simulators re-lated to
sensations and perceptions. He claims that sensa-tions are
immediate sensory impressions, while perceptionsare built by
simulators of the external world. A simulatorreceives as input the
sensations coming from the externalworld, it fills the gaps and it
may also add new informationin order to generate perceptions. The
perception of an objectis therefore more rich and expressive than
the correspondingsensation. In Gärdenfors terms, perceptions are
sensationsthat are reinforced with simulations.
The role of simulators in motor control has been exten-sively
analyzed from the neuroscience point of view, see(Wolpert, Doya,
& Kawato 2003) for a review. In this line,
90
-
(Grush 2004) proposes several cognitive architectures basedon
simulators (emulators in Grush terms). The basic archi-tecture is
made up by a feedback loop connecting the con-troller, the plant to
be controlled and a simulator of the plant.The loop is
pseudo-closed in the sense that the feedback sig-nal is not
directly generated by the plant, but by the simula-tor of the
plant, which parallels the plant and it receives asinput the
efferent copy of the control signal sent to the plant.
A more advanced architecture proposed by Grush and in-spired to
the work of (Gerdes & Happee 1994) takes into ac-count the
basic schema of the Kalman filter (Haykin 2001).In this case, the
residual correction generated by the com-parison between the
effective plant output and the simulatoroutput are sent again to
the simulator via the Kalman gain.In turns, the simulator sends its
inner variables as feedbackto the controller. According to
(Gärdenfors 2004a), the sim-ulator inner variables are more
expressive that rough plantoutputs and they may contain also
information not directlyperceived by the system, as the occurring
forces in the per-ceived scene, or the object-centred parameters,
or the vari-ables employed in causal reasoning.
(Grush 1995) also discusses the adoption of neural net-works to
learn the operations of the simulators, while (Oz-top, Wolpert,
& Kawato 2005) propose more sophisticatedlearning techniques of
simulators based on inferences of thetheory of mind of others.
The hypothesis that the content of perceptual awarenessis the
output of a comparator system is in line with the Be-havioural
Inhibition System (BIS) discussed by (Gray 1995)starting from
neuropsychological analysis. From a neuro-logical point of view,
(Llinas 2001) hypothesizes that theCNS is a reality-emulating
system and the role of sensoryinput is to characterize the
parameters of the emulation. Healso discusses (Llinas & Pare
1991) the role of this loop dur-ing dreaming activity.
An early implementation of a robot architecture based
onsimulators is due to (Mel 1986). He proposed a simulatedrobot
moving in an environment populated with simple 3Dobjects. The robot
is controlled by a neural network thatlearns the aspects of the
objects and their relationships withthe corresponding motor
commands. It becomes able to sim-ulate and to generate expectations
about the expected ob-ject views according to the motor commands;
i.e., the robotlearns to generate expectations of the external
environment.A successive system is MURPHY (Mel 1990) in which
aneural network controls a robot arm. The system is able toperform
off-line planning of the movements by means of alearned internal
simulator of the environment.
Other early implementation of robots operating with in-ternal
simulators of the external environment are: MetaToto(Stein 1991),
and the internalized plan architecture (Payton1990). In both
systems, a robot builds an inner model on theenvironment reactively
explored by simulated sensorimotoractions in order to generate
action plans.
An effective robot able to build an internal model of
theenvironment has been proposed by (Holland & Goodman2003).
The system is based on a neural network that con-trols a Khepera
minirobot and it is able to build a model ofenvironment and to
simulate perceptual activities in a sim-
Figure 2: The robot architecture.
plified environment. Following the same principles, (Hol-land,
Knight, & Newcombe 2007) describe the successiverobot CRONOS, a
complex anthropomimetic robot whoseoperations are controlled by
SIMNOS, a 3D simulator of therobot itself and its environment.
Robot architectureThe robot architecture proposed in this paper
is inspired bythe work of Grush previously presented and it is
based onan internal 3D simulator of the robot and the
environmentworld (Fig. 2). The Robot block is the robot itself and
itis equipped with motors and a video camera. It is modelledas a
block that receives in input the motor commands Mand it sends in
output the robot sensor data S, i.e., the sceneacquired by the
robot video camera.
The Controller block controls the actuators of the robotand it
sends the motor commands M to the robot. The robotmoves according
to M and its output is the 2D pixel matrixS corresponding to the
scene image acquired by the robotvideo camera.
At the same time, an efferent copy of the motor commandsis sent
to the 3D Robot/Environment simulator. The simu-lator is a 3D
reconstruction of the robot environment withthe robot itself. It is
an object-centred representation of theworld in the sense of (Marr
1982).
The simulator receives in input the controller motor com-mand M
and it simulates the corresponding motion of therobot in the 3D
simulated environment by generating a cloudof possible robot
positions {xm} according to a suitableprobability distribution that
takes into account the motorcommand, the noise, the faults of the
controllers, the slip-pery of the wheels, and so on. For each
possible positionxm, a 2D image S′m is generated as a projection of
the sim-ulated scene acquired by the robot in the hypothesized
posi-tion. Therefore, the output of the simulator is the set
{S′m}of expected 2D images. Both S and S′m are
viewer-centredrepresentations in Marr’s terms.
The acquired and the expected image scenes are thencompared by
the comparator block c and the resulting errorε is sent back to the
simulator to align the simulated robotwith the real robot (see
Sect. ). At the same time, the simu-lator send back all the
relevant 3D information P about therobot position and its
environment to the controller, in orderto adjust the motor
plans.
We claim that the perceptual awareness of the robot is
theprocess based on the acquisition of the sensory image S,
thegeneration of hypotheses {S′m} and their comparison via the
91
-
comparator block in order to build the vector P that containsall
the relevant 3D information of what the robot perceivesat a given
instant. In this sense, P is the full interpretationof robot
sensory data by means of the 3D simulator.
Planning by expectationsThe proposed framework for robot
perceptual awarenessmay be employed to allow the robot to imagine
its ownsequences of actions (Gärdenfors 2007). In this
perspec-tive, planning may be performed by taking advantage fromthe
representations in the 3D robot/environment simulator.Note that we
are not claiming that all kinds of planning mustbe performed within
a simulator, but the forms of planningthat are more directly
related to perceptual information cantake great advantage from
perception in the described frame-work.
As previously stated, P is the perception of a situation ofthe
world out there at time t. The simulator, by means of itssimulation
engine based on expectations (see below), is ableto generate
expectations of P at time t + 1, i.e., it is ableto simulate the
robot action related with motor command Mgenerated by the
controller and the relationship of the actionwith the external
world.
It should be noticed that in the described framework,
thepreconditions of an action can be simply verified by geo-metric
inspections of P at time t, while in the STRIPS-likeplanners (Fikes
& Nilsson 1971) the preconditions are veri-fied by means of
logical inferences on symbolic assertions.Also the effects of an
action are not described by adding ordeleting symbolic assertions,
as in STRIPS, but they can beeasily described by the situation
resulting from the expec-tations of the execution of the action
itself in the simulator,i.e., by considering the expected
perception P at time t + 1.
We take into account two main sources of expectations.On the one
side, expectations are generated on the basis ofthe structural
information stored in a symbolic KB which ispart of the simulator.
We call linguistic such expectations.As soon as a situation is
perceived which is the precondi-tion of a certain action, then the
symbolic description elicitthe expectation of the effect situation,
i.e., it generates theexpected perception P at time t + 1.
On the other side, expectations could also be generatedby a
purely Hebbian association between situations. Sup-pose that the
robot has learnt that when it sees somebodypointing on the right,
it must turn in that direction. The sys-tem learns to associate
these situations and to perform therelated action. We call
associative this kind of expectations.
In order to explain the planning by expectation mecha-nism, let
us suppose that the robot has perceived the currentsituation P0,
e.g., it is in a certain position of a room. Letus suppose that the
robot knows that its goal g is to be ina certain position of
another room with a certain orienta-tion. A set of expected
perceptions P1,P2, . . . of situationsis generated by means of the
interaction of both the linguisticand the associative modalities
described above. Each Pi inthis set can be recognized to be the
effect of some action re-lated with a motor command Mj in a set of
possible motorcommands M1,M2, . . . where each action (and the
corre-
Figure 3: The map of the Sala Giove of the Museum.
sponding motor command) in the set is compatible with
theperception P0 of the current situation.
The robot chooses a motor command Mj according tosome criteria;
e.g., it may be the action whose expected ef-fect has the minimum
Euclidean distance from the goal, or itmay be the action that
maximizes the utility value of the ex-pected effect. Once that the
action to be performed has beenchosen, the robot can imagine to
execute it by simulatingits effects in the 3D simulator then it may
update the situa-tion and restart the mechanism of generation of
expectationsuntil the plan is complete and ready to be
executed.
Linguistic expectations are the main source of delibera-tive
robot plans: the imagination of the effect of an action isdriven by
the description of the action in the simulator KB.This mechanism is
similar to the selection of actions in de-liberative forward
planners. Associative expectations are atthe basis of a more
reactive form of planning: in this lat-ter case, perceived
situations can immediately recall someexpected effect of an
action.
Both modalities contribute to the full plan that is imag-ined by
the robot when it simulates the plan by means of thesimulator. When
the robot terminates the generation of theplan and of its actions,
it can generate judgments about itsactions and, if necessary,
imagine alternative possibilities.
Robot at workThe presented framework has been implemented in
CiceR-obot, an autonomous robot RWI B21 equipped with sonar,laser
rangefinder and a video camera mounted on a pan tilt.The robot has
been employed as a museum tour guide op-erating at the
Archaeological Museum of Agrigento, Italyoffering guided tours in
the Sala Giove of the museum (Fig.3). A first session of
experimentations, based on a previousversion of the architecture,
has been carried out from Jan-uary to June 2005 and the results are
described in (Chella,Frixione, & Gaglio 2005) and in (Macaluso
et al. 2005).The second session, based on the architecture
described inthis paper, started in March and ended in July
2006.
The task of museum guide is considered a significant casestudy
(Burgard et al. 1999) because it concerns perception,self
perception, planning and human-robot interactions. Thetask is
therefore relevant as a test bed for robot perceptual
92
-
Figure 4: The object centered view from 3Drobot/environment
simulator.
Figure 5: The viewer centered image from the robot point
ofview.
awareness. It can be divided in many subtasks operating
inparallel, and at the same time at the best of the robot
capa-bilities.
Fig. 4 shows the object-centred view from the
3Drobot/environment simulator. As previously described, thetask of
the block is to generate the expectations of the inter-actions
between the robot and the environment at the basisof robot
perceptual awareness. It should be noted that therobot also
simulates itself and its relationships with its envi-ronment. Fig.
5 shows a 2D image generated from the 3Dsimulator from the robot
point of view.
In order to keep the simulator aligned with the
externalenvironment, the simulator engine is equipped with a
parti-cle filter algorithm (Thrun, Burgard, & Fox 2005). As
dis-cussed in the previous Sect., the simulator hypothesizes acloud
{xm} of expected possible positions of the robot. Foreach expected
position, the corresponding expected imagescene S′m is generated,
as in Fig. 6 (left). The comparatorthus generates the error measure
ε between each of the ex-pected images and the effective image
scene S as in Fig. 6(right). The error ε weights the expected
position under con-sideration; in subsequent steps, only the
winning expectedpositions that received the higher weights are
taken, whilethe other ones are dropped.
Fig. 7 (left) shows the initial distribution of expectedrobot
position and Fig. 7 (right) shows the small cluster ofwinning
positions. Now the simulator receives a new mo-tor command M
related with the chosen action and, startingfrom the winning
hypotheses, it generates a new set of hy-pothesized robot
positions.
Figure 6: The 2D image output of the robot video camera(left)
and the corresponding image generated by the simula-tor
(right).
Figure 7: The operation of the particle filter. The initial
dis-tribution of expected robot positions (left), and the cluster
ofwinning expected positions.
To compute the importance weight ε of each hypothesis,we
developed a measurement model that incorporates the3D
representation provided by the simulator. The imagesS′m, generated
by the simulator in correspondence to eachhypothesized robot
position, and the image S, acquired bythe robot camera, are both
processed to obtain the VE im-ages, in which vertical edges are
outlined. For each VEimage a vector s is computed:
si =∑
j
VEij (1)
Each vector s can be considered as a set of samples drawnfrom an
unknown distribution. To estimate such a proba-bility density
function, we adopted the Parzen-window for-mula:
pk(x) =1|s|
∑i
1h
δ(x − si
h) (2)
where the window function δ is the normal distribution.Then we
adopted the Kullback-Leibler distance:
d(p, ps) = −∫
p(x)lnps(x)p(x)
dx. (3)
as measure of the dissimilarity between the distribution
pcorresponding to the image S acquired by the robot cameraand each
of the distributions ps corresponding to the imagesS′m generated by
the simulator.
93
-
(a)
(b)
(c)
Figure 8: (a) Image acquired by robot camera. (b) VerticalEdge
image. (c) Distribution of the random variable repre-senting
vertical edges.
The importance weight ε of the hypothesized robot posi-tion xm
is the inverse of the Kullback-Leibler distance be-tween the image
Sm generated in simulation and the realimage S acquired by the
robot.
Fig. 8 (a) shows the image S obtained by the robot videocamera,
the corresponding image VE (b), and the distribu-tion of the random
variable representing the vertical edgescomputed according to Eq. 2
(c). Fig. 9 shows two exam-ple images generated by the simulator.
Fig. 9 (a),(c) and(e) correspond to the hypothesis with the highest
weight, i.e.the least Kullback-Leibler distance with respect to the
cam-era generated distribution. Fig. 9(b),(d) and (f) correspondto
an hypothesis with a lesser weight.
Fig. 10 shows the operation of the robot during the tourguide.
The Fig. shows the map build up by state-of-art laser-based robotic
algorithms. By comparing this Fig. with thereal map of the Sala
Giove in Fig. 3, it should be noticedthat the museum armchairs are
not visible to laser. However,thanks to the perceptual process, the
robot is able to integratelaser data and video camera sensory data
in a stable innermodel of the environment (Franklin 2005) and to
correctlymove and act in the museum environment (Fig. 10).
(a) (b)
(c) (d)
(e) (f)
Figure 9: (a) The simulated image corresponding to the
hy-pothesis with the highest weight. (b) A simulated image
cor-responding to an hypothesis with a lesser weight.
(c)-(d)Vertical Edge maps of the images shown in (a) and (b).
(e)-(f) Distributions of the random variables representing
verti-cal edges.
Figure 10: The operation of the robot equipped with the
ar-chitecture.
94
-
Discussions and conclusionsThe described robot perceptual
process is an active process,since it is based on a reconstruction
of the inner percepts inegocentric coordinates, but it is also
driven by the externalflow of information. It is the place in which
a global consis-tency is checked between the internal model and the
visualdata coming the sensors. Any discrepancy asks for a
read-justment of its internal model.
The robot perceptual awareness is therefore based on astage in
which two flows of information, the internal andthe external,
compete for a consistent match. There astrong analogy with the
phenomenology in human percep-tion: when one perceives the objects
of a scene he actuallyexperiences only the surfaces that are in
front of him, but atthe same time he builds an interpretation of
the objects intheir whole shape.
We maintain that the proposed system is a good startingpoint to
investigate robot phenomenology. As described inthe paper it should
be remarked that a robot equipped withperceptual awareness performs
complex tasks as museumtours, because of its inner stable
perception of itself and ofits environment.
ReferencesBurgard, W.; Cremers, A.; Fox, D.; Hähnel, D.;
Lake-meyer, G.; Schulz, D.; Steiner, W.; and Thrun, S.
1999.Experiences with and interactive museum tour-guide
robot.Artif. Intell. 114:3–55.Chella, A., and Manzotti, R., eds.
2007. Artificial Con-sciousness. Exeter, UK: Imprint
Academic.Chella, A.; Frixione, M.; and Gaglio, S. 2005. Planningby
imagination in CiceRobot, a robot for museum tours. InProceedings
of the AISB 2005 Symposium on Next Gener-ation approaches to
Machine Consciousness: Imagination,Development, Intersubjectivity,
and Embodiment, 40 – 49.Fikes, R., and Nilsson, N. 1971. STRIPS: a
new approachto the application of theorem proving to problem
solving.Artif. Intell. 2(3-4):189–208.Franklin, S. 2005.
Evolutionary pressures and a stableworld for animals and robots: A
commentary on merkel.Consciousness and Cognition 14:115 –
118.Gärdenfors, P. 2004a. Emulators as sources of hidden
cog-nitive variables. Behavioral and Brain Sciences
27(3):403.Gärdenfors, P. 2004b. How Homo Became Sapiens: Onthe
Evolution of Thinking. Oxford, UK: Oxford
UniversityPress.Gärdenfors, P. 2007. Mind-reading as control
theory. Eu-ropean Review 15(2):223 – 240.Gerdes, V., and Happee, R.
1994. The use of an inter-nal representation in fast goal-directed
movements: a mod-elling approach. Biological Cybernetics 70(6):513
– 524.Gray, J. 1995. The contents of consciousness: A
neu-ropsychological conjecture. Behavioral and Brain
Sciences18(4):659 – 722.Grush, R. 1995. Emulation and Cognition.
Ph.D. Disser-tation, University of California.
Grush, R. 2004. The emulator theory of representation:motor
control, imagery and perception. Behavioral andBrain Sciences
27:377 – 442.Haykin, S., ed. 2001. Kalman Filtering and Neural
Net-works. New York: J. Wiley and Sons.Holland, O., and Goodman, R.
2003. Robots with internalmodels - a route to machine
consciousness? Journal ofConsciousness Studies 10(4 -5):77 –
109.Holland, O.; Knight, R.; and Newcombe, R. 2007. Arobot-based
approach to machine consciousness. In Chella,A., and Manzotti, R.,
eds., Artificial Consciousness. Exeter,UK: Imprint Academic. 156 –
173.Humphrey, N. 1992. A history of mind. New York,
NY:Springer-Verlag.Llinas, R., and Pare, D. 1991. Of dreaming and
wakeful-ness. Neuroscience 44(3):521 – 535.Llinas, R. 2001. I of
the Vortex: From Neurons to Self.Cambridge, MA: MIT Press.Macaluso,
I.; Ardizzone, E.; Chella, A.; Cossentino, M.;Gentile, A.; Gradino,
R.; Infantino, I.; Liotta, M.; Rizzo,R.; and Scardino, G. 2005.
Experiences with CiceRobot,a museum guide cognitive robot. In
Bandini, S., and Man-zoni, S., eds., AI*IA 2005, volume 3673 of
Lecture Notesin Artificial Intelligence, 474 – 482. Berlin
Heidelberg:Springer-Verlag.Marr, D. 1982. Vision. New York: W.H.
Freeman and Co.Mel, B. 1986. A connectionist learning model for
3-Dmental rotation, zoom, and pan. In Proceedings of the
EigthAnnual Conference of the Cognitive Science Society, 562
–571.Mel, B. 1990. Connectionist robot motion planning:A
neurally-inspired approach to visually-guided reaching.Cambridge,
MA: Academic Press.Oztop, E.; Wolpert, D.; and Kawato, M. 2005.
Mental stateinference using visual control parameters. Cognitive
BrainResearch 22:129 – 151.Payton, D. 1990. Internalized plans: A
representationfor action resources. Robotics and Autonomous
Systems6(1):89 – 103.Stein, L. 1991. Imagination and situated
cognition. Tech-nical Report AIM-1277, MIT AI Lab.Thrun, S.;
Burgard, W.; and Fox, D. 2005. ProbabilisticRobotics. Cambridge,
MA: MIT Press.Wolpert, D.; Doya, K.; and Kawato, M. 2003. A
unify-ing computational framework for motor control and
socialinteraction. Phil. Trans. Roy. Soc. B 358:593–602.
95
/ColorImageDict > /JPEG2000ColorACSImageDict >
/JPEG2000ColorImageDict > /AntiAliasGrayImages false
/CropGrayImages true /GrayImageMinResolution 300
/GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true
/GrayImageDownsampleType /Bicubic /GrayImageResolution 300
/GrayImageDepth -1 /GrayImageMinDownsampleDepth 2
/GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true
/GrayImageFilter /DCTEncode /AutoFilterGrayImages true
/GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict >
/GrayImageDict > /JPEG2000GrayACSImageDict >
/JPEG2000GrayImageDict > /AntiAliasMonoImages false
/CropMonoImages true /MonoImageMinResolution 1200
/MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true
/MonoImageDownsampleType /Bicubic /MonoImageResolution 1200
/MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000
/EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode
/MonoImageDict > /AllowPSXObjects false /CheckCompliance [ /None
] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false
/PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000
0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true
/PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ]
/PDFXOutputIntentProfile () /PDFXOutputConditionIdentifier ()
/PDFXOutputCondition () /PDFXRegistryName () /PDFXTrapped
/False
/CreateJDFFile false /Description > /Namespace [ (Adobe)
(Common) (1.0) ] /OtherNamespaces [ > /FormElements false
/GenerateStructure false /IncludeBookmarks false /IncludeHyperlinks
false /IncludeInteractive false /IncludeLayers false
/IncludeProfiles false /MultimediaHandling /UseObjectSettings
/Namespace [ (Adobe) (CreativeSuite) (2.0) ]
/PDFXOutputIntentProfileSelector /DocumentCMYK /PreserveEditing
true /UntaggedCMYKHandling /LeaveUntagged /UntaggedRGBHandling
/UseDocumentProfile /UseDocumentBleed false >> ]>>
setdistillerparams> setpagedevice