Cognitive robotics tools and technology

Martin Peniak

Cognitive Robotics Technology and Tools

iCub – iCognitive universal body

YARP – Yet Another Robotics Platform

GPU – Graphics Processing Unit

Aquila – Acquisition of language and actions

Overview

iCub humanoid robot

The dimensions are similar to that of a 3.5 year old child

53 degrees of freedom

Came from the European Framework 6 project: RobotCub (www.robotcub.org)

There are now 20 iCubs in different labs in Europe and 1 in the US

Continued design - v2.0 to come out

Various ongoing project outcomes are distributed via an open-source software repository and via hardware upgrades

A free iCub simulator is available

iCub humanoid robotDexterous hands for object manipulation

iCub humanoid robotAlmost walking

iCub humanoid robotSimulator

Open-source

Developed as part of a joint effort with the European project iTalk

Widely adopted within cognitive robotics community

V. Tikhanoff, P. Fitzpatrick, F. Nori, L. Natale, G. Metta, and A. Cangelosi, “The icub humanoid robotsimulator,” In International Conference on Intel ligentRObots and Systems IROS, Nice, France, 2008

Can be broken down into:libYARP_OS - interfacing with the operating system(s) to support easy streaming of data across many threads across many machines

libYARP_sig - performing common signal processing tasks (visual, auditory) in an open manner easily interfaced with other commonly used libraries, for example OpenCV

libYARP_dev - interfacing with common devices used in robotics: framegrabbers, digital cameras, motor control boards, etc.

YARPYet Another Robotic Platform

Supports building a robot control system as a collection of programs communicating via tcp, udp, multicast, local, MPI

YARPYet Another Robotic Platform

YARP serverYARP module YARP module YARP module

/icub /icubSim

YARP module YARP module

YARP module

YARP and iCub simulator

Examples

yarp namespaceyarp pingyarp readyarp regressionyarp resourceyarp rpcyarp rpcserveryarp runyarp serveryarp terminateyarp topicyarp versionyarp waityarp whereyarp write

YARPTerminal commands

yarpyarp helpyarp checkyarp cleanyarp cmakeyarp confyarp detectyarp disconnectyarp existsyarp forwardyarp helpyarp nameyarp name checkyarp name listyarp name unregister

yarp rpc /icubSim/left_leg/rpc:I6 joints

yarp rpc /icubSim/right_leg/rpc:i 6 joints

yarp rpc /icubSim/torso/rpc:I3 joints

yarp rpc /icubSim/left_arm/rpc:Ithe arm includes the hand for a total of 16 controlled degrees of freedom

yarp rpc /icubSim/right_arm/rpc:Istructure is identical to the left arm

yarp rpc /icubSim/head/rpc:I6 joints

Terminal 1: yarpserverStarts YARP server

Terminal 2: iCub_SIMStarts iCub simulator

Terminal 3: yarp rpc /icubSim/left_arm/rpc:I

Terminal 3: set pos 0 – 90

Terminal 3: set vel 0 50

Terminal 3: set pos 0 90

YARP and iCub simulatorControlling motors

Terminal 1: yarpserver

Terminal 2: iCub_SIM

Terminal 3: yarpview /left

Terminal 3: yarpview /right

Terminal 3: yarp connect /icubSim/cam/left /left

Terminal 3: yarp connect /icubSim/cam/right /right

Move the iCub’s head and see the vision changing:Terminal 3: yarp rpc /icubSim/head/rpc:I

Terminal 3: set pos 0 -30 (head will move down)

Terminal 3: set pos 0 30 (head will move up)

Easier way is to use the existing graphical user interface:Terminal 3: robotMotorGui

To display camera outputs form the real iCub change the /icubSim prefix with /icub

YARP and iCub simulatorDisplaying camera outputs and controlling joints

Computation of visual, auditory, and tactile perception while performing elaborate motor control in real-time requires a lot of computation

YARP can run across any number of machines with different operating systems

YARPYet Another Robotic Platform

YARP server(Linux)

YARP module(Windows)

YARP module(OSX)

YARP module(Linux)

/icub /icubSim

YARP module(OSX)

YARP module(Linux)

YARP module(Windows)

Biologically-inspired models used in cognitive robotics are inherently parallel and can greatly benefit from massively parallel devices such as GPU processors

The GPU Computing Revolution CPUGPU

Different goals produce different designsGPU assumes work load is highly parallelCPU must be good at everything, parallel or not

CPU: minimize latency experienced by 1 threadbig on-chip cachessophisticated control logic

GPU: maximize throughput of all threads# threads in flight limited by resources => lots of resources (registers, bandwidth, etc.)multithreading can hide latency => skip the big cachesshare control logic across many threads

CPU vs. GPU

CPU vs. GPU

High throughput computationGeForce GTX 690: 2 x 2811 GFLOP/s

High bandwidth memoryGeForce GTX 690: 2 x 192 GB/s

High availability to all200+ million CUDA-capable GPUs in the world

GPU Evolution

1995 2000 2005 2010

RIVA 1283M xtors

GeForce 256 23M xtors

GeForce FX125M xtors

GeForce 8800

681M xtors

GeForce 3 60M xtors

“Fermi”3B xtors

“Kepler”

7B xtors

2012

21

Nvidia creates CUDA to facilitate the development of parallel programs on GPUs (2007)

The CUDA language is ANSI C extended with very few keywords for labeling data-parallel functions (kernels) and their associated data

Nvidia technology benefits from massive economies of scale in the gaming market, CUDA-enabled cards are very inexpensive for the performance they provide

Programming GPUs with CUDAHistory

Cognitive robotics research toolkit

Aquila

InspirationSimplification of commonly used features on the iCub and the simulator

InspirationDevelopment of bio-inspired models and tools

InspirationScalability modularity and platform-independency

/aquila/yarprun/0 /aquila/yarprun/1 /aquila/yarprun/2

GPUCPU

InspirationOvercoming computational constrains by using GPU processors

Motion compliance < 1 ms

Vision (30fps) < 33 ms

Vision (60fps) < 16 ms

We typically take 33 ms as the cut-off time. 1 complete cycle of everything critical MUST be completed in that time.

Of course some processes are not critical and their information can be used as and when it becomes available, subject to various constraints.

Aquila 2.0

Questions?