1 Robots Unlimited PHD Summerschool July 2006 Alexander Brändle Marco Combetto.

1

Robots Unlimited

PHD SummerschoolJuly 2006

Alexander BrändleMarco Combetto

2

Agenda

• Who we are?

• Why robotics?

• Activity overview

• Future ideas

3

Intelligent Environments Team

Alexander Braendle

MarcoCombetto

Pierre-LouisXech

AndreasHeil

(PHD Student)

4

Why robotics?• How will Future Environments look like?

– Challenges/Needs– Personal Devices, Embedded Devices, Sensors, Robots, Appliances?– Enabling technologies,Programming paradigms,Interaction paradigms?

• How will future applications look like?– Wide range of user types?– Mixing real and virtual world?

Robots could form a part of it!

5

The Robotics Wave• Service and consumer markets

just emerging– Remote assistance/presence– Assistive– Facilities maintenance– Security– Education– Entertainment

• Academic Research– Complex topic – Moving from 8/16 to 32 bit– Lots of hand-coded solutions– Education and Hobbyist channel– Still mostly 8 bit, starting to shift

• Strong governmental interest worldwide EURON roadmap:

2015 -> It‘s all about software

6

Robots!• Wide range of applications

– Commercial use– Academic use– Personal use

• How to program them?

• Challenges– Complexity– Reusability– Reliability– Resources– Tools– Technologies– Choice– Sharing– Transference of skills/experience

7

Links

• European Robotics Network (EURON)

• European Robotics Platform (EUROP)

• IEEE RAS Technical Commitee on ‚Programming Environments in Robotics and Automation‘

8

Robots, too?• Software

– Internet Explorer– Media Player – MS Agent– Mobile Phones– …

Definition: A robot is a device, hard- or software with the capability of sensing and (re-)acting.

9

Robotics at Microsoft [Research (Cambridge)] Robots in human environments

• End-user programming– Developer Tools (Fischertechnik-

Demo, Scatterweb-Demo)– Visual Programming (.FUN-Demo)– End-user „debugging“

– Microsoft Robotics Studio– Coding4Fun

• Enabling new applications– Emotional oriented computing– Personal robotics– Self-reconfigurable structures– Learn from interaction with animals

10

The Fischertechnik ROBO Interface

– The in- and outputs• 8 digital inputs • 2 digital and analog distance sensors • 4 analog sensors for resistance and voltage• 4 motors with 8 different speeds

• The board• Serial port, COM, RS232• USB • Infrared• R/F module available• Ethernet for the next

hardware revision planned

TS%SystemRoot%\System32\mstsc.exe

11

Adding Control

– Now we want to control the robot

– Let‘s take another off the shelf product

– Ordinary Joystick

+

12

Coding4Fun

http://msdn.microsoft.com/coding4fun/

13

Integrating Sensors– Sensors are they eyes and ears of robots– Increasing Demand for easy robust sensor platform from biology, ecology, civil

engineering et al.

How to get easy access to Sensor Nodes systems

A challenge on it‘s own:

• Where are sensor node platforms heading?

• Dealing with vast amounts of real-time data becoming available

14

Sensor Networks in the real world

• Robustness … – WSNs have to work 24/7– WSNs have to offer uniform APIs– Long-lived, autonomous networks– Self-organizing and energy efficient– Routing– Data aggregation– Deployment & Setup

• …Better development support– Realistic simulation tools– Heterogeneous testbeds– Useful traces– Tool integration (in well know platforms)– Development for heterogeneous systems

ScatterWeb/.NET

SN

GWSN

SN

SNSN

SN SN

SN

SNSN

SN

GW

GW

GW

Bluetooth, …

Ethernet

SN

GPRS WLAN

15

Sensors (ScatterWeb) An open and flexible platform for rapid protoyping & implementation of

wireless sensor networks

• Nodes– with/without sensors

• Sensors– Luminosity, noise detection,

vibration, PIR movement detection,• Microphone/speaker• IR sender/receiver

– stand-alone/modular• Acceleration, humidity, temperature, luminosity, noise detection,

vibration, PIR movement detection on demand– Stand-by: 7.6µA, 5 years life-time with AA battery and 1% duty-cycle

• Gateways– WLAN, Ethernet, Bluetooth, GPS, GSM/GPRS, USB, RS485 , serial…

• Software– Management, flashing, routing, ns-2 simulation models– TCP/IP, web server (http://193.10.67.150/), Contiki, TinyOS, …– Basic functions for energy management, routing

• www.scatterweb.net

Jochen Schiller,Free University Berlin

16

.NET for ScatterWeb

• Sensor world available to every developer

• Easy access to sensor values, events, and functions

• Understandable namespaces and interfaces

• Support for IntelliSense and dynamic help

• Well known programming model (events, methods, properties)

• Extensibility of the nodes’ logic (Tiny C#) and instant IntelliSense-Update

• Development, deployment and debugging within Visual Studio

• Extend the .NET tools and architecture to small devices

Jochen Schiller,Free University Berlin

17

A first step

• Attractive for developers

• End-users?

18

.FUN

• A compelling & engaging programmable environment to play & learn for children (introduce children to Computing in new ways)

• Make technologies of tomorrow accessible to non technical market (children, nurse, elderly, machine operator)

• Linking real and virtual world

• Wider market potential of Robotics (Industrial, Assistive technology, new consumer products, health)

Technical University Berlin

19

Next steps

• Keep graphical representation

• Using context data

• Domain specific language

• Changing the paradigm …

20

Integrating a Context Server 1/2

Context Server Application(s)Sensors

DB

Raw Sensor Data Context Events

Context Requests

Torben Weis, University of Stuttgart

21

Integrating a Context Server 2/2

Context Server Application(s)Context Simulator

DB

Raw Sensor Data Context Events

Context Requests3D Data

Torben Weis, University of Stuttgart

22 Torben Weis, University of Stuttgart

25

Microsoft Robotics Studio

• Concurrency Concurrency • Services Services infrastructureinfrastructure

• Samples and Samples and tutorialstutorials• Robot servicesRobot services• Robot modelsRobot models•Technology Technology servicesservices•

Microsoft Robotics StudioMicrosoft Robotics Studio

• Simulation ToolSimulation Tool

• Visual Visual Programming Programming Language Language

A development platform for robotics community, supporting a wide A development platform for robotics community, supporting a wide variety of users, hardware, and application scenarios.variety of users, hardware, and application scenarios.

RuntimeRuntime AuthoringAuthoring ToolsTools Services and SamplesServices and Samples

Make it easy to manage asynchronous componentsAvoid need to understand manual threading, semaphores, etc.

•Provide a scalable programming model •Make state observable, easily accessible•Provide for reusability and failure•Support component discovery and composition•Support remote/distributed execution

26

• Support standalone and distributed processing scenarios

Connectedoperation(remote execution on PC)

Disconnectedautonomous operation (with optional networked monitoring)

Distributed execution(execution across compute units)

Microsoft Robotics StudioKey Runtime Features

27

Microsoft Robotics Studio

• Extensible to a wide variety of hardware

28

Microsoft Robotics StudioServices and Samples

• Over 15 tutorials– VB.Net, C#, JScript

• Support for– LEGO® Mindstorms® RCX

– LEGO® Mindstorms® NXT

– fischertechnik®

– MobileRobots Pioneer P3™

29

Microsoft Robotics StudioOther University Support

• Bryn Mawr College• Carnegie Mellon University• Cornell University• Georgia Tech• Massachusetts Institute of Technology• Stanford University• University of Pennsylvania • University of Pisa• University of Southern California• University of Washington

31

Robotics at Microsoft [Research (Cambridge)] Robots in human environments

• End-user programming– Developer Tools (Fischertechnik-

Demo, Scatterweb-Demo)– Visual Programming (.FUN-Demo)– End-user „debugging“

– Microsoft Robotics Studio– Coding4Fun

• Enabling new applications– Emotional oriented computing– Personal robotics– Self-reconfigurable structures– Learn from interaction with animals

32

Emotion-Oriented Computing

• General Goal: Make interaction between human and machine more natural for the humans

• Machine should be able to:– To register human emotions– To convey and comunicate emotion– To understand the emotional relevance of the event

04/10/23 32Microsoft Internal Only

34

Human Centred, Affects and Emotions

04/10/23 34Microsoft Internal Only

• Enhance communication through compelling, fun and emotional interactions with computing. Seeking to make the process more intuitive, interactive and appealing to a wider group of people. Experimenting, evolving, evaluate emotional models

• Support privacy, intimacy and different level of information sharing

• Extending the software as a new medium – Affective Media and Mobile media (affective loop)– Participatory creation, exchange, authoring and fruition– Storing the intimacy, the private value of the things– Fusion of different kind of data sensors, embodiments,

35

SensingSensingInteraction Interaction

with Environmentswith Environments

Some examples Some examples

36

Affective DiaryDesigning for bodily expressiveness and self-reflection

• Collecting memories – including body memorabilia mingled with mobile materials (SMS, MMS, photographs, music listened to, video,..

• Offering a diary medium in which those memories can be mirrored and organised

• Empowering the user to create meaning and alter those representations

• Prototype – build on TabletPC and Smartphone– Sensordata (movement, arousal)

04/10/23 Microsoft Internal Only 36

37

BSP: Interactive storytelling in vast location based

• Pervasive computing: Location based games provide navigational challenges e.g. Chasing

• Mobile Media: A flexible media and framework for interaction • Interactive storytelling: Balance linearity with user control, believable

characters • The concept of believable environments, and our implementation, put a

research focus on possibilities to:– Enrich pervasive games with location

dependent narratives– Design “the stage set” to improve

interactive storytelling

04/10/23 Microsoft Internal Only 37

38

SensingSensingInteraction Interaction with Robotswith Robots

An example An example

40 40

The mind and the body

• Neurophysiologists suggest that the body plays a crucial role in our cognitive process

• The brain experiences the world through the body:– The environment is well known to the brain– Outside environment is a set of states of inside

• Agents with body (embodied agents) try to mimic this schema to produce intelligent behavior

• Dautenhan and Jakobi show that body’s presence influences the behavior of software (anything already heard?)

41 41

Component structure

Knowledge Base

Body Map

WiFi Roblet

MotionRoblet

Vision Roblet

Sensors

Internal perception

Network perception

Arm Roblet

42 42

Robotics4.net

• Realization of a platform for define the body of a robot as a set of agents (Roblets) that acts as intermediaries between the devices and the reasoning software

• The robot provides a physical body and a set of core services to support the development of highly autonomous agent situated in both cyberspace (e.g Internet, virtual entities) and real world

• Definition of an Object Model, an API, a set of abstraction that allows:– Portability (on different type of platforms)– Scalability from small (e.g. Lego) to complex systems– Integration with the MS Development Platform and MS OS and

Application suites

43 43

Functions implemented

• The platform provide the following base services to support the software implementing the roblets:– General purpose functions (I/O)– Basic motion abilities– Collision avoidance software– Vision software for face recognition– Speech and gesturing recognition– Positioning and navigation aided by video input– Network perception– IM interface

– The Architecture is available as a simple Software Development Kit freely downloadable from http://www.robotics4.net

44

Perception

04/10/23 Microsoft Internal Only 44

45 45

Evolving the framework

• Reduce the programming model of Robotics4.NET to the one adopted by the Robotics Studio CTP

• Extending the Mind/Body model with self-developing tools, model and methods to verify the behavior of systems imposing very high level constrains and how that can be both formally and programmatically verified

• Testing interaction in real scenarios

46

Questions