Microsoft Robotics Studio Runtime, Simulation & VPL.

Microsoft Robotics StudioMicrosoft Robotics Studio

Runtime, Simulation & VPLRuntime, Simulation & VPL

Microsoft Robotics StudioMicrosoft Robotics StudioRuntime, Simulation & VPL Runtime, Simulation & VPL OverviewOverview

Zeddy IskandarZeddy IskandarAcademic DeveloperAcademic DeveloperMicrosoft IndonesiaMicrosoft Indonesia

Presentation OutlinePresentation Outline

Microsoft Robotics Studio RuntimeMicrosoft Robotics Studio RuntimeOverview, Components and ConceptsOverview, Components and Concepts

Decentralized Software Services (DSS)Decentralized Software Services (DSS)A lightweight, service oriented application A lightweight, service oriented application model based around state manipulationmodel based around state manipulation

Concurrency and Coordination Runtime Concurrency and Coordination Runtime (CCR) (CCR)

A message based model enabling fine A message based model enabling fine grained concurrency and coordination grained concurrency and coordination without resorting to threads, locks while without resorting to threads, locks while avoiding traditional asynchronous avoiding traditional asynchronous programming pitfallsprogramming pitfalls

Highly Diverse Market…Highly Diverse Market…

Shared Challenges… Shared Challenges… Input from industry, hobbyists, academia, research, Input from industry, hobbyists, academia, research, ……

Configuring sensors and actuators in running Configuring sensors and actuators in running systemsystem

Coordinating sensors and actuators Coordinating sensors and actuators asynchronouslyasynchronously

Starting and stopping components dynamicallyStarting and stopping components dynamically

Monitoring/Interacting/Debugging running Monitoring/Interacting/Debugging running systemsystem

Development when access to robot is limited Development when access to robot is limited

Span multiple compute unitsSpan multiple compute units

Re-use of components across hardware Re-use of components across hardware platforms and devicesplatforms and devices

Overall Runtime GoalsOverall Runtime Goals

Distributed Application Model (DSS)Distributed Application Model (DSS)Simple, flexible, and service orientedSimple, flexible, and service oriented

Coarse grained and loosely coupledCoarse grained and loosely coupled

Compatible with existing Web Compatible with existing Web infrastructureinfrastructure

Concurrent Programming Model Concurrent Programming Model (CCR)(CCR)

Focus on coordination of messagesFocus on coordination of messages

Hide traditional threads and locks Hide traditional threads and locks primitivesprimitives

Enable non thread blocking sequential Enable non thread blocking sequential execution between I/O, avoiding execution between I/O, avoiding nesting/callback chainsnesting/callback chains

DSS Application ModelDSS Application Model

The definition of a Robotics The definition of a Robotics applicationapplication

What is a Robotics What is a Robotics Application?Application?

An application is a An application is a compositioncomposition of loosely- of loosely-coupled components concurrently coupled components concurrently executingexecuting

Orchestration of sensors/actuatorsOrchestration of sensors/actuators

User interfaceUser interface

Controlled/Autonomous behaviorControlled/Autonomous behavior

A service is the unit of orchestrationA service is the unit of orchestration

MotorBumper 1(Sensor)

Orchestrator

MotorBumper 2(Sensor)

Infrared detector(Sensor)

Message Box

(Actuator)

Service PropertiesService PropertiesIdentityIdentityStructured StateStructured StateComposition through Composition through partneringpartnering

Uniform BehaviorUniform BehaviorState retrieval and State retrieval and manipulation manipulation Service creation & Service creation & TerminationTerminationEvent notificationsEvent notifications

Anatomy of a ServiceAnatomy of a Service

Port

State

Handlers

Service

FIFO

Service AbstractionService Abstraction

Services can represent any Services can represent any computationcomputation

Hardware: Sensors, actuators,…Hardware: Sensors, actuators,…

Software: UI, Storage,…Software: UI, Storage,…

Aggregation: Sensor fusion, mash-ups,…Aggregation: Sensor fusion, mash-ups,…

Reuse through compositionReuse through compositionSeparate State from Behavior (don’t hide Separate State from Behavior (don’t hide state!)state!)

Sample Bumper ServiceSample Bumper Service

Structured StateStructured State

Non-structured state handled out of Non-structured state handled out of bandband

Uniform OperationsUniform OperationsDefine operations over the stateDefine operations over the state

GET, QUERY, UPDATE, DELETE, DROP, GET, QUERY, UPDATE, DELETE, DROP, SUBSCRIBESUBSCRIBE

Sample Partner Sample Partner RelationshipRelationship

Orchestrator Service subscribes to Orchestrator Service subscribes to bumper Service waiting for sensor bumper Service waiting for sensor inputinput

A service can partner with arbitrary A service can partner with arbitrary number of other servicesnumber of other services

MotorBumper 1(Sensor)

Orchestrator

MotorBumper 2(Sensor)

Infrared detector(Sensor)

Message Box

(Actuator)

Service InteractionsService Interactions

Runtime, Hosting Environment, and Runtime, Hosting Environment, and System ServicesSystem Services

Service InteractionService Interaction

Service Interaction happens through Service Interaction happens through protocolsprotocols

HTTP is available for interactions HTTP is available for interactions from Web browser and for providing from Web browser and for providing UIUI

Enables deep integration with existing Enables deep integration with existing Web infrastructureWeb infrastructure

DSSP is a SOAP-based protocol DSSP is a SOAP-based protocol available for inter-service available for inter-service communicationcommunication

Augments HTTP model with structured Augments HTTP model with structured data manipulation and event data manipulation and event notificationsnotifications

Distributed ApplicationsDistributed Applications

Services can run Services can run directly on a directly on a single robotsingle robotServices can run Services can run on swarms of on swarms of collaborating collaborating robotsrobotsServices can run Services can run on computers in on computers in the networkthe networkApplications Applications typically a mixtypically a mix

Runtime PerformanceRuntime Performance

Service-to-Service message Service-to-Service message throughput:throughput:

Same process (with full cloning): 50,000 Same process (with full cloning): 50,000 msgs/secondmsgs/second

Cross node,cross machine: 3,000 Cross node,cross machine: 3,000 msgs/secmsgs/sec

Numbers from dual core 1.8GHz, 2GB Numbers from dual core 1.8GHz, 2GB RAMRAM

Performance allows consistent use of Performance allows consistent use of service model for both local and service model for both local and distributed casedistributed case

CCR ProgrammingCCR Programming

The foundation for writing Robotics The foundation for writing Robotics ApplicationsApplications

The CCR RuntimeThe CCR Runtime

Asynchronous Programming modelAsynchronous Programming modelA concurrency model without manual A concurrency model without manual threading, locks, semaphores, etc.threading, locks, semaphores, etc.

Based on asynchronous message passingBased on asynchronous message passing

Focus on coordination primitivesFocus on coordination primitives

Sequential execution but with no thread Sequential execution but with no thread blocking, with no need for callbacks, when blocking, with no need for callbacks, when necessarynecessary

Execution context for servicesExecution context for servicesIsolation from infrastructureIsolation from infrastructure

Isolation from other servicesIsolation from other services

Messages, Ports, and Messages, Ports, and ArbitersArbiters

Messages are sent to PortsMessages are sent to PortsPort<int> myPort = new Port<int>() ;Port<int> myPort = new Port<int>() ;myPort.Post (42) ;myPort.Post (42) ;

Ports containPorts containA FIFO data-structure holding values in a portA FIFO data-structure holding values in a portA list of continuations that can be executed A list of continuations that can be executed pending message arrival and arbitrationpending message arrival and arbitration

A continuation is represented by a C# delegateA continuation is represented by a C# delegateCan either be named or anonymousCan either be named or anonymous

ArbitersArbitersImplement common concurrency constructs Implement common concurrency constructs like join patterns and interleaved calculationslike join patterns and interleaved calculations

Example: Iterators make a Example: Iterators make a differencedifference

IEnumerator<ITask> MultistageIterator()IEnumerator<ITask> MultistageIterator()

{{

bool result = false;bool result = false;

Port<bool> port = new Port<bool>();Port<bool> port = new Port<bool>();

// Stage A, send request// Stage A, send request

port.Post(true);port.Post(true);

yield return yield return Arbiter.Receive(false, port, delegate(bool b) Arbiter.Receive(false, port, delegate(bool b) {result=b; });{result=b; });

// Stage B, received first result, send second request// Stage B, received first result, send second request

port.Post(result);port.Post(result);

yield return yield return Arbiter.Receive(false, port, delegate(bool b) Arbiter.Receive(false, port, delegate(bool b) {result=b; });{result=b; });

// Stage C, received second result, print it// Stage C, received second result, print it

Console.WriteLine(result);Console.WriteLine(result);

}}

Example: ChoiceExample: ChoicePortSet<int, string> port = new PortSet<int, string>();PortSet<int, string> port = new PortSet<int, string>();

Activate(Activate(Arbiter.Choice(port, MyIntHandler, MyStringHandler)Arbiter.Choice(port, MyIntHandler, MyStringHandler)

););

void MyIntHandler(int i)void MyIntHandler(int i){{ Console.WriteLine("Received: " + i);Console.WriteLine("Received: " + i);}}

void MyStringHandler(string s)void MyStringHandler(string s){{ Console.WriteLine("Received: " + s);Console.WriteLine("Received: " + s);}}

Example: Dynamic JoinExample: Dynamic JoinPortSet<Result,Exception> resultsPort = PortSet<Result,Exception> resultsPort = new PortSet<int>();new PortSet<int>();

// parallel computation by posting requests// parallel computation by posting requestsFor (int i=0;i<N;i++)For (int i=0;i<N;i++){{ computePort.Post(new DoWork(someData,resultsPort));computePort.Post(new DoWork(someData,resultsPort));}}

// requests complete asynchronously with unknown number// requests complete asynchronously with unknown number// of failures vs. successes// of failures vs. successesActivate(Activate(

Arbiter.MultipleItemReceive(resultsPort,Arbiter.MultipleItemReceive(resultsPort, delegate (ICollection<Result> successes, delegate (ICollection<Result> successes, ICollection<Exception> failures)ICollection<Exception> failures){{ foreach(Result r in results) foreach(Result r in results) {{ …… …… }}});});

[ServiceHandler(ServiceHandlerBehavior.[ServiceHandler(ServiceHandlerBehavior.ConcurrentConcurrent)] )] public IEnumerator<ITask> GetHandler(Get get) public IEnumerator<ITask> GetHandler(Get get) { { get.ResponsePort.Post(_state);get.ResponsePort.Post(_state); yield break; yield break; }}

[ServiceHandler(ServiceHandlerBehavior.[ServiceHandler(ServiceHandlerBehavior.ExclusiveExclusive)] )] public IEnumerator<ITask> UpdateHandler(Update update) public IEnumerator<ITask> UpdateHandler(Update update) { { _state.CurrentResult += update.Body.Value; _state.CurrentResult += update.Body.Value; update.ResponsePort.Post(new UpdateResponse()); update.ResponsePort.Post(new UpdateResponse()); yield break; yield break; }}

Declarative Coordination for Declarative Coordination for ServicesServices

SimulationSimulation

Because hardware is expensive Because hardware is expensive

Why a Simulator?Why a Simulator?

Robotics hardware Robotics hardware is expensiveis expensive

Hardware can be Hardware can be difficult to debugdifficult to debug

Hard for a team to Hard for a team to work concurrently work concurrently with limited with limited hardwarehardware

Why a Simulator?Why a Simulator?BenefitsBenefits

Low barrier to Low barrier to entryentry

Staged approachStaged approach

Easy prototypingEasy prototyping

Useful for Useful for educationeducation

Good learning and Good learning and research toolresearch tool

Why a Simulator?Why a Simulator?LimitationsLimitations

Lack of noisy dataLack of noisy data

Incomplete or Incomplete or inaccurate modelsinaccurate models

Accurate tuning Accurate tuning takes timetakes time

Simulator ArchitectureSimulator ArchitectureThe Simulator Engine ServiceThe Simulator Engine Service

Implemented as a Implemented as a serviceservice

Maintains world stateMaintains world state

Manages input devicesManages input devices

3D rendering using 3D rendering using XNAXNA

Ageia Physics Ageia Physics SimulationSimulation

Graphical User Graphical User InterfaceInterface

Editor for modeling Editor for modeling and debuggingand debugging

SimulationEngineService

XNAGraphicsLibrary

Display Hardware

AgeiaPhysicsEngine

User Interface / Editor

Visual Programming Visual Programming LanguageLanguage

Rapid Robot PrototypingRapid Robot Prototyping

MicrosoftMicrosoftVisual Programming LanguageVisual Programming Language

Visual programming environmentVisual programming environment

Simple programming with drag and Simple programming with drag and dropdrop

Application are diagramsApplication are diagramsBlocksBlocks

ConnectionsConnections

Integrated into Microsoft Robotics Integrated into Microsoft Robotics StudioStudio

Guided TourGuided Tour

Hello, World!Hello, World!

How does it work?How does it work?

Exchange of messages between Exchange of messages between activitiesactivities

ActivitiesActivitiesPerform an action on input messagePerform an action on input message

Send result of the action as output Send result of the action as output messagemessage

ConnectionsConnectionsMake output of one activity input of Make output of one activity input of secondsecond

Input message

Output message

How does it work?How does it work?

Exchange of messages between Exchange of messages between activitiesactivities

ActivitiesActivitiesPerform an action on input messagePerform an action on input message

Send result of the action as output Send result of the action as output messagemessage

ConnectionsConnectionsMake output of one activity input of Make output of one activity input of secondsecond

Input message

Data ActivityData Activity

Output message is a new data valueOutput message is a new data value

Ignores content of input messageIgnores content of input message

Different types of dataDifferent types of dataint, string, double, bool, …int, string, double, bool, …

Service ActivitiesService Activities

SimpleDialog is a DSS ServiceSimpleDialog is a DSS Service

All DSS services are activities in VPLAll DSS services are activities in VPLProvide access robot hardware, input Provide access robot hardware, input devices, behavior …devices, behavior …

You can add your own activities!You can add your own activities!

Service ActivitiesService Activities

Services can have multiple actions, Services can have multiple actions, e.g.e.g.

AlertDialog, PromptDialog, AlertDialog, PromptDialog, ConfirmDialogConfirmDialog

Most actions have two output Most actions have two output messagesmessages

Success – may contain output valuesSuccess – may contain output values

Fault – action could not completeFault – action could not complete

Selecting ServicesSelecting Services

Services appear in Services appear in the the ServicesServices toolboxtoolbox

Use the search box Use the search box to quickly find to quickly find services.services.

Making ConnectionsMaking Connections

Drag output message of first activity Drag output message of first activity ……

Making ConnectionsMaking Connections

… … to second activityto second activity

Possible targets are highlightedPossible targets are highlighted

Selecting ActionsSelecting Actions

When connecting an activity with When connecting an activity with multiple actions, select the desired multiple actions, select the desired action.action.

Selects the action to which to send the message.

Selects the output message that is used as input message for the action.

Data ConnectionsData Connections

Service actions define format Service actions define format required for input messagerequired for input message

Data Connections format messagesData Connections format messages

AlertDialog requires an input value called AlertText.

Selects the part of the input message that is used for the target.

Hello, World! RevisitedHello, World! Revisited

Hello, World! RevisitedHello, World! Revisited

Not connected. Automatically receives an empty input message.

Value:Value:

Hello, World! RevisitedHello, World! Revisited

Value:“Hello, World!”(string)

Value:“Hello, World!”(string)

Hello, World! RevisitedHello, World! Revisited

Value:“Hello, World!”(string)

Value:“Hello, World!”(string)

Hello, World! RevisitedHello, World! RevisitedData connections formats the input message before it is sent to the activity

Value:AlertText = “Hello, World!”(AlertText string)

Value:AlertText = “Hello, World!”(AlertText string)

Hello, World! RevisitedHello, World! Revisited

Not connected. Output message is ignored.

Value:Empty(Success)

Value:Empty(Success)

Example 1Example 1

Create a new activity that translates Create a new activity that translates the thumb-stick value of the XBox360 the thumb-stick value of the XBox360 controller to left and right power controller to left and right power values of a differential drive.values of a differential drive.

User defined activitiesUser defined activities

XBox360 ControllerXBox360 Controller

ThumbsticksChangThumbsticksChangee notification notification

X, Y coordinates X, Y coordinates for both thumb-for both thumb-sticks:sticks:

LeftX, LeftY LeftX, LeftY

RightX, RightY RightX, RightY

floatfloat

XX

YY

1.01.0-1.0-1.0

-1.0-1.0

1.01.0

Generic Differential DriveGeneric Differential Drive

Very simple drive systemVery simple drive systemControlled by applying different power to Controlled by applying different power to left and right motorleft and right motor

SetDrivePowerSetDrivePower Action ActionLeftWheelPower, RightWheelPower:LeftWheelPower, RightWheelPower:values between -1.0 and 1.0 (double)values between -1.0 and 1.0 (double)

Conversion ActivityConversion Activity

??

User ActivitiesUser Activities

Similar to service activitiesSimilar to service activities

Multiple actionsMultiple actions

Can send notificationsCan send notifications

Have their own stateHave their own state

Implementing ActionsImplementing ActionsInside an ActivityInside an Activity

Implementing ActionsImplementing ActionsInside an ActivityInside an Activity

Double-click opens the activity and shows its implementation.

Implementing ActionsImplementing ActionsInside an ActivityInside an Activity

Selects the current action that is shown.

Implementing ActionsImplementing ActionsInside an ActivityInside an Activity

Implementing ActionsImplementing ActionsInside an ActivityInside an Activity

Input message of the action.

Output message of the action.

Implementing ActionsImplementing ActionsInside an ActivityInside an Activity

Input message of the action.

Output message of the action.

Implementing ActionsImplementing ActionsAction PropertiesAction Properties

Input message of the action.

Output message of the action.

Implementing ActionsImplementing ActionsAction PropertiesAction Properties

Shows more properties of the action and allows to add and delete actions.

Shows more properties of the action and allows to add and delete actions.

List of actions in the activity.

Implementing ActionsImplementing ActionsAction PropertiesAction Properties

Implementing ActionsImplementing ActionsAction PropertiesAction Properties

Change name and description to something useful.

Implementing ActionsImplementing ActionsAction PropertiesAction Properties

Add input values for the thumb-stick coordinates.

Implementing ActionsImplementing ActionsAction PropertiesAction Properties

Add output values for the left and right drive power.

Implementing ActionsImplementing ActionsConnecting the Input MessageConnecting the Input Message

Use the input message like any other message.

Implementing ActionsImplementing ActionsConnecting the Input MessageConnecting the Input Message

Input values become part of the message.

Implementing ActionsImplementing ActionsConnecting the Input MessageConnecting the Input Message

Implementing ActionsImplementing ActionsCreating the Output MessageCreating the Output Message

Must put two message into one message.

Must put two messages into one message.

Implementing ActionsImplementing ActionsCreating the Output MessageCreating the Output Message

Join combines two or more messages.

Implementing ActionsImplementing ActionsCreating the Output MessageCreating the Output Message

Value:0.3251(float)

Value:0.3251(float)

Value:-0.73(float)

Value:-0.73(float)

Implementing ActionsImplementing ActionsCreating the Output MessageCreating the Output Message

Value:Left = 0.3251Right = -0.73(Left float, Right Float)

Value:Left = 0.3251Right = -0.73(Left float, Right Float)

Join only creates the output message if there are two input messages.

Implementing ActionsImplementing ActionsSending the Output MessageSending the Output Message

Connect the result message to the right border.

Implementing ActionsImplementing ActionsSending the Output MessageSending the Output Message

Implementing ActionsImplementing ActionsSending the Output MessageSending the Output Message

Conversion ActivityConversion Activity

??

Using your ActivityUsing your Activity

Control Robot with Control Robot with WiimoteWiimote

Control Robot with Control Robot with SpeechSpeech

Additional ResourcesAdditional Resources

Web siteWeb sitehttp://www.microsoft.com/robotics

Community NewsgroupCommunity Newsgrouphttp://msdn.microsoft.com/robotics/

Princeton MSRS PagePrinceton MSRS Pagehttp://pave.princeton.edu/main/urban-challenge/msrs/

© 2006 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions,

it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.