Distributed Control Lab - A component-based application Overview Architecture Experiments
Distributed Control Lab-
A component-based application
OverviewArchitectureExperiments
Outline
MotivationThe Distributed Control Lab (DCL) Architecture“Foucault's Pendulum Details”
– Hardware / Software Architecture– Control Algorithms
“Hau den Lukas”– Hardware Architecture– Control Software
Additional ExperimentsMalicious Code Problem / Solutions
Motivation
Online access to physical experiments over the WebTest-bed for interconnected middleware-components and embedded systemsReach a predictable system behaviour in unstable environmentsStudy techniques to prevent malicious code damaging physical equipment Foucault's Pendulum demonstrates usage of dynamic reconfiguration for online replacement of user control
Distributed Control Lab
2001 project start at Hasso-Plattner-InstitutePractice of writing control algorithms for real-time control problemsstudy of system predictability, availability and security in context of middleware-based dynamic control systems Extensible architecture for hosting physical control experimentsInvestigation of algorithms for user code observation and replacement of control componentsExperiment : physical installation and specific control software
The Distributed Control Lab
Distributed Control Lab (V2)Architecture
Admin
ExperimentManager
Ticket Service
Job Queue
R
R
SOAPFrontend
R
RResult Manager
Job Results
ExperimentController
Tickets Users
DCLFrontend
User
R
SOAP
ExperimentController
R
RSOAP
R
Experiment Service
DISCOURSEUsers
ClientWeb-Browser
DCL Messaging Protocols
http/htmlASP.NET
Page
IIS ASP.NETWeb/Application
Server
DCL-Web Service
IIS ASP.NETWeb/Application
Server
DCL Experiment-/
Resultmanager
.NET RemotingService
DCLExperiment-
Controller
.NET RemotingService
EmbeddedControlDevice
RT-Linux PCBrick OS
...
DBMS
SOAP
.NETRemoting
ODBC
.NETRemoting
TCP/IP
IrDA
Problem : Malicous Code
Investigation of Solution for malicious code detection– Source code analysis– Experiment-specific languages / Language limitations – .NET code access security– Simulation before execution on physical experiment– Analytic Redundancy
Online observation of user programsReplacement of user programs before reach of uncontrollable stateDynamic reconfiguration of component-based control applicationMonitoring of environmental settings and component states
Foucault’s Pendulum
First installation 1848 by Leon Foucault in the Pantheon in ParisDemonstrates earth rotationToday many installation including one in UN-building in New York Problem : Pendulum must be kept swingingSolution : electro magnet under an iron ball Experiment: Find best control algorithm to keep the pendulum swinging
– Using minimum energy – Reaching the highest amplitude
USB Host-Controller
User-Controller
Safety-Controller
EventDuplication
Operating System I/OCheck HalfFull Flag
ConfigurationManager
Binary Reader / Filter, Smooth
Event Creation
Binary Writer
Binary Stream Generation
User Mode
Kernel Mode
Cypress EZ-USB
Read
Write
64 Byte
I/O request packetLight Sensor
Electro Magnet
FIFO-Memory 4 KBytes
ReadFile() WriteFile()
Pendulum Control – detailed• Laser light barriers sampled into
4KByte FIFO-memorys with 23,4 kHz
• USB-Controller checks half-full-Flag
• 64 Byte blocks of data transferred via USB 1.1
• Real-Time OS-Threads process in-coming signals / produce out-going bit stream
Pendulum-API : Control Event
public class ControlEvent{
public int nr; // sensor or actuator identifierpublic int state; // actuator 1 ON - 0 OFF
// light barrier // 1 light -> dark// 0 dark -> light
public long timestamp; // global time stamp}
Pendulum-API : Interface
public interface Pendulum{
// Dequeue next event// Blocks if no event present until next event occurspublic ControlEvent GetNext();
// Queue next event to put energy on / offpublic bool SendEvent(ControlEvent input);
// Get global time stamp 1 micro seconds logical resolution public long GetTime();
}
Pendulum: Example Control Code
while(true){
// get next eventControlEvent ev = pendel.GetNext();if(ev.state == 1) // pendulum enters light barrier{
// switch magnet onpendel.SendEvent(new ControlEvent(ev.timestamp+1000,0,1));// switch it off after 5 mspendel.SendEvent(new ControlEvent(ev.timestamp+6000,0,0));
}}
Pendulum Control ProcessGeneration
while(true){
// get next eventControlEvent ev = pendel.GetNext();if(ev.state == 1) // pendulum enters light barrier{
// switch magnet onpendel.SendEvent(new ControlEvent(ev.timestamp+1000,0,1));// switch it off after 5/23 secondspendel.SendEvent(new ControlEvent(ev.timestamp+6000,0,0));
}}
send to DCL while(true){
// get next eventControlEvent ev = pendel.GetNext();if(ev.state == 1) // pendulum enters light barrier{
// switch magnet onpendel.SendEvent(new ControlEvent(ev.timestamp+1000,0,1));// switch it off after 5/23 secondspendel.SendEvent(new ControlEvent(ev.timestamp+6000,0,0));
}}
Public Class UserControl
{
Public Static void Main(String[] args)
{
}
}
Configuration Code
Control Code
generate user control class
user control code
compile
control
Pendulum : Increase of amplitude
1. Get current oscillation time Toz ( time for a full oscillation )
2. Get current time the ball is covers the light barrier Ts ( indicates speed )
3. If ball enters light barrier :1. Queue Enable Magnet Event at Tcurrent+Toz/42. Queue Disable Mag. Event at :
Tcurrent + Toz/2 + Ts/2 – Tmf1. Tmf – duration of magnetic field shut down
4. Repeat steps 1-4
Code Access Security-CAS in .NET
Supplements operating system security mechanisms Trust to assemblies vary depending of origin and publisherBased on evidences of assemblies
– Application directory– Strong name– URL– Signature
Permissions restrict access to system resourcesDCL: Configuration of usable class library classes
Code Access Permissions
Represent access to a protected resource or the ability to perform a protected operationEnvironment VariablesFile DialogFile I/OIsolated Storage FileReflectionRegistry SecurityUser InterfaceDNS
PrintingEvent LogSocket AccessWeb AccessPerformance CounterDirectory ServicesMessage QueueService ControllerSQL Client
Evidence Based Security
Assembly evidence is matched against code groups to gain permissionCode group consist of – Membership condition– Set of code access permissions
Security policies contain code groups that map assemblies to permissionsPolicies in .Net : Enterprise, Machine, User, Application DomainPolicy Evaluation: top-down
.NET Security in the Pendulum Experiment
Code group in machine policy applies only to usercode.exe assembly
– Custom Permission Set assign to the code group
No permissions, except:– Execution permission– Access to driver component via .NET Remoting
Localhost, one specified socket, one URI– File I/O in the local directory
Security exception if any restriction is violated– Check required permissions before code execution– .NET Attributes
Pendulum ExperimentDynamic Properties
Code Access Security provides safe environment for execution of foreign codeThere are still dynamic properties at the experiment that can not be handled by CAS
– Expiration of available execution time (fairness)– Expiration of available energy– Overheating of the coil– the pendulum falls below a critical amplitude (energy) - so it
could not be kept swinging– Unexpected termination of user programs– Exaggerated use of memory
Our Approach : Dynamic Reconfiguration as safe-guard mechanism
Mapping of profiles to application configurations based on environmental conditions and component statesSelection of application configuration according to conditions provides best service for a given situationDefinition of
– observer : monitoring of environmental settings and component states
– profiles : mapping of environmental conditions to application configurations
– configurations of component-based applicationsOnline monitoring of environment and componentsChange of application configuration using dynamic reconfiguration if required (changed conditions)
Configuration Creator Tool
Pendulum Experiment Control Configurations
USB-Driver
Event Queuing
SafetyController
UserProgram
Event Duplicator
UserProgram
Configuration 1 : safety controller
Configuration 2 : user program (cold standby)
Configuration 3 : user program (warm standby)
USB
SafetyController
SafetyController
Measurements:Abnormal Termination of User Program
„Higher Striker“ – „Hau den Lukas“
Similar hardware to pendulum experiment– Parallel I/O / 38 kHz sample rate / 256 Byte buffer
Use of Real Time OS – Smaller Buffers, Higher Sampling Frequency – Short control delay– COTS x86 PC
Intel Celeron 633 MHz, 128 MB RAM (max 64 MB usable)10 Mbit/s LAN (NE 2000 PCI)
Combination of non-RT .Net and RT applicationCE-PC Windows Ce.Net 4.2
“Higher Striker” - Architecture
A/D TransformationFIFO-Memory
Optical Signal Gen.
CE-PCPlattformBuilder/
Debug Host
.Net DCLController
PC
OS Image
DebuggingParallel Port I/O
Download COMPILED User ControlAlgorithms
User Code
light barriers
Switch
Power
DCL
„Higher Striker“Minimal Program
INITIALIZE;do{READ();WRITE(buffer);GETSTATUS()if(EMPTY_FLAG_LS) Sleep(1);
}while(!EOF(buffer))
Controlling the Experiment
Abstract from Direct Hardware Programming Provide Higher Level API to the experiment2 possible control scenarios :– Calculation before runtime, analysis of result after
experiment execution for next run– Calculation of control signals during runtime – hard
deadlinesInvestigation of various operating systems.Net Compact Framework and real time
„Higher Striker“ - Event List - API
Definition before runtimeTransformation into byte stream before runtimeSimple checks possible (temperature of magnet)Generation of event list after runtime for analysis for next experiment runsNot flexible – but simple to implement
0;1;10;0;20;2;25;0;50;4;80;0;
11111111110000000000222220000000000000000000000000000088888888888888888888888888880000000000000
Higher Striker Control APIint HDLInitialize ( );int HDLInitialize ( unsigned char Writeahead,
unsigned char Initial[]);
int HDLStartClock( );int HDLStopClock( );
int HDLPerform( unsigned char * Write, unsigned char * Read );
int HDLGetStatus();int HDLGetError( int * LSError, int * MSError );
Higher Striker – Watch Dog
Init Active Stopsignal
not signaled for 5 reboots
if not signaled for 5 sec reboot
70 secpower on
power off
Hardware Watchdog connected / signaled via serial communication interface of control PC
In case of system hang-up control PC will be rebooted
Atmel AVR 8-Bit Tiny 12 microcontroller
Higher Striker Simulation
Used Buffer (0-255 Byte)
Cylinder Speed
Height
VideoAnimation
DCL - Grid Integration
Heterogeneous – X86,Itanium, PowerPC– Windows 2000/XP, Linux,
Mac OS X
DRMAA – Job Submission and Control for Clusters and GridsGLOBUSIDLE-Time
– Condor– Sun Grid Engine, Condor
Grid
Grid
- Increased Throughput
- Increased Response Time
Povray
Simulation
Fischertechnik Production Line
Control via Fischertechnik Intelligent Interface– Serial communication interface, 9600 Bits/sec– 4+4 Digital Outputs, 8+8 Digital Inputs, 2 Analog
Inputs– Installation:
4 Intelligent InterfacesPneumatic Actuators 1 robot arm ( 2 free degrees )
Controlling Fischertechnik using Beckhoff Industrial-PCs and the DCL
DCL
.NET Controller
IIF
ADS.NET Visual Studio Plug-In
TwinCat PLC-Control
Märklin Railway
Märklin Digital Model RailwayControl via serial interface 9600 Bit/secS-88 Bus and controllerConfigurable train speedFeedback via rail sensorsAbout 20 switchesControlled by x86 PCCurrently running W2K
Running .NET on devices
Why ?– Rapid software development for embedded devices– Object-oriented programming model, type safety– Many available tools, know-how, many experts– Code access security for remote lab experiments
How ? Possibilities:– PC-based execution – Communication to device for I/O– Native execution of translated IL-Code– Interpretation of stripped IL-Code on the device– Running .NET on more powerful embedded
devices to control small embedded devices
CLI2RCX- .NET runtime for Lego Mindstorm
Our gcc (Gnu Compiler Collection) frontend supports the fullECMA-335 standard and can parse any conformant .NET assembly. Port for Renesas/Hitachi H8-300 microcontroller underway
– Extremely small footprint (32 KB memory)– Variable and method definitions, method calls, integer ops up and
running
ildasm.NetAssemblyC#
VB .Net
C++ .Net
RCXbinary(srec -format)
targetbinaries
GCC
CILFront End RTL
HitachiH8-300
Back End
Backends
mnemonicCIL
Assembly
HitachiH8300
Backend
Mirco.NET - .NET Interpreter for Lego Mindstorm
• Application assemblies merged and striped (metadata, portable executable header, references)
• Application merged to one „micro“ assembly and transferred to device embedded device
• Interpreter reads “micro” executions according instructions using a separate runtime
MicroIL
Assembly
ApplicationAssembly Dependencies
Base ClassLibrary
DependenciesDependency
MicroIL
Assembly
Runtime
Interpreter
OperatingSystem
Hardware
Embeddded Device Host
stripmergedownload
CLI2RCX- .NET – Current State
Version 1.0 release implements the following features of the .NET platform:– primitive datatypes: bool, byte, short, int– classes, including instance attributes and properties.– static and instance methods, including parameters,
local variables, and constructors.– arithmethic operations– control flow operations: conditional and
unconditional branch instructions.
CLI2RCX- .NET – Current State
The following features are not yet supported (as are most features not mentioned in either list):
– most value types (enums, structs, delegates, floats, doubles)– strings– single-dimensional zero-based arrays (partially complete)– multi-dimensional or non-zero-based arrays– Inheritance, polymorphism, and late binding– interfaces– exceptions– static class attributes, class constructors, events– boxing and unboxing– arithmetic instructions that detect overflows– any predefined class except for System.Object
Related Work
“Verbund Virtuelles Labor” project at University Reutlingen / GermanyiLab project (WebLab) at MIT Virtual Lab at University of Hagen /GermanyTele-Laboratory at University of PisaTele-Lab / Simplex architecture “Ein ferngesteuertes Labor im Internet”www.remote-lab.de