International Conference “NANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS”

International ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALSD. Metafas1, M. Rangoussi1, B. Meparishvili2, G. Goderdzishvili2

(1) Department of Electronics Eng., TEI Piraeus, Greece(2) Faculty of Informatics and Management Systems, Georgian Technical University, Georgia

Dynamic Applications on Complex Distributed Systems and Machine Learning algorithmsInternational ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS Tbilisi, Georgia 6-9 Jun 2013

Internet of Things (IoT)

IoT: Focus on Critical Infrastructures

SCADA and Wireless Sensors and Actuator Networks (WSANs)

Review of existing middleware for networked robots and Wireless Sensor Networks

The need of a WSAN middleware

WSAN middleware: Focus on autonomous decision-making

Machine LearningOutlineInternational ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS

Internet of Things Internet of Things (IoT) the next step in always on computing promising a world of networked and interconnected devices. ITU 2005Machine-to-Machine, smart systems, web of things, sensor web, The real (i.e., physical), virtual, and digital worlds are converging, thanks to an ever proliferation of connected (smart) sensors and objects, ubiquitous wireless networks, communications standards and the activities of humans themselves Economist 2010

International ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS

Master of Science in Embedded Systems DesignALaRI Faculty of informatics - University of Lugano, USI

Internet of ThingsThe vision of IoT is changing from emerging realizations, via e.g., RFIDs and WSANs, to include Internet-connected objects, whereby objects will include any possible physical entity that could be classified according to size, mobility, power, connectivity, automation, physical/logical type, etc. Internet-connected objects is a central part within International/European strategies/roadmaps. This change is characterized as metamorphosis of objects in EU Cluster of European Research Projects on the IoT. A key enabler/prerequisite for the proliferation of internet-connected objects is the widespread deployment of IPv6.



IoT: Focus on Critical InfrastructuresInternational ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS


IoT: Focus on Critical InfrastructuresOur main focus is Critical Infrastructures (CIs), i.e., those facilities that provide essential services for everyday life (such as energy, food, water, transport, communications, health).

Protecting Critical Infrastructures (CIs) against disruption of any kind is deemed vital to the national/international security, public health and safety, as well as economic prosperity.PCS (Process Control Systems)/SCADA (Supervisory Control and Data Acquisition) systems are largely used for data acquisition and control over large and geographically distributed infrastructures; these systems are critical elements in every aspect of literally every CI (e.g., Oil/Gas, Electricity, Transportation). International ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS


Critical Infrastructures: SCADA systemsSCADA systems range from relatively simple networks that monitor environmental conditions of a given location to incredibly complex systems that monitor all the activity in a power plant or a municipal water system.



SCADA and Wireless Sensors and Actuator Networks (WSANs)SCADA systems are moving from expensive, complex, proprietary technologies to affordable, standards-based, open solutions. One of the promising (in cost and usage) technologies for use in SCADA systems includes IEEE 802.15.4-based Wireless Sensors and Actuator Networks (WSANs) such as ZigBee, WirelessHART and ISA100.11a.

but ..

Wireless systems are vulnerable to in terms of cyber attacks.CIs (like oil and gas and power grid) are attractive target for cyber-attacks.The use of WSANs in CIs results in a deviation from the WSAN assumption of densely deployed nodes. This is not the case when they are used for SCADA of CIs, and a non-communicating node can have much more significant effect than the standard WSAN scenario.International ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS


WSAN middlewareMost current middleware solutions for WSAN systems fail to take into account two key aspects: robustness of the solution and support for heterogeneous configurations. This is especially important in environments where their complexity makes it absolutely necessary to have different types of techniques and monitoring equipment that needs to interact to project CIs.

For example, the case where oil pipelines are to be monitored require the cooperation and interaction of sensors, actuators and mobile devices (UAVs, robots, etc.) that interact in a seamless way to achieve a common goal. International ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS


Dynamic Applications for CIs and the need of a high level middlewareDynamic applications for CIs require complex distributed systems consisting of a number of embedded hardware and software components. Since these components are heterogeneous, it is necessary to find ways to mask this heterogeneity and offer some innovative solutions to develop the applications. In addition, advanced and efficient techniques for cooperation and collaboration among them are required to achieve the desired goals. Therefore, there must be new software services (middleware) that act as the glue to link everything together in an efficient manner, supporting concurrency-intensive operations, enhancing collaboration, and insuring efficiency and robustness. How to get efficient communication links, design better ways to exchange and use information over these links and manage dynamic work environments are some of the common issues.Proposed middleware approaches for specific technologies such as ad-hoc networks, wireless sensor networks and robotics are addressing a number of issues but clearly in the broader scope of Dynamic applications for CIs there is a need of a widely accepted high-level middleware addressing all open issues and meet the design and implementation of different challenges. International ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS


Dynamic Applications for CIs and the need of a high level middleware (2)Objective: To design a distributed middleware and communications framework that will equip objects/nodes with advanced functionalities and abilities to understand their environment for collaborative and autonomous decision-making. To prescribe all the components of the distributed middleware, which shall enable cooperation, intelligence and autonomy among heterogeneous, spatially distributed nodes/objects, while providing self-X capabilities (self-configuration, self-healing, self-management, self-optimization).To develop a mission-to-system translation methodology allowing for a systematic decomposition of high-level mission requirements, to low-level system primitives and parameters. The methodology will allow for reusability across various application areas, potentially reducing the effort and the development time.International ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS


Reviewing the middleware for Networked Robotsand WSNsNetworked Robots middleware:Concerning the communication model: MIRO, PEIS Kernel, Orca (CORBA but also non standard models)Concerning the Interoperability of the software components: Marie, Middle LayerConcerning the Automatic Discovery, Configuration and Integration support: PEIS Kernel, UPnPConcerning the Specific/Expandable Services: Sensory Data Processing Middleware, the AWARE data-centric model, MIRO, MarieConcerning the communication performance issues (reliability, availability or QoS): RSCA (Robot Software Communication Architecture)Concerning the support of Low Resources devices: PEIS KernelWSN middleware: Moteview, ScatterViewer, Hourglass, SenseWeb, jWebDust, GSN, TinyDB, Hood, SNACK, Kairos, International ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS


WSAN middleware: Focus on autonomous decision-makingGiven the model of the WSAN system ..Train the system using different scenarios, conclude to AI brains and provide them to the AI agents. An innovative idea we are exploring is the use of Q-Learning algorithms and model the Q as a Model Tree.


Machine LearningClassification:GivenTraining dataLearnA model for making a single prediction or decisionInternational ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALSSource: Lisa Torrey, Univ. of Wisconsin Madison (2009)


Machine LearningProcedural Learning:Learning how to act to accomplish goalsGivenEnvironment that contains rewardsLearnA policy for acting

Important differences from classificationYou dont get examples of correct answersYou have to try things in order to learnInternational ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS


Machine LearningWe dont know all the effects of our decisions so, the procedural learning drive us to the Reinforcement Learning (RL) (acting and observing the environment - or the model of it in our case - ).What kind of RL ? Model-free: learn a policy without a strict model. Use Temporal difference methods (TD).What kind of Model-free RL ? Q-Learning



Q-LearningCurrent state: sCurrent action: a

Transition function: (s, a) = s

Reward function: r(s, a) R

Policy (s) = a

Q(s, a) value of taking action a from state sInternational ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS


Q(s, a)The Q (state, action) table is huge ..

What we could do ? Approximate the Q with a non-linear function.

What kind of function ? A Neural Net or a Model Tree.

Lets test the Model Tree. If its application is successful we could have a much more comprehensive and flexible set of rules for our AI agents.

Model trees are similar to decision trees, but instead of predicting discrete classes, they predict real valued functions. Model trees recursively partition the feature-space into regions, by choosing one attribute as a split criterion at every level of the tree. In the leaves of the trees, regression models are trained to predict a numerical value from the features of an instance.

What algorithm ?

A variant of the Quinlan's M5 algorithm (Quinlan 1992). This algorithm first grows a tree by selecting splitting criteria so as to minimize the target variable's variance, and then builds linear regression models in the nodes. Finally the tree is pruned, which means that sub-trees are replaced with leaves, as long as the prediction error of the resulting tree does not exceed a certain threshold.



Decision Trees (ID3)International ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS


M5 algorithmExampleInternational ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS


The problems ..The main disadvantage in using model trees for value function approximation is that there is currently no algorithm for online training. To refine the predictions of a model tree, we must therefore rebuild it from scratch, using not only the new training examples, but also the old ones, that were used for the previous model trees, i.e. a new model tree approximation of the Q-functions (more than one because of the hierarchical model trees) has to be built from the whole updated training set, and these trees are then used to play the next set of training data. Even though there is not experience with using model trees in reinforcement learning we believe that it is a promising approach.International ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS


Next steps ..To prescribe the components of the distributed middleware, which shall enable cooperation, intelligence and autonomy among heterogeneous, spatially distributed nodes/objects.To develop a mission-to-system translation methodology allowing for a systematic decomposition of high-level mission requirements, to low-level system primitives and parameters.If the experiments of the proposed approach are successful, the methodology could be based on hierarchical Model Trees generated AI agents, trained for a number of scenarios.International ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS


Thank you!International ConferenceNANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS


****

International Conference “NANOSENSORY SYSTEMS AND QUANTUM SENSORY MATERIALS”

Documents

smart systems

management systems

data acquisition systems

complex distributed

internetconnected objects

internet of things iot

quantum sensory materialsd

critical infrastructures