iCognos: Cognitive Stimulation through te

iCognos:Cognitive stimulation through task-oriented telerobotics

Sergi TorrellasBarcelona, June 2013

Advisor: Gregrorio Robledo

Table of contents

Introduction Context of the domain System components iCognos description

Communication Interfaces Cognitive Stimulation Control

System Cognitive Stimulation Telerobotics

System Analysis of the validation Conclusions Future Work

Table of contents

Introduction Context of the domain System components iCognos description

Communication Interfaces Cognitive Stimulation Control

System Cognitive Stimulation Telerobotics

System Analysis of the validation Conclusions Future Work

Introduction (I)

Human cognition relies on cognitive abilities, skills to carry out any task from the simplest to the most complex

The Bloom’s Taxonomy, created in 1956, is a formal description for the cognitive skills

Cognitive skills are combined to acoomplish the Adaptive behaviour

Adaptive behavior enable individuals to perform real life acitivities boosting independence, dignity, and autonomy

Introduction (II)

Brain diseases (e.g. psychiatric, psychological and cognitive) account for a 35% of the total burden of all diseases in Europe

Sources of impairment are diverse

Age-related like Dementia People surviving stroke Brain injury from car accidents Children with learning disabilities

Source: Cost of disorders of the Brain in Europe, European Journal of Neurology, Volume 12, September 2005

… but affectations remain the same in most occasions

Introduction(III)

Motivation is key for effectiveness of therapies Difficult to keep high Influences drastically on

performance

iCognos will improve motivation of end-users through the application of SmartObjects in cognitive stimulation

therapies

Objectives

iCOGNOS-OBJ-001Breakthrough on the traditional cognitive stimulation incorporating SmartObjects

iCOGNOS-OBJ-002Improvement of motivation of the end-users incorporating ecologic cognition tasks

iCOGNOS-OBJ-003Low-cost and affordable platform enlarging the number of beneficiaries

iCOGNOS-OBJ-004Provide an enhanced user experience accessible for everyone with the novel Hex-O-Spell interface

Table of contents

Introduction Context of the domain System components iCognos description

Communication Interfaces Cognitive Stimulation Control

System Cognitive Stimulation Telerobotics

System Analysis of the validation Conclusions Future Work

Robot-aided Rehabilitation

Personal Assistant Robots are robotic assistants which either help an individual in doing some task or a robot able to assist the user in anything s/he requires

REEM, Q.bo and Paro assistive robotics

• Interacts with environment and people• Provision assistance in everyday life activities• Few of them provide cognitive stimulation

Open-source Hardware

Open Source Hardware (OSHW) is a term for tangible artifacts whose design has been released to the public in such a way that anyone can make, modify, distribute, and use those things

• OSWH facilitates hardware customisation

• OSHW reduces the cost of hardware design

• OSHW contributes knowledge sharing

OSHW principles are applied to a wide range of projects from low-level electronics to high-performance applications

Table of contents

Introduction Context of the domain System components iCognos description

Communication Interfaces Cognitive Stimulation Control System Cognitive Stimulation Telerobotics

System Analysis of the validation Conclusions Future Work

Raspberry PI (I)

Raspberry PI is a credit-card-sized single-board computer with a Broadcom BCM2835 system on a chip (SoC) with an ARM processor Raspberry PI contains an ARM-based processor which makes it to operate on just 5V

R-PI Processor architectureR-PI Board Components

Raspberry PI (III)

DistributionsThe Raspberry PI is a single-board computer with resources enough to run a full operative Operating System

The Raspberry PI Foundation provides several GNU/Linux ARM distributions for download

Debian Wheezy

Official Debian distribution ported for the -R-PI Hardware

Raspbian

GNU/Linux distribution based on Debian taking advantage of the

hard-float feature

ePuck: The educational robot (I)

The robot was Created by École Polytechnique Fédérale de Lausanne (EPFL)

The Price of the appliance is between 450 and 550 €

Applied in multiple fields such as Automatic control, Signal processing or Distributed intelligent systems

ePuck robot is a low-cost robot for educational and research following OSHW principles. Similarly, the e-puck software is fully open source, providing low level access

Analog/Digital Module Bluetooth Module Camera Module Codec Module

ePuck: The educational robot (III)

ePuck hardware is published under OSHW but also software libraries are published under a OSS license

ePuck software libraries enable access and control over the different services of the robot:

FFT Module I2C Module library Motor/led Module UART Module

e-puck software libraries are published under the OSS license GNU GPL v3

Wii Remote

Wii Remote is the Wii's main input device for the Nintendo’s Wii video-game console

Built around a Broadcom BCM2042 Contains multiple peripherals that provide data to it Uses (and, at times, abuses) the standard Bluetooth HID protocol to communicate with the host

Qt framework

Qt is a cross-platform application framework that is widely used for developing application software with a graphical user interface (GUI)

Qt is available under a OSS licenses GPL v3 and LGPL v2 Qt runs on the major desktop platforms and some of the mobile platforms Qt also includes non-GUI feature such as SQL database access, XML parsing, thread management, network support

Table of contents

Introduction Context of the domain System components iCognos description

Communication Interfaces Cognitive Stimulation Control System Cognitive Stimulation Telerobotics

System Analysis of the validation Conclusions Future Work

iCognos Description

iCognos has been developed upon different OSS project which have dramatically facilitated the development:

Qt Project is an a software framework developed in C++ mainly used to facilitate the development of UI

Cwiid is an open-source software package which enables the connection with the Nintendo’s WiiMote

Raspbian is an open source project to create a GNU/Linux based distribution for the R-PI

e-puck Libraries is a software project for the e-puck robot that abstracts the user from the low-level development

iCognos Description

General overviewiCognos provides a new concept in the cognitive stimulation therapies incorporating the principles of ecologic cognition

SmartObjects are devices capable of understanding the environment and react accordingly

iCognos is composed of two different appliances

CSCS : Cognitive Stimulation Control System CSTS: Cognitive Stimulation Telerobotics System

iCognos Description : CSCS (I)

CSCS development might be realised within the R-PI itself but limited number of resources discard this choice

The alternative is to develop the code in a Development station lately executed by the R-PI with a cross-compilation scheme

A Cross-compiler is a software program able of creating binary code for a architecture other than itself

iCognos Description : CSCS (II)

CSCS is to provide the UI to guide the user and maintain the proper communication between all the devices: the Input Device and the CSTS

CwiidCommunicator holds the communication with the Input Controller

ePuckCommunicator maintains the connection with the ePuck implementing the necessary functions to send and receive commands

EventDispatcher connects the events from the different modules

hosContainer handles the events of the Hex-O-Spell

VisualInterface shapes the interface for the user

iCognos Description : CSTS (I)

The ePuck robot contains at its heart a dsPIC microcontroller which is the piece of hardware able to be programmed with a dedicated software (i.e. a firmware)

Microchip, dsPIC manufacturer, provides a software for development of firmware, called MPLAB IDE, a free, integrated toolset for the development of embedded applications

iCognos Description : CSTS (II)

CSTS is a hardware platform responsible for executing the commands issued by the CSCS to perform the requested task

Architectural diagram

Flow diagram

iCognos Description

iCognos appearance

CSTS

CSCS

Input Control

See demonstration video here

Table of contents

Introduction Context of the domain System components iCognos description

Communication Interfaces Cognitive Stimulation Control System Cognitive Stimulation Telerobotics

System Analysis of the validation Conclusions Future Work

Validation Analysis (I)

The validation was performed with a common set-up

Different profile were selected Thesis supervisor Researcher in rehabilitation Journalist

Collected feedback classified Concept Usability and Accessibility Improvements

Validation Analysis (II)

Concept The system might include algorithms to understand the interaction of the user iCognos is nearer than a serious game to the activities of daily life The concept is very attractive for everyone

Usability and accessibility Limitated by the usage of a single interface HoS interface is not the best interface Exploit the possibilities offered by the WiiMote

Improvements Usage of more features from the robotic platforms Define levels in the cognitive tasks Include instructions for every task to be performed

Table of contents

Introduction Context of the domain System components iCognos description

Communication Interfaces Cognitive Stimulation Control System Cognitive Stimulation Telerobotics

System Analysis of the validation Conclusions Future Work

Conclusions

Prototype completed Cognitive Rehabilitation platform Robotic platform is attractive Open-Source and Open-source Hardware

Different OS projects working in concert Raspberry PI ePuck WiiMote – Cwiid

Acquired competences Embedded system programming GNU/Linux for low-level software Usage of OSS libraries

iCognos relies on different projects

Future work

Deeper study on designing cognitive stimulation tasks Contact therapists Include end-user from the conception phase

Features of Robotic Platform Sensors: Microphone, 3D Accelerometer, Camera, IR sensors… Actuators: Speaker, Arms…

Add data mining capabilities Personalisation of therapies Study evolution of disability

iCognos to market: Professional requirements and objectives Real End-User testing Implement feedback Product / Marketing

Thanks for your attention