Shyamanta M. Hazarika, Adity Saikia, Simanta Bordoloi, Ujjal Sharma And Nayantara Kotoky Department Of Computer Sc. & Engineering, Tezpur University Tezpur,

Shyamanta M. Hazarika, Adity Saikia, Simanta Bordoloi, Ujjal Sharma And Nayantara Kotoky

Department Of Computer Sc. & Engineering, Tezpur University

Tezpur, India

[email protected]

Brain Computer Interface as Sensor for Ambient Intelligent

Living: A Position Paper

Biomimetic and Cognitive Robotics @ TU

BCR@TU conducts research in the area of Cognitive Robotics and Knowledge Representation & Reasoning. We are particularly interested in Qualitative Spatial and Temporal Reasoning. This translates into interest in Cognitive Vision and Rehabilitation Robotics.

Our research within Cognitive Robotics and KR &R is driven by biomimetics i.e., examination of nature particularly human intelligence and skills, its models, systems, processes, and elements to emulate or take inspiration from these designs and processes.

For development of prostheses and assistive devices within Rehabilitation Robotics we undertake biomimetic design, which is NOT JUST A COPY of the geometry! For us biomimetic design is biomimetic geometry together with functional biomimesis.

Brain Computer Interfaces

Brain computer interfaces – Use computers to sense human

thoughts and enable the users to control external devices

– Infer a user’s intentions using only brain activity

– Provide a non-muscular avenue for communication

Applications– BCIs are aimed at assisting,

augmenting, or repairing human cognitive or sensory-motor functions. e.g. locked-in syndrome (cognitively unimpaired, but no motor control)

•

Brain Computer Interfaces• Depending on application, BCI can be classified

as • Cognitive• Sensory• Motor

• Motor BMI seeks to translate brain activity from the central or peripheral nervous system into useful commands to external devices.

• Drive Prosthetics • Functional electrical stimulation

• Motor BMI can be categorized as • Invasive• Partially Invasive• Non-Invasive

What is an EEG?An electroencephalogram is a measure of the brain's voltage fluctuations as detected from scalp electrodes. It is an approximation of the cumulative electrical activity of neurons.

• Brain– set of interconnected modules– performs information processing

operations at various levels• sensory input analysis• memory storage and retrieval• reasoning• feelings• consciousness

• Neurons– basic computational elements– heavily interconnected with other

neurons

Beta Rhythm Alpha & Mu Rhythm

Grounding

Electrode PlacementStandard 10:20 System

Experiment Protocol

Bispectrum of EEG Signal• Bispectrum is the expectation of three

frequencies; two direct frequency components and the third the conjugate frequency of the sum of those two frequencies.

• Knowing the Fourier frequency components X(f) the bispectrum B(f1, f2) can be estimated using the Fourier-Stieltjes representation.

B(f1, f2) = E(X(f1)X(f2)X*(f1+ f2)) Where X*(f) is the complex conjugate of X(f) and E( ) is the statistical expectation operator

Bispectrum Analysis

Bispectrum Analysis

• Bispectrum analysis provide a way to evaluate mental representation during observation and imagination of hand movement

• Prior visual representation of motor acts make difference during motor imagination.

Another experiment • Aim is to classify four different motor imagery,

namely,– Both Hands Up– Tighten Both Fists– Left Hand Up– Right Hand Up

• The Protocol

Start Audio Cue Action Audio Cue Stop Audio Cue

Relax and keep your eyes closed.

Imagine

the action.End the task

and relax.

The Architecture

EEG Unit Noise & Amplitude

Normalization

Feature Extraction

UnitK-fold cross validation SVM

Filtration & Normalization Unit

Motor Imagery

Types

Classification Unit

Hybrid Features of Bispectrum• Here we do not make use of the bispectrum feature

directly rather we use following two hybrid features of bispectrum in order to retain the temporal as well as frequency information within the EEG data.• Sum of Logarithmic Amplitudes (SLA) to characterizes temporal

bispectral information.

θ gives the principal domain.

• First Order Spectral Moment (FOSM) to characterizes frequency information of the bispectrum.

N is the number of diagonal elements of Bispectrum

Figure : Bispectrum Estimation of the EEG Signals. Top-left: left hand motor imagery; top-right: right hand motor imagery;

bottom-left: both hands motor imagery & bottom-right: both fists motor imagery.

Bispectrum Analysis

Classification• We have used RBF kernel

SVM for classification of the MIs.

• The original SVM algorithm was proposed by Vladimir Vapnik in 1970.

• The result is cross-validated through 10-Fold Cross Validation.

Confusion Matrix

BCI Based Maze Game

With an aim of developing a non-invasive BCI to be used as an intelligent assistive system, we have designed and developed a simple maze game, where a player plays the game in real time by using his brain signals.

Mapping of Motor Imageries with Game Moves

Motor Imagery Game Move

Both Hands Up Move Forward

Right Hand Up Move Right

Left Hand Up Move Left

Tight Both Fists Move Backward

What we have done so far?

BCI Integrated Collaborative Control

• The idea is to integrate a BCI with a cognitive architecture for collaborative control of a smart wheelchair.

• The cognitive architecture mediates based on the extent of automatic vs. manual control to be achieved.

• AIM…– To help people with mobility disability (with or without cognitive

impairment) to achieve a level of independence so that carryout their daily activities.

BCI Integrated Collaborative Control Architecture

Automatic control Module

Adaptation Module Mediator Manual control

Module

Sensing Control

Sensor Role Actor Role

Brain Computer Interface

Assistive DeviceIntelligent/Smart Wheelchair

BCI Integrated Collaborative Control Architecture

• Three layered control architecture – BCI; Superior Control and Local Control.

• The BCI plays a dual role that of an actor as well as a sensor.

• It not only does provide control commands to drive the wheelchair but also monitor the cognitive state of the user - his confidence, cognitive workload and wellbeing, depending on which BCI could provide assistance range from partial control of navigation to complete autonomous mode .

Final Comments• Over the years AI has drifted away from its main

aim. This work is an attempt to focus on integrated systems rather than component algorithms.

• The cognitive systems paradigm needs to have its source of ideas in human cognition. This position paper describes work done at the Biomimetic and Cognitive Robotics Lab at Tezpur University for development of a BCI Integrated Collaborative Controller for an intelligent wheelchair.