Modified report

Eye controlled HMI

CHAPTER 1

INTRODUCTION TO EYE-HMI

Bio-based human computer interface (HCI) has the potential to enable severely

disabled people to drive computers directly by bioelectricity rather than by physical

means. A study on the group of persons with severe disabilities shows that many of them

have the ability to control their eye movements, which could be used to develop new

human computer interface systems to help them communicate with other persons or

control some special instruments.

Furthermore, this application of EOG-based HCI could be extended to the group

of normal persons for game or other entertainments. Nowadays, some methods which

attain user’s eye movements are developed.

In this project our objective is to design a Human Machine interface, which can be

controlled using EOG Signals and final output is to be used to move cursor on the

Graphic Display which has several buttons and each button on clicking by blinking of

eyes activated corresponding appliance or action. We will provide RF interface between

acquisition/processing part and application so that it’s easy to handle and easy to install in

homes and hospitals.

ECE, SBMSIT 1

Eye controlled HMI

ELECTRO-OCULOGRAPHY (EOG) PRINCIPLE

Electro-oculography (EOG) is a new technology of placing electrodes on user’s

forehead around the eyes to record eye movements. EOG is a very small electrical

potential that can be detected using electrodes. Compared with the EEG, EOG signals

have the characteristics as follows: the amplitude is relatively high (15-200uV), the

relationship between EOG and eye movements is linear, and the waveform is easy to

detect. Considering the characteristics of EOG mentioned above, EOG based HCI is

becoming the hotspot of bio-based HCI research in recent years.

Basically EOG is a bio-electrical skin potential measured around the eyes but first we have to understand eye itself:

Anatomy of the Eye

The main features visible at the front of the eye are shown in Figure .The lens,

directly behind the pupil, focuses light coming in through the opening in the centre of the

eye, the pupil, onto the light sensitive tissue at the back of the eye, the retina. The iris is

the coloured part of the eye and it controls the amount of light that can enter the eye by

changing the size of the pupil, contracting the pupil in bright light and expanding the

ECE, SBMSIT 2

Eye controlled HMI

pupil in darker conditions. The pupil has very different reflectance properties than the

surrounding iris and usually appears black in normal lighting conditions. Light rays

entering through the pupil first pass through the cornea, the clear tissue covering the front

of the eye. The cornea and vitreous fluid in the eye bend and refract this light. The

conjuctiva is a membrane that lines the eyelids and covers the sclera, the white part of the

eye. The boundary between the iris and the sclera is known as the limbus, and is often

used in eye tracking.

The light rays falling on the retina cause chemical changes in the photosensitive

cells of the retina. These cells convert the light rays to electrical impulses which are

transmitted to the brain via the optic nerve. There are two types of photosensitive cells in

the retina, cones and rods. The rods are extremely sensitive to light allowing the eye to

respond to light in dimly lit environments. They do not distinguish between colours,

however, and have low visual acuity, or attention to detail. The cones are much less

responsive to light but have a much higher visual acuity. Different cones respond to

different wavelengths of light, enabling colour vision. The fovea is an area of the retina of

particular importance. It is a dip in the retina directly opposite the lens and is densely

packed with cone cells, allowing humans to see fine detail, such as small print. The

human eye is capable of moving in a number of different manners to observe, read or

examine the world in front of them.

The Electrooculogram

The electrooculogram (EOG) is the electrical signal produced by the potential

difference between the retina and the cornea of the eye. This difference is due to the large

presence of electrically active nerves in the retina compared to the front of the eye. Many

experiments show that the corneal part is a positive pole and the retina part is a negative

pole in the eyeball. Eye movement will respectively generates voltage up to 16uV and

14uV per 1° in horizontal and vertical way. The typical EOG waveforms generated by

eye movements are shown in Figure 1.

In Figure 1, positive or negative pulses will be generated when the eyes rolling

upward or downward. The amplitude of pulse will be increased with the increment of

ECE, SBMSIT 3

Eye controlled HMI

rolling angle, and the width of the positive (negative) pulse is proportional to the duration

of the eyeball rolling process.

METHODOLOGY

In our HCI system, four to five electrodes are employed to attain the EOG signals.

Figure 2 shows the electrode placement.

ECE, SBMSIT 4

Eye controlled HMI

1 & 4 for detecting vertical movement

2 & 3 for detecting horizontal movement

5 is for reference(can be omitted or place at

forehead).

Blink detection is by separate algorithm based on

EOG signals

Figure 2: electrode placements

Advantages of the EOG over other methods

The visual systems mentioned in our projet offer robust methods of eye tracking,

usually with very good accuracy. While in certain circumstances, visual methods may be

more appropriate, the electrooculogram offers a number of advantages. Some of the

reasons for favouring the EOG over other options

• Range

The EOG typically has a larger range than visual methods which are constrained

for large vertical rotations where the cornea and iris tend to disappear behind the eyelid.

Angular deviations of up to 80◦ can be recorded along both the horizontal and vertical

planes of rotation using electrooculography.

• Linearity

The reflective properties of ocular structures used to calculate eye position in

visual methods are linear only for a restricted range, compared to the EOG where the

ECE, SBMSIT 5

Eye controlled HMI

voltage difference is essentially linearly related to the angle of gaze for ±30◦ and to the

sine of the angle for ±30◦ to ±60◦

• Head Movements are Permissible

The EOG has the advantage that the signal recorded is the actual eyeball position

with respect to the head. Thus for systems designed to measure relative eyeball position

to control switches (e.g. looking up, down, left and right could translate to four separate

switch presses) head movements will not hinder accurate recording.

• Non-invasive

Unlike techniques such as the magnetic search coil technique, EOG recordings do

not require anything to be fixed to the eye which might cause discomfort or interfere with

normal vision. EOG recording only requires three electrodes (for one channel recording),

or five electrodes (for two channel recording), which are affixed externally to the skin.

• Obstacles in front of the eye

In visual methods, measurements may be interfered with by scratches on the

cornea or by contact lenses. Bifocal glasses and hard contact lenses seem to cause

particular problems for these systems. EOG measurements are not affected by these

obstacles.

• Cost

EOG based recordings are typically cheaper than visual methods, as they can be

made with some relatively inexpensive electrodes, some form of data acquisition card and

appropriate software.

ECE, SBMSIT 6

Eye controlled HMI

• Lighting Conditions

Variable lighting conditions may make some of the visual systems unsuitable or at

least require re-calibration when the user moves between different environments. One

such scenario which could pose problems is where the eye tracking system is attached to

a user.

• Eye Closure is Permissible

The EOG is commonly used to record eye movement patterns when the eye is

closed, for example during sleep. Visual methods require the eye to remain open to know

where the eye is positioned relative to the head, whereas an attenuated version of the

EOG signal is still present when the eye is closed.

• Real-Time

The EOG can be used in real-time as the EOG signal responds instantaneously to

a change in eye position and the eye position can be quickly inferred from the change.

The EOG is linear up to 30◦.

Limitations of EOG-Based Eye tracking

The measured EOG voltage varies for two reasons. Either the eye moves (which

we want to record), or baseline drift occurs (which we want to ignore). Baseline drift

occurs due to the following factors:

ECE, SBMSIT 7

Eye controlled HMI

• Lighting Conditions

The DC level of the EOG signal varies with lighting conditions over long periods

of time. When the source of the light entering the eye changes from dark conditions to

room lighting, as per studies it can take anywhere from between 29-52 minutes for the

measured potential to stabilise to within 10% of the baseline, and anywhere between 17-

51 minutes when the transition is from room lighting to darkness.

• Electrode Contact

The baseline may vary due to the spontaneous movement of ions between the skin

and the electrode used to pick up the EOG voltage. The mostly commonly used electrode

type is silver-silver chloride (Ag-AgCl). Large DC potentials of up to 50mV can develop

across a pair of Ag-AgCl electrodes in the absence of any bioelectric event, due to

differences in the properties of the two electrode surfaces with respect to the electrolytic

conduction gel. The extent of the ion movement is related to a number of variables

including the state of the electrode gel used, variables in the subject’s skin and the

strength of the contact between the skin and the electrode. Proper preparation of the skin

is necessary to maximize conduction between the skin and the conduction gel, usually by

brushing the skin with alcohol to remove facial oils.

• Artifacts due to EMG or Changes in Skin Potential

The baseline signal may change due to interference from other bioelectrical

signals in the body, such as the electromyogram (EMG) or the skin potential. EMG

activity arises from movement of the muscles close to the eyes, for example if the subject

frowns or speaks. These signals may be effectively rejected by careful positioning of the

ECE, SBMSIT 8

Eye controlled HMI

electrodes and through low pass filtering the signal. Skin potential changes due to

sweating oremotional anxiety pose a more serious problem.

• Age and Sex

sex have a significant effect on baseline voltage levels, although this should not

pose a problem if a system is calibrated initially for each particular user.

• Diurnal Variations

The baseline potential possibly varies throughout the day. Manual calibration is

often used to compensate for DC drift - the subject shifts his gaze between points of

known visual angle and the amplifier is balanced until one achieves the desired

relationship between voltage output and degree of eye rotation. With frequent re-

calibration, accuracies of up to ±30′ can be obtained. While manual calibration may be

acceptable practice in clinical tests that use the EOG, this restriction hinders the EOG

from being used independently as a control and communication tool by people with

disabilities.

ECE, SBMSIT 9

Eye controlled HMI

BLOCK DIAGRAM AND DESCRIPTION

Block Diagram of Eye controller Human Machine interface:

ECE, SBMSIT 10

Eye Movements

output

RF Interface RF RX Module

Application Part

Display Screen and

Device controller

Devices to be controlled by

eye movements

Graphics LCD

(MENU)

RF Interface RF TX Module

Acquisition and Processing Part

Electrodes near Eye to sense signal from eyes

EOG signals

Instrumentation AmplifierAnd active

filter

A/D Convertor

Embedded ControllerFor data

acquisition using I2C ADC

Processing for Eye Movement and Eye Blink

detections

Eye controlled HMI

DESCRIPTION

Acquisition Part

For four-five different electrodes two separate acquisition electronics is required

Electrodes:

First thing to interface to body is Electrodes here we are using reusable electrodes

to connect electronics with human body and these electrodes will pickup signals which

corresponds to eye movements signals mixed with some others signals which are noise

for us.

We are going to use Ag-AgCl electrodes as they are low cost and easily available.

Instrumentation Amplifier:

Signals from electrodes are received and sent to Instrumentation Amplifier.

An instrumentation amplifier is a type of differential amplifier that has been outfitted with

input buffers, which eliminate the need for input impedance matching and thus make the

amplifier particularly suitable for use in measurement and test equipment. Additional

characteristics include very low DC offset, low drift, low noise, very high open-loop gain,

very high common-mode rejection ratio, and very high input impedances. Instrumentation

amplifiers are used where great accuracy and stability of the circuit both short- and long-

term are required.

We are using AD620 which is precision Instrumentation amplifier.

ECE, SBMSIT 11

Eye controlled HMI

Active Filters and Gain Blocks:

Opamp based Active filters are used we have low pass filter so that only eye

signals are going future in the circuit, cutoff frequency for this filter is 20Hz-40Hz. And

high pass filter to block DC and frequencies upto 0.1-0.3Hz. These filters and gain blocks

are implemented using LM324 Opamp.

Analog to Digital Convertor:

Final amplified and filtered analog output is converted into Digital signal using

I2C Based 4 channel A2D convertor-PCF8591 to save space as ADC0808 is little bigger

in size.

Acquisition and processing microcontroller:

This is 8051 class of microcontroller and it has to acquire signals from A/D

convertor for both chains up-down electrode chain and left-right electrode chain. As our

microcontroller is fast and powerful we will process the signal here itself and transmit

final eye move outputs to application part wirelessly.

Cmds sent:

01– CL: Right eye movement

02– CR: Left eye movement

03– CU: Up eye movement

04– CD: Down eye movement

05– BL: Blinking of eye

RF Transmitter:

Here we can use 315/433Mhz Tx modules along with HT640 Encoder to send eye

movement commands to the application part.

ECE, SBMSIT 12

Eye controlled HMI

Application Part

RF Receiver:

Wireless signals transmitted by our acquisition part are received in this section,

here we use 315/433Mhz Rx modules along with HT648 decoder. Output of RF receiver

goes to application part directly.

Display and appliance controller:

This is a again a microcontroller which receives eye movements signals (R L U D

B) as described above via UART interface. We are using P89V51RD2 from NXP

(Philips), this microcontroller is connected to Graphic LCD which is displaying Cursor

and 4 buttons

1. TV

2. FAN

3. Lights

4. Alarm

Using eye movements a cursor is controlled and using blink click operation is

done, each button is toggle button i.e. if appliance is on it will become off and vice versa.

But alarm button is different when clicked a On-off alarm is generated to call assistance.

And assistant has to come and reset the alarm.

Now this controller is also connected to relay board so button action is converted

into relays getting switch off and on. And hence appliances are getting turned on and off.

ECE, SBMSIT 13

Eye controlled HMI

Many other applications are also possible like Computer Mouse interface, virtual

keyboard interface so disable can talk via this keyboard and send mails.

ECE, SBMSIT 14

Eye controlled HMI

BIO-POTENTIALS & ELECTRODES

Biopotentials

An electric potential that is measured between points in living cells, tissues, and

organisms, and which accompanies all biochemical processes. Also describes the transfer

of information between and within cells

ECE, SBMSIT 15

Eye controlled HMI

Mechanism behind Bio Potentials

• Concentration of potassium (K+) ions is 30-50 times higher inside as compared to

outside

• Sodium ion (Na+) concentration is 10 times higher outside the membrane than

inside

• In resting state the member is permeable only for potassium ions

Potassium flows outwards leaving an equal number of negative ions inside

Electrostatic attraction pulls potassium and chloride ions close to the

membrane

Electric field directed inward forms

Electrostatic force vs. diffusional force

Different Types of potentials are discussed here

The Membrane Potential

A potential difference usually exists between the inside and outside of any cell

membrane, including the neuron. The membrane potential of a cell usually refers to the

potential of the inside of the cell relative to the outside of the cell i.e. the extracellular

fluid surrounding the cell is taken to be at zero potential. When no external triggers are

acting on a cell, the cell is described as being in its resting state. A human nerve or

skeletal muscle cell has a resting potential of between -55mV and -100mV . This

potential difference arises from a difference in concentration of the ions K+ and Na+

inside and outside the cell. The selectively permeable cell membrane allows K+ ions to

pass through but blocks Na+ ions. A mechanism known as the ATPase pump pumps only

two K+ ions into the cell for every three Na+ cells pumped out of the cell resulting in the

outside of the cell being more positive than the inside. The origin of the resting potential

is explained in further detail in.

The Action Potential

ECE, SBMSIT 16

Eye controlled HMI

As mentioned already, the function of the nerve cell is to transmit information

throughout the body. A neuron is an excitable cell which may be activated by a stimulus.

The neuron’s dendrites are its stimulus receptors. If the stimulus is sufficient to cause the

cell membrane to be depolarised beyond the gate threshold potential, then an electrical

discharge of the cell will be triggered. This produces an electrical pulse called the action

potential or nerve impulse. The action potential is a sequence of depolarisation and

repolarisation of the cell membrane generated by a Na+ current into the cell followed by a

K+ current out of the cell. The stages of an action potential are shown in Figure

Figure 3.4: An Action Potential. This graph shows the change in membrane potential

as a function of time when an action potential is elicited by a stimulus.

• Stage 1 – Activation

When the dendrites receive an “activation stimulus” the Na+ channels begin to

open and the Na+ concentration inside the cell increases, making the inside of the cell

more positive. Once the membrane potential is raised past a threshold (typically around -

50mV), an action potential occurs.

• Stage 2 – Depolarisation

ECE, SBMSIT 17

Eye controlled HMI

As more Na+ channels open, more Na+ ions enter the cell and the inside of the

cell membrane rapidly loses its negative charge. This stage is also known as the rising

phase of the action potential. It typically lasts 0.2 - 0.5ms.

• Stage 3 – Overshoot

The inside of the cell eventually becomes positve relative to the outside of the

cell. The positive portion of the action potential is known as the overshoot.

• Stage 4 – Repolarisation

The Na+ channels close and the K+ channels open. The cell membrane begins to

repolarise towards the resting potential.

• Stage 5 – Hyperpolarisation

The membrane potential may temporarily become even more negative than the

resting potential. This is to prevent the neuron from responding to another stimulus during

this time, or at least to raise the threshold for any new stimulus.

• Stage 6

The membrane returns to its resting potential.

Propagation of the Action Potential

An action potential in a cell membrane is triggered by an initial stimulus to the

neuron. That action potential provides the stimulus for a neighbouring segment of cell

membrane and so on until the neuron’s axon is reached. The action potential then

propagates down the axon, or nerve fibre, by successive stimulation of sections of the

axon membrane. Because an action potential is an all-or-nothing reaction, once the gate

ECE, SBMSIT 18

Eye controlled HMI

threshold is reached, the amplitude of the action potential will be constant along the path

of propagation. The speed, or conduction velocity, at which the action potential travels

down the nerve fibre depends on a number of factors, including the initial resting

potential of the cell, the nerve fibre diameter and also whether or not the nerve fibre is

myelinated. Myelinated nerve fibres have a faster conduction velocity as the action

potential jumps between the nodes of Ranvier.

Synaptic Transmission

The action potential propagates along the axon until it reaches the axonal ending.

From there, the action potential is transmitted to another cell, which may be another nerve

cell, a glandular cell or a muscle cell. The junction of the axonal ending with another cell

is called a synapse. The action potential is usually transmitted to the next cell through a

chemical process at the synapse.

Resting potential:

Nerve and muscle cells are encased in a semi-permeable membrane that permits

selected substances to pass through while others are kept out. Body fluids surrounding

cells are conductive solutions containing charged atoms known as ions. In their resting

state, membranes of excitable cells readily permit the entry of K+ and Cl- ions, but

effectively block the entry of Nu+ ions (the permeability for K+ is 50-100 times that for

Na+). Various ions seek to establish a balance between the inside and the outside of a cell

according to charge and concentration. The inability of Nu+ to penetrate a cell membrane

results in the polarization that is called as Resting Potential.

ECE, SBMSIT 19

Eye controlled HMI

EOG ELECTRODES

Because of the very low amplitude of the EOG, the electrodes represent the

weakest link in the entire recording system. The following properties are desirable in an

EOG electrode:

(a) Stable electrode potential: Spontaneous fluctuations of only 2 or 3mV in the potential

difference between an electrode and the surrounding electrolyte will produce artifacts

very much larger than the EOG.

(b) Equal electrode potentials: A small standing potential difference between a pair of

electrodes will not present major difficulties, apart from producing a temporary deflection

of the trace and possibly blocking of the amplifiers when the electrodes are first

connected to the recorder. However, if the current flow between the electrode varies

owing to changing contact resistances, artifact may result, As it is in practice never

possible to ensure that conventional electrodes are of equal potential, it follows that a

third desirable characteristic is constant electrode contact resistances

(c) Equal electrode resistances: EOG recording is bedeviled by electrical interference -

particularly from ac mains; there are generally unwanted changes in potential difference

between the subject and the ECG machine that are seen as common mode signals and can

he rejected by the use of differential amplifiers. Unequal electrode resistances, however,

unbalance the system and produce an out-of-phase component that will appear in the

tracing.

(d) Low electrode resistance: With modern amplifier design, it is now easy to ensure that

the electrode resistances are very much less than the input impedance so that as much as

possible of the ECG signal is applied at the input of the amplifier. The effects of unequal

electrode resistances are less marked when the actual values are low. In general when the

other criteria above are satisfied, the electrode resistance is to be less than 5k and

measurement of resistance provides a good check on the quality of electrode preparation

and application.

ECE, SBMSIT 20

Eye controlled HMI

The desirable characteristics above can generally be satisfied by the use of

nonpolarisable electrodes, so far as identical physical and chemical structure, securely

attached to skin that has first been cleaned and abraded to remove the outer layer which is

of high resistance.

TYPES OF ELECTRODES

Ag-AgCl electrodes and disposable electrodes were used when the data was

recorded from the frontal region. These electrode types that are used were shown in fig.

Figure: Different types of electrodes

ECE, SBMSIT 21

Eye controlled HMI

BIO-POTENTIAL AMPLIFIERS

Bio signals are recorded as potentials, voltages, and electrical field strengths

generated by nerves and muscles. The measurements involve voltages at very low levels,

typically ranging between 1 μV and 100 mV, with high source impedances and

superimposed high level interference signals and noise. The signals need to be amplified

to make them compatible with devices such as displays, recorders, or A/D converters for

computerized equipment. Amplifiers adequate to measure these signals have to satisfy

very specific requirements. They have to provide amplification selective to the

physiological signal, reject superimposed noise and interference signals, and guarantee

protection from damages through voltage and current surges for both patient and

electronic equipment. Amplifiers featuring these specifications are known as biopotential

amplifiers

. Basic requirements and features, as well as some specialized systems,

The basic requirements that a biopotential amplifier has to satisfy are:

the physiological process to be monitored should not be influenced in any way by

the amplifier

the measured signal should not be distorted

the amplifier should provide the best possible separation of signal and

interferences

the amplifier has to offer protection of the patient from any hazard of electrical

shock

the amplifier itself has to be protected against damages that might result from

high input voltages as they occur during the application of defibrillators or

electrosurgical instrumentation

A typical configuration for the measurement of bio potentials is shown in figure.

Three electrodes, two of them are picking up the biological signal and the third providing

the reference potential, connect the subject to the amplifier. The input signal to the

ECE, SBMSIT 22

Eye controlled HMI

amplifier consists of five components: (1) the desired biopotential, (2) undesired

biopotentials, (3) a power line interference signal of 50 Hz and its harmonics, (4)

interference signals generated by the tissue/electrode interface, and (5) noise. Proper

design of the amplifier provides rejection of a large portion of the signal interferences.

The main task of the differential amplifier as shown in Figure is to reject the line

frequency interference that is electrostatically or magnetically coupled into the subject.

The desired biopotential appears as a voltage between the two input terminals of the

differential amplifier and is referred to as the differential signal. . The line frequency

interference signal shows only very small differences in amplitude and phase between the

two measuring electrodes, causing approximately the same potential at both inputs, and

thus appears only between the inputs and ground and is called the common mode signal.

Strong rejection of the common mode signal is one of the most important characteristics

of a good biopotential amplifier.

Fig : Typical configuration for the measurement of biopotentials. The biological signal V

appears between the two measuring electrodes at the right and left arm of the patient, and

is fed to the inverting and the non-inverting inputs of the differential amplifier. The right

leg electrode provides the reference potential for the amplifier with a common mode

voltage Vc as indiacted.

common mode rejection ratio

CMRR of an amplifier is defined as the ratio of the differential mode gain over the

common mode gain. The output of a real biopotential amplifier will always consist of the

ECE, SBMSIT 23

Eye controlled HMI

desired output component due to a differential biosignal, an undesired component due to

incomplete rejection of common mode interference signals as a function of CMRR, and

an undesired component due to source impedance unbalance allowing a small proportion

of a common mode signal to appear as a differential signal to the amplifier. Since source

impedance unbalances of 5,000 to 10,000 Ω, mainly caused by electrodes, are not

uncommon, and sufficient rejection of line frequency interferences requires a minimum

CMRR of 100 dB, the input impedance of the amplifier should be at least 109 Ω at 60 Hz

to prevent source impedance unbalances from deteriorating the overall CMRR of the

amplifier. State-of-the-art biopotential amplifiers provide a CMRR of 120 to 140 dB.

In order to provide optimum signal quality and adequate voltage level for further signal

processing, the amplifier has to provide a gain of 100 to 50,000 and needs to maintain the

best possible signal-to noise ratio. The presence of high level interference signals not only

deteriorates the quality of the physiological signals, but also restricts the design of the

biopotential amplifier. In order to prevent the amplifier from going into saturation,

this component has to be eliminated before the required gain can be

provided for the physiological signal.

Fig :Schematic design of the main stages of a biopotential amplifier. Three electrodes

connect the patient

A typical design of the various stages of a biopotential amplifier is shown in above figure.

The electrodes which provide the transition between the ionic flow of currents in

ECE, SBMSIT 24

Eye controlled HMI

biological tissue and the electronic flow of current in the amplifier, represent a complex

electrochemical system. The electrodes determine to a large extent the composition of the

measured signal. The preamplifier represents the most critical part of the amplifier itself

since it sets the stage for the quality of the biosignal. With proper design, the preamplifier

can eliminate, or at least minimize, most of the signals interfering with the measurement

of biopotentials.

Instrumentation Amplifier

An important stage of all biopotential amplifiers is the input preamplifier which

substantially contributes to the overall quality of the system. The main tasks of the

preamplifier are to sense the voltage between two measuring electrodes while rejecting

the common mode signal, and minimizing the effect of electrode polarization over

potentials. Crucial to the performance of the preamplifier is the input impedance which

should be as high as possible. Such a differential amplifier cannot be realized using a

standard single operational amplifier (op-amp) design since this does not provide the

necessary high input impedance. The general solution to the problem involves voltage

followers, or noninverting amplifiers, to attain high

input impedances. A possible realization is shown in figure(a). The main disadvantage of

this circuit is that it requires high CMRR both in the followers and in the final op-amp.

With the input buffers working at unity gain, all the common-mode rejection must be

accomplished in the output amplifier, requiring very precise resistor matching.

Additionally, the noise of the final op-amp is added at a low signal level, decreasing the

signal-to-noise ratio unnecessarily. The circuit in Fig(b) eliminates this disadvantage. It

represents the standard instrumentation amplifier configuration. The two input op-amps

provide high differential gain and unity common-mode gain without the requirement of

close resistor matching.

ECE, SBMSIT 25

Eye controlled HMI

Fig :Circuit drawings for three different realizations of instrumentation amplifiers for

biomedical applications. Voltage follower input stage (a), improved, amplifying input

stage (b) 2 op-amp version (c).

1

21 21

R

RG

3

42 R

RG

The preamplifier, often implemented as a separate device which is placed close to

the electrodes or even directly attached to the electrodes, also acts as an impedance

converter which allows the transmission of even weak signals to the remote monitoring

unit. Due to the low output impedance of the preamplifier, the input impedance of the

ECE, SBMSIT 26

Eye controlled HMI

following amplifier stage can be low, and still the influence of interference signals

coupled into the transmission lines is reduced.

We are using AD620 its specification and details can be found in datasheet.

ECE, SBMSIT 27

Eye controlled HMI

ACQUISITION FRONT END

Electrodes capture the biopotentials from the body but these signals are very

weak and very noisy so there is invariable need of advance acquisition system which

comprises of precision instrumentation amplifier, active filters, multiple gain block and

for interfacing to ADC we have to do dc shifting(or clamping) of signal followed by

clipping to avoid any residual negative voltages.

Circuit is given below:

Figure: Acquisition circuit diagram (same is for up-down and left-right).

ECE, SBMSIT 28

Eye controlled HMI

In above circuit diagram IA AD620 is set for gain of 50, which is followed by low

pass filter and high pass filter together the made band pass filter and we are getting

frequencies 0.5 to 4Hz at output, this is amplified buy gain block 1 which has variable

gain now our signal range is few mv, we need one more gain block followed by active

low pass filter to reject all high frequency noises above 3Hz and some gain also can be

provided if required at this stage. After this we have dc level shifter and clipper ckt to

ensure only positive voltages are going to ADC.

Similar circuit is there for up-down but only we have to adjust the again using

variable resistor and dc level shift voltage. Both the final outputs are fed to I2C based 4

channel ADC PCF8591, left right signal is fed to ch0 and up-down signal is fed to ch1.

ECE, SBMSIT 29

Eye controlled HMI

I2C Based serial ADC PCF8591

Introduction to I2C Protocol:

Name I2C is shorthand for a standard Inter-IC (integrated circuit) bus.

Philips originally developed I2C for communication between devices inside of a

TV set. Examples of simple I2C-compatible devices found in embedded systems include

EEPROMs, thermal sensors, and real-time clocks.

The main objective behind the invention of I2C bus is to establish a simple low

pin count bus that can connect different ICs on a circuit board of Television or Radio.

Later I2C grew beyond the limits of TV and Radio and now it can be found in almost

every computer motherboards and other embedded devices. I2C can also be used for

communication between multiple circuit boards in equipments with or without using a

shielded cable depending on the distance and speed of data transfer.

Standard I2C devices operate up to 100Kbps, while fast-mode devices operate at

up to 400Kbps. A 1998 revision of the I2C specification (v. 2.0) added a high-speed mode

running at up to 3.4Mbps. Most of the I2C devices available today support 400Kbps

operation. Higher-speed operation may allow I2C to keep up with the rising demand for

bandwidth in multimedia and other applications.

I2C is appropriate for interfacing to devices on a single board, and can be stretched

across multiple boards inside a closed system, but not much further. An example is a host

CPU on a main embedded board using I2C to communicate with user interface devices

located on a separate front panel board. A second example is SDRAM DIMMs, which

can feature an I2C EEPROM containing parameters needed to correctly configure a

memory controller for that module.

ECE, SBMSIT 30

Eye controlled HMI

I2C is a two-wire serial bus, as shown in Figure 1. There's no need for chip select or

arbitration logic, making it cheap and simple to implement in hardware.

The two I2C signals are serial data (SDA) and serial clock (SCL). Together, these

signals make it possible to support serial transmission of 8-bit bytes of data-7-bit device

addresses plus control bits-over the two-wire serial bus. The device that initiates a

transaction on the I2C bus is termed the master. The master normally controls the clock

signal. A device being addressed by the master is called a slave.

In a bind, an I2C slave can hold off the master in the middle of a transaction using

what's called clock stretching (the slave keeps SCL pulled low until it's ready to

continue). Most I2C slave devices don't use this feature, but every master should support

it.

The I2C protocol supports multiple masters, but most system designs include only

one. There may be one or more slaves on the bus. Both masters and slaves can receive

and transmit data bytes.

Each I2C-compatible hardware slave device comes with a predefined device

address, the lower bits of which may be configurable at the board level. The master

transmits the device address of the intended slave at the beginning of every transaction.

Each slave is responsible for monitoring the bus and responding only to its own address.

This addressing scheme limits the number of identical slave devices that can exist on an

I2C bus without contention, with the limit set by the number of user-configurable address

bits (typically two bits, allowing up to four identical devices).

ECE, SBMSIT 31

Eye controlled HMI

Communication

As you can see in Figure 2, the master begins the communication by issuing the start

condition (S). The master continues by sending a unique 7-bit slave device address, with

the most significant bit (MSB) first. The eighth bit after the start, read/not-write (),

specifies whether the slave is now to receive (0) or to transmit (1). This is followed by an

ACK bit issued by the receiver, acknowledging receipt of the previous byte. Then the

transmitter (slave or master, as indicated by the bit) transmits a byte of data starting with

the MSB. At the end of the byte, the receiver (whether master or slave) issues a new ACK

bit. This 9-bit pattern is repeated if more bytes need to be transmitted.

In a write transaction (slave receiving), when the master is done transmitting all of

the data bytes it wants to send, it monitors the last ACK and then issues the stop condition

(P). In a read transaction (slave transmitting), the master does not acknowledge the final

byte it receives. This tells the slave that its transmission is done. The master then issues

the stop condition.

I2C offers good support for communication with on-board devices that are

accessed on an occasional basis. I2C's competitive advantage over other low-speed short-

distance communication schemes is that its cost and complexity don't scale up with the

number of devices on the bus. On the other hand, the complexity of the supporting I2C

software components can be significantly higher than that of several competing schemes

(SPI and MicroWire, to name two) in a very simple configuration. With its built-in

addressing scheme and straightforward means to transfer strings of bytes, I2C is an

elegant, minimalist solution for modest, "inside the box" communication needs.

ECE, SBMSIT 32

Eye controlled HMI

Advantages of I2C

• Only two bus lines are required to establish full-fledged bus.

• Each slave device connected is uniquely addressable using slave addresses

• Can choose a short 7 bit addressing or 10 bit addressing (which can accommodate

large number of devices on the same bus, but less popular).

• No strict baud rate specified since the clock is driven directly by the master.

Supports up to 3.4 Mbits/sec transfer speeds.

• True multimaster support with up to 8 masters in a single bus system.

• Very simple protocol which can be emulated by microcontrollers without

integrated I2C peripheral device. And its Inexpensive

Limitations of I2C

• 7 bit addressing supports only a very small number of devices.

• Different devices from different manufacturers come with hard coded slave

address or address will be configurable in a small range only. This can lead to

address clashes sometimes.

• No automatic bus configuration or plug and play

Applications of I2C

• I²C is appropriate for peripherals where simplicity and low manufacturing cost are

more important than speed. Common applications of the I²C bus are:

• Reading configuration data from SPD EEPROMs on SDRAM, DDR SDRAM,

DDR2 SDRAM memory sticks (DIMM) and other stacked PC boards

• Supporting systems management for PCI cards, through an SMBus 2.0

connection.

• Accessing NVRAM chips that keep user settings.

ECE, SBMSIT 33

Eye controlled HMI

• Accessing low speed DACs and ADCs.

• Changing contrast, hue, and color balance settings in monitors (Display Data

Channel).

• Changing sound volume in intelligent speakers.

• Controlling OLED/LCD displays, like in a cellphone.

• Reading hardware monitors and diagnostic sensors, like a CPU thermostat and fan

speed.

• Reading real time clocks.

• Turning on and turning off the power supply of system components.

• A particular strength of I²C is that a microcontroller can control a network of

device chips with just two general-purpose I/O pins and software.

• Peripherals can also be added to or removed from the I²C bus while the system is

running, which makes it ideal for applications that require hot swapping of

components.

Characteristics Of The I2C-Bus

The I2C-bus is for bidirectional, two-line communication between different ICs or

modules. The two lines are a serial data line (SDA) and a serial clock line (SCL). Both

lines must be connected to a positive supply via a pull-up resistor. Data transfer may be

initiated only when the bus is not busy.

Bit transfer

One data bit is transferred during each clock pulse. The data on the SDA line must

remain stable during the HIGH period of the clock pulse as changes in the data line at this

time will be interpreted as a control signal.

ECE, SBMSIT 34

Eye controlled HMI

Start and stop conditions

Both data and clock lines remain HIGH when the bus is not busy. A HIGH-to-LOW

transition of the data line, while the clock is HIGH, is defined as the start condition (S). A

LOW-to-HIGH transition of the data line while the clock is HIGH, is defined as the stop

condition (P).

Acknowledge

The number of data bytes transferred between the start and stop conditions from

transmitter to receiver is not limited. Each data byte of eight bits is followed by one

acknowledge bit. The acknowledge bit is a HIGH level put on the bus by the transmitter

whereas the master also generates an extra acknowledge related clock pulse.

A slave receiver which is addressed must generate an acknowledge after the

reception of each byte. Also a master must generate an acknowledge after the reception of

each byte that has been clocked out of the slave transmitter. The device that

acknowledges has to pull down the SDA line during the acknowledge clock pulse, so that

the SDA line is stable LOW during the HIGH period of the acknowledge related clock

pulse. A master receiver must signal an end of data to the transmitter by not generating an

ECE, SBMSIT 35

Eye controlled HMI

acknowledge on the last byte that has been clocked out of the slave. In this event the

transmitter must leave the data line HIGH to enable the master to generate a stop

condition.

Out of various devices we are using I2C Serial ADC (Analog to Digital

Convertor) for acquiring EOG signals.

ECE, SBMSIT 36

Eye controlled HMI

I2C Based Serial ADC PCF8591:

An analog-to-digital converter (ADC) is a device which converts a continuous

quantity to a discrete time digital representation. An ADC may also provide an isolated

measurement. The reverse operation is performed by a digital-to-analog converter (DAC).

Typically, an ADC is an electronic device that converts an input analog voltage or

current to a digital number proportional to the magnitude of the voltage or current.

Various ADC are available with serial and parallel interfacing. To save the pin

count and board space we have decided to use serial ADC, again in serial ADC we have

SPI interface and I2C interface, we have choose I2C serial ADC as it requires only 2

lines for interfacing and data rates are sufficient for most of the system. In our project we

are using PCF8591 from Philips which is I2C serial ADC/DAC chip.

PCF8591 Description:

The PCF8591 is a single-chip, single-supply low power 8-bit CMOS data

acquisition device with four analog inputs, one analog output and a serial I2C-bus

interface. Three address pins A0, A1 and A2 are used for programming the hardware

address, allowing the use of up to eight devices connected to the I2C-bus without

additional hardware. Address, control and data to and from the device are transferred

serially via the two-line bidirectional I2C-bus.

The functions of the device include analog input multiplexing, on-chip track and

hold function, 8-bit analog-to-digital conversion and an 8-bit digital-to-analog

conversion. The maximum conversion rate is given by the maximum speed of the I2C-

bus.

Features:

• Single power supply

ECE, SBMSIT 37

Eye controlled HMI

• Operating supply voltage 2.5 V to 6 V

• Low standby current

• Serial input/output via I2C-bus

• Address by 3 hardware address pins

• Sampling rate given by I2C-bus speed

• 4 analog inputs programmable as single-ended or differential inputs

• Auto-incremented channel selection

• Analog voltage range from VSS to VDD

• On-chip track and hold circuit

• 8-bit successive approximation A/D conversion

• Multiplying DAC with one analog output.

Applications:

• Closed loop control systems

• Low power converter for remote data acquisition

• Battery operated equipment

• Acquisition of analog values in automotive, audio and TV applications.

•

ECE, SBMSIT 38

Eye controlled HMI

Block Diagram:

Pin Description and Pin Diagram:

FUNCTIONAL DESCRIPTION

ECE, SBMSIT 39

Eye controlled HMI

Addressing

Each PCF8591 device in an I2C-bus system is activated by sending a valid

address to the device. The address consists of a fixed part and a programmable part. The

programmable part must be set according to the address pins A0, A1 and A2. The address

always has to be sent as the first byte after the start condition in the I2C-bus protocol. The

last bit of the address byte is the read/write-bit which sets the direction of the following

data transfer .

Control byte

The second byte sent to a PCF8591 device will be stored in its control register and

is required to control the device function. The upper nibble of the control register is used

for enabling the analog output, and for programming the analog inputs as single-ended or

differential inputs. The lower nibble selects one of the analog input channels defined by

the upper nibble. If the auto-increment flag is set, the channel number is incremented

automatically after each A/D conversion.

ECE, SBMSIT 40

Eye controlled HMI

A/D conversion

The A/D converter makes use of the successive approximation conversion

technique, the on-chip D/A converter and a high-gain comparator are used temporarily

during an A/D conversion cycle. An A/D conversion cycle is always started after sending

a valid read mode address to a PCF8591 device. The A/D conversion cycle is triggered at

the trailing edge of the acknowledge clock pulse and is executed while transmitting the

result of the previous conversion.

Once a conversion cycle is triggered an input voltage sample of the selected

channel is stored on the chip and is converted to the corresponding 8-bit binary code.

Samples picked up from differential inputs are converted to an 8-bit twos complement

code. The conversion result is stored in the ADC data register and awaits transmission.

If the auto-increment flag is set the next channel is selected. The first byte

transmitted in a read cycle contains the conversion result code of the previous read cycle.

ECE, SBMSIT 41

Eye controlled HMI

After a Power-on reset condition the first byte read is a hexadecimal 80. The maximum

A/D conversion rate is given by the actual speed of the I2C-bus.

Before above steps always we have to send channel selection control byte as write

command, steps are as follows:

o Start

o Address with Write cmd(0)

o Control byte with channel no(00 , 01 , 10 , 11)

o Stop

We are using P89V51RD2 to acquire and process the data and final eye

movement cmd is send via RF Tx module using Ht640 encoder, at receiving end after RF

Rx module data goes to decoder HT648 and then to Application Microcontroller all

interfacing is described in following chapters.

MICROCONTROLLER P89V51RX2

Computer in its simplest form needs at least 3 basic blocks: CPU, I/O and the

RAM/ROM. The integrated form of CPU is the microprocessor. As the use of

microprocessors in control applications increased, development of microcontroller unit or

ECE, SBMSIT 42

Eye controlled HMI

MCU took shape, wherein CPU, I/O and some limited memory on a single chip was

fabricated. Intention was to reduce the chip count as much as possible. We decided to use

P89V51RXX series of Microcontroller.

The P89V51RB2/RC2/RD2 are 80C51 microcontrollers with 16/32/64 kB flash

and 1024 B of data RAM. A key feature of the P89V51RB2/RC2/RD2 is its X2 mode

option. The design engineer can choose to run the application with the conventional

80C51 clock rate (12 clocks per machine cycle) or select the X2 mode (six clocks per

machine cycle) to achieve twice the throughput at the same clock frequency. Another way

to benefit from this feature is to keep the same performance by reducing the clock

frequency by half, thus dramatically reducing the EMI.

The flash program memory supports both parallel programming and in serial ISP.

Parallel programming mode offers gang-programming at high speed, reducing

programming costs and time to market. ISP allows a device to be reprogrammed in the

end product under software control. The capability to field/update the application

firmware makes a wide range of applications possible. The P89V51RB2/RC2/RD2 is also

capable of IAP, allowing the flash program memory to be reconfigured even while the

application is running.

Features of P89V51RXX:

80C51 CPU Core

5 V operating voltage from 0 MHz to 40 MHz

16/32/64 kB of on-chip flash user code memory with ISP and IAP

Supports 12-clock (default) or 6-clock mode selection via software or ISP

SPI and enhanced UART

PCA with PWM and capture/compare functions

Four 8-bit I/O ports with three high-current port 1 pins (16 mA each)

Three 16-bit timers/counters

Programmable watchdog timer

Eight interrupt sources with four priority levels

Second DPTR register

Low EMI mode (ALE inhibit)

ECE, SBMSIT 43

Eye controlled HMI

TTL- and CMOS-compatible logic levels

Brownout detection

Low power modes

o Power-down mode with external interrupt wake-up

o Idle mode

DIP40, PLCC44 and TQFP44 packages

Block Diagram:

Pin Diagram:

ECE, SBMSIT 44

Eye controlled HMI

Pin description of P89CV51RXX

Port 0: Port 0 is an 8-bit open drain bidirectional I/O port. Port 0 pins that have ‘1’s

written to them float, and in this state can be used as high-impedance inputs. Port 0 is also

the multiplexed low-order address and data bus during accesses to external code and data

memory. In this application, it uses strong internal pull-ups when transitioning to ‘1’s.

Port 0 also receives the code bytes during the external host mode programming, and

outputs the code bytes during the external host mode verification. External pull-ups are

required during program verification or as a general purpose I/O port.

Port 1: Port 1 is an 8-bit bidirectional I/O port with internal pull-ups. The Port 1 pins are

pulled high by the internal pull-ups when ‘1’s are written to them and can be used as

inputs in this state. As inputs, Port 1 pins that are externally pulled LOW will source

current (IIL) because of the internal pull-ups. P1.5, P1.6, P1.7 have high current drive of

ECE, SBMSIT 45

Eye controlled HMI

16 mA. Port 1 also receives the low-order address bytes during the external host mode

programming and verification.

Port 2: Port 2 is an 8-bit bidirectional I/O port with internal pull-ups. Port 2 pins are

pulled HIGH by the internal pull-ups when ‘1’s are written to them and can be used as

inputs in this state. As inputs, Port 2 pins that are externally pulled LOW will source

current (IIL) because of the internal pull-ups. Port 2 sends the high-order address byte

during fetches from external program memory and during accesses to external Data

Memory that use 16-bit address (MOVX@DPTR). In this application, it uses strong

internal pull-ups when transitioning to ‘1’s. Port 2 also receives some control signals and

a partial of high-order address bits during the external host mode programming and

verification.

Port 3: Port 3 is an 8-bit bidirectional I/O port with internal pull-ups. Port 3 pins are

pulled HIGH by the internal pull-ups when ‘1’s are written to them and can be used as

inputs in this state. As inputs, Port 3 pins that are externally pulled LOW will source

current (IIL) because of the internal pull-ups. Port 3 also receives some control signals

and a partial of high-order address bits during the external host mode programming and

verification.

PSEN: Program Store Enable is the read strobe for external program memory.

Reset: While the oscillator is running, a HIGH logic state on this pin for two machine

cycles will reset the device.

External Access Enable: EA must be connected to VSS in order to enable the device to

fetch code from the external program memory. EA must be strapped to VDD for internal

program execution.

ECE, SBMSIT 46

Eye controlled HMI

Address Latch Enable: ALE is the output signal for latching the low byte of the

address during an access to external memory. This pin is also the programming pulse

input (PROG) for flash programming.

Crystal 1: Input to the inverting oscillator amplifier and input to the internal clock

generator circuits.

Crystal 2: Output from the inverting oscillator amplifier.

VCC: Supply voltage.

GND: Ground.

The additional feature of Port3

ECE, SBMSIT 47

Eye controlled HMI

CHAPTER

RF INTERFACING

For transmission of data/Cmds from one point to another point wirelessly we need

to have RF interface, RF interfacing requires 4 parts:

1> RF Transmitter Module

2> RF Receiver Module

3> Transmitter Encoder

4> Receiver Decoder

RF Transmitter Module ST433/315

The STT-433/315 is ideal for remote control applications where low cost and

longer range is required. The transmitter operates from a 1.5-12V supply, making it ideal

for battery-powered applications. The transmitter employs a SAW-stabilized oscillator,

ensuring accurate frequency control for best range performance. Output power and

harmonic emissions are easy to control, making FCC and ETSI compliance easy. The

manufacturing-friendly SIP style package and low-cost make the STT-433/315 suitable

for high volume applications.

Features

433.92/315 MHz Frequency

Low Cost

1.5-12V operation

11mA current consumption at 3V

Small size

4 dBm output power at 3V

ECE, SBMSIT 48

Eye controlled HMI

Applications

Car security system

Sensor reporting

Automation system

Remote Keyless Entry (RKE)

Remote Lighting Controls

On-Site Paging

Asset Tracking

Wireless Alarm and Security Systems

Long Range RFID

Automated Resource Management

Note: 3pin RF Tx module is having helical wire antenna

RF Receiver Module STR-433/315

The STR-433/315 is ideal for short-range remote control applications where cost

is a primary concern. The receiver module requires no external RF components except for

the antenna. It generates virtually no emissions, making FCC and ETSI approvals easy.

The super-regenerative design exhibits exceptional sensitivity at a very low cost. The

manufacturing-friendly SIP style package and low-cost make the STR-433/315 suitable

for high volume applications.

ECE, SBMSIT 49

Eye controlled HMI

Features

Low Cost

5V operation

3.5mA current drain

No External Parts are required

Receiver Frequency: 433.92/315 MHZ

Typical sensitivity: -105dBm

IF Frequency: 1MHz

Applications: Same as Transmitter Module

ECE, SBMSIT 50

Eye controlled HMI

Note: 3pin RF Rx module is having helical wire antenna

Transmitter Encoder HT12E (212 series)

The 212 encoders are a series of CMOS LSIs for remote control system

applications. They are capable of encoding information which consists of N address bits

and 12-N data bits. Each address/data input can be set to one of the two logic states. The

programmed addresses/ data are transmitted together with the header bits via an RF or an

infrared transmission medium upon receipt of a trigger signal. The capability to select a

TE trigger on the HT12E or a DATA trigger on the HT12A further enhances the

application flexibility of the 212 series of encoders. The HT12A additionally provides a

38kHz carrier for infrared systems.

Features

Operating voltage 2.4V~5V for the HT12A 2.4V~12V for the HT12E

Low power and high noise immunity CMOS technology

ECE, SBMSIT 51

Eye controlled HMI

Low standby current: 0.1_A (typ.) at VDD=5V

HT12A with a 38kHz carrier for infrared transmission medium

Minimum transmission word

Four words for the HT12E

One word for the HT12A

Built-in oscillator needs only 5% resistor

Data code has positive polarity

Minimal external components

Pair with Holtek’s 212 series of decoders

18-pin DIP, 20-pin SOP package

Applications

Burglar alarm system

Smoke and fire alarm system

Garage door controllers

Car door controllers

Car alarm system

Security system

Cordless telephones

Other remote control systems

ECE, SBMSIT 52

Eye controlled HMI

Block diagram and Pin Diagram:

ECE, SBMSIT 53

Eye controlled HMI

ECE, SBMSIT 54

Eye controlled HMI

ECE, SBMSIT 55

Eye controlled HMI

Flow chart and application circuit of HT12E working:

ECE, SBMSIT 56

Eye controlled HMI

ECE, SBMSIT 57

Eye controlled HMI

Receiver Decoder HT12D (212 series)

The 212 decoders are a series of CMOS LSIs for remote control system

applications. They are paired with Holtek’s 212 series of encoders (refer to the

encoder/decoder cross reference table). For proper operation, a pair of encoder/decoder

with the same number of addresses and data format should be chosen. The decoders

receive serial addresses and data from a programmed 212 series of encoders that are

transmitted by a carrier using an RF or an IR transmission medium. They compare the

serial input data three times continuously with their local addresses. If no error or

unmatched codes are found, the input data codes are decoded and then transferred to the

output pins. The VT pin also goes high to indicate a valid transmission. The 212 series of

decoders are capable of decoding information that consists of N bits of address and 12_N

bits of data. Of this series, the HT12D is arranged to provide 8 address bits and 4 data

bits, and HT12F is used to decode 12 bits of address information.

Features

Operating voltage: 2.4V~12V

Low power and high noise immunity CMOS technology

Low standby current

Capable of decoding 12 bits of information

Binary address setting

Received codes are checked 3 times

Address/Data number combination

HT12D: 8 address bits and 4 data bits

HT12F: 12 address bits only

Built-in oscillator needs only 5% resistor

Valid transmission indicator

Easy interface with an RF or an infrared transmission medium

Minimal external components

Pair with Holtek’s 212 series of encoders

18-pin DIP, 20-pin SOP package

ECE, SBMSIT 58

Eye controlled HMI

Applications

Burglar alarm system

Smoke and fire alarm system

Garage door controllers

Car door controllers

Car alarm system

Security system

Cordless telephones

Other remote control systems

Block Diagram and Pin Diagram:

ECE, SBMSIT 59

Eye controlled HMI

ECE, SBMSIT 60

Eye controlled HMI

Operation

The 212 series of decoders provides various combinations of addresses and data

pins in different packages so as to pair with the 212 series of encoders. The decoders

receive data that are transmitted by an encoder and interpret the first N bits of code period

as addresses and the last 12_N bits as data, where N is the address code number. A signal

on the DIN pin activates the oscillator which in turn decodes the incoming address and

data. The decoders will then check the received address three times continuously. If the

received address codes all match the contents of the decoder’s local address, the 12_N

bits of data are decoded to activate the output pins and the VT pin is set high to indicate a

valid transmission. This will last unless the address code is incorrect or no signal is

received. The output of the VT pin is high only when the transmission

is valid. Otherwise it is always low.

Output Type

The 212 series of decoders, the HT12F has no data output pin but its VT pin can

be used as a momentary data output. The HT12D, on the other hand, provides 4 latch type

data pins whose data remain unchanged until new data are received.

ECE, SBMSIT 61

Eye controlled HMI

Flow Chart and Application Circuit

The oscillator is disabled in the standby state and activated when a logic “high”

signal applies to the DIN pin. That is to say, the DIN should be kept low if there is no

signal input

ECE, SBMSIT 62

Eye controlled HMI

Flow Chart:

Transmitter Module Interfacing

This is complete data transmission module, Microcontroller first send data to data

lines of encoder, address lines of encoder are hardwire, after giving data microcontroller

enables TE pin of encoder in order to start encoding process and at Dout pin of encoder

serial encoded data is coming this serial encoded data contains not only data but it

ECE, SBMSIT 63

EncoderHT12E

Microcontroller

TX RF Module

Data line

Transmit Enable

Microcontroller

DecoderHT12D

RX RF Module

Data line

VTValid

Eye controlled HMI

contains module address and sync bits, this signal is fed to RF Tx module which does

OOK modulation and transmits data wirelessly.

Receiver Module Interfacing

This is complete data Reception module; RF Rx module receives the wireless bit

stream this is fed to Decoder which first compares the address and sync if all is ok then it

compares data two times if both are same then only it enables VT pin (Valid

Transmission) and latch the output at data lines, when new data is received VT goes low

and then again with same logic it is set to high if same new data is received two times.

Rising edge of VT indicates new data has come falling edge of VT indicates data has

stopped or new data is going to come. We can read data at both edges. This can work in

two modes polling VT mode and Interrupt Mode in which VT is connected to INT0 pin of

MCU.

ECE, SBMSIT 64

Eye controlled HMI

ACQUISITION AND PROCESSING SYSTEM

Acquisition front end system will interface to the body get the EOG signal,

amplify it, filter it and pre process it to suit to ADC PCF8591 I2C Based 4 Channel ADC.

Left-right signal is given to channel 0 and up-down signal is given channel-1, this ADC is

interfaced to Microcontroller P89V51RD2 which is having I2C communication routines.

The microcontroller reads the data from ADC using I2C protocol and starts processing.

Once data is processed and if any eye movement was there it will conclude which eye

movement was made and decodes which command is given using eye. After decoding it

sends the command via RF transmitter module using HT12E Encoder.

Circuit diagram is given below:

ECE, SBMSIT 65

Eye controlled HMI

Figure: Acquisition and processing part

Processing of data to decode the eye movements:

Basically we get digital data from ADC for each channel, first we are checking for

straight sight to avoid noise and electrode not in use case. This we are doing by checking

that signal is not varying much it’s in some band near center. After the if signal goes up

for sufficient time > 200ms then its right eye movement in case of L-R and Up movement

in case of U-D, but if signal goes down then its left or down depending on which channel

you are processing. Any of the case if it come back before sufficient time then movement

is ignored but in case of up down, if signal is up for >50ms to <100ms then its consider as

blink movement.

Flow chart is for above processing is given below:

ECE, SBMSIT 66

Eye controlled HMI

We have looked in straight direction and signals are stabilized, now its turn to give

commands using eye movements.

Following are the follow charts to detect left, right, up, down and blink eye movements.

Figure: Flow chart for Left-Right Detection

ECE, SBMSIT 67

Eye controlled HMI

Figure: Flow chart for up-down and blink detections.

These commands are sent via RF Tx module at application part end there is RF

Rx module which receives the commands and send it to application controller which then

drives the cursor and operates buttons on Graphic LCD

ECE, SBMSIT 68

Eye controlled HMI

Graphics LCD JHD12864

JHD12864J is a light weight, low power consumption liquid crystal graphic

display. The module measures 54.0x50.0mm only. Supply voltage is 5V matching the

voltage for most microcontrollers. The LCD controller is Samsung KS0108B.

This LCD has 20 line interfacing which are described below:

Pin Description

Symbol  Level  Function 

Vss  0V  Ground 

Vdd  +5V  Power supply for logic 

Vo  -  Operating voltage for LCD (contrast adjusting)

RS  H/L  Register selection H:Display data L:Instruction code 

R/W  H/L  Read/Write selection H:Read operation L:Write Operation 

E H,H-

>L 

Enable Signal.Read data when E is high,Write data at the falling

Edge of E 

DB0  H/L  Data bit 0  

DB1  H/L  Data bit 1 

DB2  H/L  Data bit 2 

ECE, SBMSIT 69

Eye controlled HMI

DB3  H/L  Data bit 3 

DB4  H/L  Data bit 4 

DB5  H/L  Data bit 5 

DB6  H/L  Data bit 6 

DB7  H/L  Data bit 7 

CS1  H  select the right half of display the CS1 bit is set 

CS2  H  select the left half of display the CS2 bit is set 

/RST  L  Reset signal, active low 

Vout  -10V  Output voltage for LCD driving  

LEDA  +5V  Power supply for LED back light 

LEDB  0V  GND for LED back light 

The display is split logically in half. It contains two controllers with controller #1 (Chip

select 1) controlling the left half of the display and controller #2 (Chip select 2)

controlling the right half. Each controller must be addressed independently. The page

addresses, 0-7, specify one of the 8 horizontal pages which are 8 bits (1 byte) high. A

drawing of the display and how it is mapped to the refresh memory is shown below

.

ECE, SBMSIT 70

Eye controlled HMI

Block Diagram of GLCD:

ECE, SBMSIT 71

Eye controlled HMI

Following are display control Instruction which MCU has to give while

interfacing to GLCD module:

ECE, SBMSIT 72

Eye controlled HMI

Basic interfacing diagram with MCU:

ECE, SBMSIT 73

Eye controlled HMI

Algorithm and flowchart for GLCD interfacing

1.Send the display off command 3eh

2. Send the display on command 3fh

3.If required you can use 11xx xxxx instruction to set the display line start

4. Set the Y-adddress to first coloumn 40h

5.Set the X-address to first page 0B8h

6.Blank the Display( clear all 128x64 pixels)

ECE, SBMSIT 74

Eye controlled HMI

Application Part

Application part consists of Graphics LCD, P89V51RD2 microcontroller, RF

receiver module with decoder HT12D, ULN2008 high current Darlington driver for

controlling high current devices or relays.

Eye Movements commands are sent to application part via RF transmitter, the RF

receiver receives the data and HT12D does channel decoding and give digital data to

MCU and with VT pin signal it gives indication that data is received. Which trigger the

ECE, SBMSIT 75

Eye controlled HMI

interrupt and in ISR we are read output of encoder into the MCU, and data is decoded for

cursor commands and blinks (clicks). According to commands cursor is move on the

screen but menu is only visible after 4 blinks. After moving to cursor to required button 2

blinks are required to click the button and designated operation is performed after 2blinks

are received.

This microcontroller control the graphic lcd, it prints messages on screen, it

creates menu on screen with 4 buttons and one cursor, for each cursor command cursor on

screen is moved using creating the new cursor at new position and removing old cursor.

4 devices are connected at four MCU pins, and this pins goes to input of

ULN2803 so devices using voltages 5-12V and current upto 400-500mA can be

controlled directly and high current and high voltages devices can be controlled via

relays.

Buzzer and FAN in our demo project are duty cycle, we duty cycle FAN to save

power and Buzzer is duty cycled so that patient does not have to activate buzzer again and

again till someone comes and attain him. Buzzer is made off by a buzzer reset switch.

Circuit diagram and flowcharts are given in following pages

Flow chart for Application

ECE, SBMSIT 76

Eye controlled HMI

ECE, SBMSIT 77

Eye controlled HMI

Circuit for Application part:

ECE, SBMSIT 78

Eye controlled HMI

Software and hardware requirements

Software and Hardware requirements along with component list

PC Requirements:

Pentium 4 PC or higher OS: Win XP or higher.Minimum 1GB RAM.Software: KEIL, Flash Magic and HyperTerminal

Hardware components:

RF Transmitter and receiver – 315/433 MHz

ENCODER & DECODER BRDS - WITH HT12E 648

RELAY BOARD - Transistor Based

GRAPHIC DISPLAY - 128X64

HT12E - 4 BIT ENCODER

HT12D - 4 BIT DECODER

LM324– OPAMP X2

INA (Instrumentation amplifier) AD620 X2

I2C based ADC PCF8591

Power Supply 9V DC adapter, and 9V Batteries X378L05, 79L05

FAN, Buzzer and Cables

P89V51RD2 and General Purpose MCU Board X2

Finally Electrodes and Medical Accessories

ECE, SBMSIT 79

Eye controlled HMI

Applications and Future work

Applications

Applications are not only limited to disable persons although this technology is most

useful for them:

Control of House hold appliances and Emergency call Bell.

A screen is provided with some buttons on that disable person move the cursor with eye

movement/ or cursor is slowly moving and to click a button he/she blinks the eye two

times and whatever button is meant for its executed, it can be turning on & off light

lights, calling assistance using bell etc.

Speech system for disable

this system enables disable person to talk via computer which has a special application

running having on screen keyboard, a text box not only this it as prediction of text logic

as he/she is trying to type by using eye movements on on screen keyboard system provide

predicted text in the drop down of text box which one can select to speed

text entry process

Email facility for disable

like previous application it has all interfaces and along with that email client is there

which sends email to fixed address or address can be typed via eye movement or selected

from address book again using eye signals only.

Eye movement controlled wheel chair

Wheel chair can go forward, backward, turn left and right using eye movements of the

disable person sitting on it.

US army is doing research on eye movement based Air craft and tank controls

ECE, SBMSIT 80

Eye controlled HMI

Hands free mouse using eye movement controller very useful specially in gaming 

TV operation channel up & down volume up & down, turn on and off TV using

eye movements very useful for disables

Eye movements controlled mp3 player when ur hands are busy you can use this

system to play/pause, track change, and volume up&down using eye movements

ECE, SBMSIT 81

Eye controlled HMI

CONCLUSION

EXPECTED OUTPUT AND CONCLUSION

Intermediate Output of the system is Eye movements and Eye Blinking Commands

CU - Eye up movement detected

CD - Eye down movement detected

CL - Eye Left movement detected

CR - Eye Right movement detected

BL - Eye Blink detected

Above o/p is sent to application part which interprets and move cursors accordingly and

button are clicked using same o/p and corresponding relay/device is operated and hence

appliance is controlled using eye movements, in this project we have successfully

controlled appliances using eye movement.

Achievement:

Detection of Eye movements and eye blink at minimum 10sec rate with 1Sigma accuracy.

ECE, SBMSIT 82

Eye controlled HMI

REFERENCES

http://en.wikipedia.org/wiki/Eye_tracking

The 8051 Microcontroller, Kenneth J Ayala

C and the 8051, Thomas W. Schultz

IEEE Paper: EOG signal detection for home appliances activation

Human-Computer Systems Interaction: Backgrounds and Applications By

Zdzislaw S. Hippe, Juliusz L. Kulikowski

Hand book Of Biomedical Instrumentation  By Khandpur.

Fun n' Games  By Panos Markopoulos, Boris de Ruyter, Wijnand Ijsselsteijn.

Intelligent wearable interfaces  By Yangsheng Xu, Wen J. Li, Ka Keung Caramon

Lee

Manuals in keil software

I2C-bus specification (version 2.1), from NXP semiconductors (Philips).

Datasheets of all the IC’s used in the system

ECE, SBMSIT 83