Report

PAACET

1

ROBOTIC ARM AND ITS CONTROL

ABSTRACT

In this project, we design and build a versatile robotic arm system. The arm has the

ability to manipulate objects such as pick and place operations. Firstly, the robotic arm is

built in order to interface with a prosthetic control board. The circuit board enables user to

completely control the robotic arm and moreover, enables feedbacks from user. The

control circuit board uses a powerful integrated microcontroller, a PIC (Programmable

Interface Controller). The PIC is primarily programmed using assembly programming

language and it is used as the „brain‟ of the arm.

The second part of the project is to use speech recognition control on the robotic

arm. A speech recognition circuit board is constructed with onboard components such as

PIC and other integrated circuits. The robotic arm is able to receive instructions as spoken

commands through a speech recognition system via a microphone and perform operations

with respect to the commands such as picking and placing operations.

PAACET

2


1. INTRODUCTION

An upper limb myoelectric prosthetic arm is an aid that tries to give a chance of a

better quality of life to disabled people. It tries to give back some of the natural and

fundamental functions of a physiological human arm, even if the movements that is able

to perform are not so deeply similar to those of a natural arm. To control such a device,

several ways are possible. The more traditional one is nowadays the EMG

(electromyography) control, which is based on EMG signals extracted from surface

electrodes of user‟s arm or forearm, while the simplest technique is using buttons or

switches when the electromyography activity of patient muscles is not so good or clear.

Lately, in the last decade more complex ways were explored to widen the range of

possible input sources for the controller of a prosthetic arm, so neuro cortical control ,

foot control with wireless wearable insoles , control with implantable myoelectric sensors

(IMES), with MMG sensors (mechanomyographic) and ultrasonic sensors were

investigated, even if is not clear if these techniques are really used by patients in their

everyday life or if they are just interesting theoretical contributions in the wide field of

prosthetic arm control. All these techniques start from the assumption that the prosthetic

motion is directly “linked” to the human motion, the source being both the EMG activity

or the foot motion or something else. Since the most common control scheme for an upper

limb prosthetic arm is a sequential control (where signals or switches are used to change

control from one degree of freedom to another), it follows that all these techniques have

the same problem: when the patient has to perform a complex task, formed by a

predetermined and precise sequence of movements, he has to do a precise sequence of

contractions/movements, always remembering which motor is selected in every instant of

time. This is not as simple as one can believe, especially when there are more degrees of

freedom (typically three for a transhomerus or a shoulder disarticulated patient: the

flection/extention of the elbow, the prono/supination of the wrist and the opening/closing

of the hand).

A myoelectric prosthesis uses EMG signals or potentials from voluntarily

contracted muscles within a person's residual limb on the surface of the skin to control the

PAACET

3


movements of the prosthesis, such as elbow flexion/extension, wrist supination/pronation

(rotation) or hand opening/closing of the fingers. Prosthesis of this type utilizes the

residual neuro-muscular system of the human body to control the functions of an electric

powered prosthetic hand, wrist or elbow. This is as opposed to an electric switch

prosthesis, which requires straps and/or cables actuated by body movements to actuate or

operate switches that control the movements of prosthesis or one that is totally

mechanical. It is not clear whether those few prostheses that provide feedback signals to

those muscles are also myoelectric in nature. It has a self suspending socket with pick up

electrodes placed over flexors and extensors for the movement of flexion and extension

respectively.

Let consider the following case: a patient has to bring a bottle and pour water into

his glass. The sequence of contractions that he must do, in the case of sequential control

of the motors and thinking about three sources of emg signal, is illustrated in table 1:

TABLE I

From the table above we can understand that even if the motion task seems to be

very easy, the patient has to do a precise sequence of twelve contractions to perform it. If

PAACET

4


we think that now, in some cases, there is the idea to add, besides the elbow, the wrist and

the hand motors, also a shoulder motor group with two motors, one for the intra-extra

rotation and one for the elevation-adduction, (so adding four possible movements, with

other four sources of emg signals) it is clear that controlling the prosthetic device only

with emg signals could become more and more difficult, also because the possible EMG

sources located in the muscles near the amputation line are not utilizable, due to a bad or

insufficient EMG activity. For this reason we thought to another alternative input source,

potentially efficient and easy to be used, and overall disconnected from the human body

motion. In particular we focused on the voice control.

PAACET

5


2. HOW DOES THE MYOELECTRIC ARM WORK?

Electric prostheses use small electric motors to move the replaced limb. These

motors can be found in the terminal device (hand or hook), wrist and elbow. An

electrically-powered prosthesis utilizes a rechargeable battery system to power the

motors. Since electric motors are used to operate hand function, grip force of the hand is

significantly increased in comparison to earlier functional prostheses, often in excess of

20-32 pounds (Motion Control).

There are many ways to control an electrical prosthesis, one of the more popular

being myoelectric control. Whenever a muscle in the body is contracted, or flexed, a small

electrical signal called an EMG in the range of 5 to 20 microvolts is created by a chemical

interaction in the body (Animated Prosthetics). A typical light bulb uses 110 to 120 volts,

so the signal generated by the body is less than a millionth of the strength of a light bulb

(Animated Prosthetics).

One of the key components of the myoelectric arm is the electrode attached to the

surface of the skin to record the EMG signal. Once recorded, the signal is amplified, then

processed by a controller that switches the motors on or off in the hand, wrist, or elbow to

produce movement and function (Animated Prosthetics).

PAACET

6


Not everyone can wear the myoelectric arm. Users must be able to produce an

EMG strong enough to be recorded and sufficiently amplified. Users must also be able to

separate muscle contractions. Separating contraction means that when one muscle is

contracted, the opposing muscle is relaxed. If both muscles were contracted at the same

time (co-contraction), the controller would receive signals to both turn the motor on and

off at the same time. This would signal the hand to open and close simultaneously,

resulting in no function.

PAACET

7


2.1 ADVANTAGES AND DISADVANTAGES

There are several advantages to wearing an electric prosthesis like the myoelectric

arm. Most people prefer this type of control because non-electric prostheses are often

laborious to operate, whereas simply flexing a muscle can control myoelectrically

powered prostheses. They eliminate the need for the tight harness amputees have to wear

if they choose a non-electric prosthesis. Since electric prostheses do not have to utilize a

control cable or harness, cosmetic skin made of silicon or latex can be applied to the

prosthesis, greatly enhancing the cosmetic restoration (Advanced Arm Dynamics).

Perhaps the greatest advantage of the myoelectric arm is the operational range. It

can be used over the head, down by the feet, and out to the sides of the body. Such

movements are nearly impossible with cumbersome, non-electric prostheses.

Unfortunately, the myoelectric hand is not perfect. One of the major inconveniences

of electrically powered prostheses is the required battery system. Such a system needs a

certain level of maintenance, including charging, discharging, and the eventual disposal

and replacement of the battery. Electrically powered prostheses also tend to be heavier

than other prosthetic options due to the weight of the motor and batteries. However,

advanced suspension designs have minimized the weight greatly.

Another disadvantage is potential malfunction of the arm, resulting in costly

repairs. Wearers also have to be very cautious around water. Severe damage to the motor

and controller can result from water exposure.

Cosmetically there seems to be no disadvantages over traditional prostheses. Yet

under extreme conditions, latex covered prostheses are prone to staining, so several

coverings may be necessary throughout the device's lifetime.

There are several companies that currently produce the myoelectric arm, including

Motion Control, Otto Bock Orthopedic Industry, Hosmer, and Liberty and Technology

Prosthetics and Orthopedics.

PAACET

8


3. BASIC BLOCK DIAGRAM OF THE SYSTEM

MIC

3.1 USING THE VOICE TO CONTROL A DEVICE

Nowadays using the voice to control an electronic device is a quite common

process, and there are several electronic equipments that can be commanded by voice,

such as telephones, surgery robots, wheelchairs, military devices and so on. A voice

recognition system is composed by an input device, typically a microphone, and an

intelligent core that performs the recognition operations, which are, for the most part,

software elaborations of the signal acquired from the input device. Explaining the

recognition techniques is not the aim of the paper, but information can be found in. In this

work we used the voice recognition system in which the core is the HM 2007 IC by

Hualon. The board is connected to an embedded hardware which is the control board of

the prosthetic device, which acquires the inputs voice signals, elaborates them and

performs the motor actions requested by the patient.

The voice recognition process is articulated in two different phases: the first one,

called training phase, where the module is taught with the words that it must recognize,

and a second phase, the standard operation, where one pronounces a word and the module

compares it with the stored words and decide which of them the most similar one is. The

module is programmed in voice dependent mode, so it can recognize a word only if it is

pronounced by the same person that has done the training phase. In this context, the voice

command is not intended to completely substitute the traditional EMG control, but just to

join it, to expand the possibilities of controlling the device, and to simplify the control

process in case of complex and repetitive motion tasks.

VOICE RECOGNITION SYSTEM PROSTHETIC CONTROL

BOARD

To the

arm Fig 3: BASIC BLOCK DIAGRAM OF THE SYSTEM

PAACET

9


3.2 VOICE RECOGNITION

Speech recognition is the process of converting an acoustic signal, captured by a

microphone or a telephone, to a set of words. The recognized words can be the final

result, as for applications such as command & control, data entry, and document

preparation or retrieval. The basic assumption of the whole word pattern matching

approach is that different utterance of the same word by a particular talker result in similar

patterns of sound. There will be variation in spectrum shape at corresponding parts of the

patterns from the same word. There will also be variations in the time scale of the

patterns, and this will make it difficult to compare corresponding parts.

The basic building block of speech is the phoneme. There is one phoneme for every

basic sound in the language. For example, the word 'cat' is constructed from three

phonemes -'k', 'a' and‟t‟. A Speech Recognition Engine will need to construct the

sequence of the phonemes in the speech, before it can produce the sequence of words.

This is typically carried out in a number of distinct stages.

PAACET

10


ADC PARAMETER

EXTRACTION OUTPUT

DEVICE

TEMPLATE

MEMORY

PATTERN

MATCHING

3.3 VOICE RECOGNITION SYSTEM

Voice recognition involves inputting of information in to a computer using human

voice and the computer listening and recognizing the human speech. Voice recognition is

still being actively researched as problems posed are more difficult than those of speech

synthesis. Thus, successful commercial speech recognition systems are few and far

between the more successful ones are speaker dependent single-work systems. Such

systems operate in one of two modes. In the training mode the user trains the system to

recognize his/her voice by speaking each word to be recognized in to a microphone. The

system digitizes and creates a template of each word and stores this in its memory. In the

recognition mode each spoken word is again digitized and its template compared with the

templates in memory. When a match occurs, the word has been recognized and the system

informs the user or takes some action. The performance of such systems is affected by

speakers not passing long enough after each word, background noise, and how clearly and

carefully the work is spoken. The two important DSP operations in a recognizer are

parameter extraction, where distinct patterns are obtained from the spoken word and used

to create template and pattern matching where the templates are compared with those

stored in memory; see fig.

For most people, voice is the most natural form of communication, being faster than

writing or typing. Thus, in the office environment, voice systems now exist which allows

application programs to be driven by voice commands instead of by keyboard entries.

Systems which will allow the usual office documents, such as letters and memos, to be

Fig 4: BLOCK DIAGRAM OF VOICE RECOGNITION SYSTEM

PAACET

11


created and sent by voice are envisaged. Word recognizers are being incorporated in to

consumers products, such as voice operated telephone dialing systems, and are used in

voice activated domestic appliance for disabled people with limited movement. This

increases their independence by enabling them to perform simple tasks such as turning

on/off lights, radio or TV.

There are of course numerous potential applications of voice recognition. However,

it appears that future advances in this area will rely significantly on artificial intelligence

techniques because of the need for machines to understand as well as recognize speech.

A speech recognition control system capable of controlling the robotic arm using

voice commands is also constructed, where hands-free operation is desired. The ability to

communicate with a robot through speech is the ultimate user interface. When a robot

obtains the ability to recognize words, it is well on its way to becoming a true humanoid.

This speech recognition control circuit to be built provides a simple and effective means

for humans to specify a task for the robot to acquire new skills without any additional

hard coded programming. Robots have become important over a wide range of

applications--from manufacturing, to surgery, to the handling of hazardous materials.

Consequently, it's important to understand how they work, and what problems exist in

designing effective robots.

PAACET

12


3.4 VOICE RECOGNITION CHIP

HM2007

HM2007is a single chip CMOS voice recognition LSI circuit with the on-chip

analog front end voice analysis, recognition process and system control functions. A 40

isolated-word voice recognition system can be composed of external microphone,

keyboard, 64K SRAM and some other components .Combined with the microprocessor,

an intelligent recognition system can be built.

FEATURES

Single chip voice recognition CMOS LSL

Speaker-dependent isolates-word recognition system.

External 64K SRAM can be connected directly.

Maximum 40 words can be recognized forODCchip.

Maximum 1.92 sec of word can be recognized,

Multiple-chip configuration is possible.

A microphone can be connected directly.

Two control modes are supported: Manual mode and CPU mode.

Response time : less than 300 ms.

5V single power supply.

48-pin PDIP, 51 pin PLCC. 48 pad bare chip.

TC8860F

The voice recognition chip used for processing the input speech is TC8860F. It is a

single chip LSI with onchip circuits and functions required for voice recognition including

analog circuit, registration RAM, and pattern matching function. It is possible to construct

a voice recognition system only by externally connecting a microphone and keyboard to

this LSI. The chip can be operated in manual/CPU mode. In manual mode of operation, a

4 x 3 keypad matrix is used for inputting the commands to the chip. The chip has a 4Kbit

volatile built in SRAM.

PAACET

13


FEATURES OF TC8860F:

Single chip voice recognition LSI

Speaker dependent word recognition system

Linear matching system

No. of words that can be registered: Max 10 words

Response time is Max 0.60sec, average 0.35sec

Input voice time length allowed: 0.16 ~ 0.96sec

Built-in 4Kbit RAM for registration

A microphone for inputting the voice

Built-in 800KHz oscillator circuit

5V single power supply

PAACET

14


3.5 COMPARISON OF DIFFERENT VOICE RECOGNITION-

CHIPS

Chip Manufacturer SNR in dB Cost

TC8860F Toshiba 30dB 15$

HM2007 Huilon <10dB 25$

RSC-64 Sensory devices 15dB 20$

TMS-320C2X Texas Instruments 25dB 30$

Table 2: Table for comparison of different chips available

The comparison of different speech recognition chips yields us the information that

HM2007 is having an average signal to noise ratio. Apart from the cost of the chip, its

availability was given more importance compared to its counterparts. This is the reason

why it is selected for designing the voice controlled prosthetic arm project. The pin out of

the HM2007 IC is given in the figure below.

PAACET

15


Fig 5: PIN OUT OF HM 2007 IC

PAACET

16


4. MOTION SYSTEM USING VOICE RECOGNITION

CONTROL

The speech recognition circuit functions as a standalone circuit and it works

independently. Words are recognized through interrupt operations whereby the

recognition line is connected to the robot‟s interrupt lines. This is much better than using

polling operation that causes CPU overhead. The main component of the circuit is the HM

2007 speech recognition chip. The HM 2007 chip is a CMOS voice recognition chip with

voice analysis, recognition process and system control functions. The other major

components are the 64K CMOS Static RAM chip, microphone, 12 button-keypad and

74LS373 chip. Data can be written and read from the SRAM chip and the 74LS373

functions as a latch with 3-state outputs. There are also two BCD to 7-segment converters

used to display the output the words recognition. It functions as an indicator to user as the

circuit is working properly. The circuit is a speaker dependent system whereby it is only

able to recognizing the individual that train the circuit. It a capable of providing high

functioning output as high as 95% accuracy. However, there is constraint of the circuit

Fig 6: CIRCUIT DIAGRAM FOR SPEECH RECOGNITION SYSTEM

PAACET

17


concerning the style of speech it can recognize. For example, it can only recognize words

that spoken separately with pause in between each word.

It is programmable to recognize 40 unique words of 0.96s length and a maximum of

20 words of 1.952s length. The length of the words affects the number of words able to be

store in the 8K x 8 static RAM chip. The circuit is able to detect voice as far as one foot

from the microphone. This speech circuit provides many advantages compared with other

circuits as the response time is less than 300 ms, it requires only a 5 V DC power supply

and it can support CPU mode and manual mode whereby the manual mode is connected

to a keypad and CPU mode is connected to a microcontroller

PAACET

18


4.1 TESTING AND TRAINING OF SPEECH RECOGNITION

CIRCUIT

The circuit shown in Fig. 6 was constructed into on breadboard for testing and

training. Testing and training the HM 2007 chip in manual mode requires the keypad and

microphone. When the circuit is powered on, the HM 2007 checks the static RAM and

display “00” on the 7-segment and also lights the LED. The system is in ready state and

ready to be trained. Training procedures of the circuit includes:

1. Press “01” and the 7-segment will display “01”. Led will turn off.

2. Then press “train” and Led will on again.

3. Hold the microphone close to user and say training word.

4. If word is recognize by circuit, Led will blink.

5. Repeat the training word and “01” will be display if word is accepted.

6. Continue training with other words and train from 02” to a maximum of “40”.

The output is connected to a PIC microcontroller to read the all the 8-bit outputs

from the circuit. The 8 outputs are taken from the output of the 74LS373 latch. The PIC is

then connected to the serial servo controller circuit and control the movement of the 8

servo motors. The circuit was constructed a few times and troubleshooting was done by

ensuring all connections are correct and all necessary pin connections are connected.

PAACET

19


5. PROSTHETIC CONTROL BOARD

The output from the speech processing board is allowed for controlling of servo

motors. The programmable Interrupt Controller (PIC) is used for actuating the motors.

The PIC is being programmed using MPLAB-IDE. Assembly language is written in this

workbench and written into the PIC using a hardware called INCHWORM

PROGRAMMER provided by MICROCHIP (Manufacturer of PIC). The programmer

hardware is shown below.

An ordinary human arm consists of the following parts:

Upper arm

Elbow

Wrist

Fingers

The robotic arm is designed to be similar with a human arm with nine degrees of

freedom where each part of the arm is actuated with servo motors.Our objective is to

attain all the degrees of freedom. This is accomplished using high torque motors. Among

different types of DC motors available, servo motors have higher torque capacity. This

enables the prosthetic arm to do jobs in close relation with an ordinary arm. The table

Fig 7: INCHWORM PROGRAMMER

PAACET

20


shown below gives a clear idea about the relation between output combinations of speech

processing system with that of the arm movements. The robotic arm is controlled by the

control board which is based on the PIC 16F877A, a type flash programmable controller.

The main objective of designing using a microcontroller is that a large amount of

electronics needed for certain applications can be eliminated.

BIT

COMBINATIONS

B3 B2 B1 B0

ARM MOVEMENTS

0 0 0 1 Shoulder motor right

0 0 1 0 Shoulder motor left

0 0 1 1 Elbow motor up

0 1 0 0 Elbow motor down

0 1 0 1 Wrist motor right

0 1 1 0 Wrist motor left

0 1 1 1 Plunger for finger movement (IN)

1 0 0 0 Plunger for finger movement (IN)

(Fig 12 Truth table for arm movement Designing)

PAACET

21


5.1 PIC 16F877A

This is an 8-bit controller with programmable flash memory.

PERIPHERAL FEATURES:

Timer0: 8-bit timer/counter with 8-bit prescaler \

Timer1: 16-bit timer/counter with prescaler, can be incremented during Sleep via

external crystal/clock

Timer2: 8-bit timer/counter with 8-bit period register, prescaler and postscaler

Two Capture, Compare, PWM modules

- Capture is 16-bit, max. resolution is 12.5 ns

- Compare is 16-bit, max. resolution is 200 ns

- PWM max. resolution is 10-bit

Synchronous Serial Port (SSP) with SPI™

(Master mode) and I2C™ (Master/Slave)

Universal Synchronous Asynchronous Receiver

Transmitter (USART/SCI) with 9-bit address detection

Parallel Slave Port (PSP) – 8 bits wide with external RD, WR and CS controls

(40/44-pin only)

Brown-out detection circuitry for Brown-out Reset (BOR)

ANALOG FEATURES:

10-bit, up to 8-channel Analog-to-Digital

Converter (A/D)

Brown-out Reset (BOR)

Analog Comparator module with:

- Two analog comparators

- Programmable on-chip voltage reference

(VREF) module

PAACET

22


- Programmable input multiplexing from device inputs and internal voltage

reference

- Comparator outputs are externally accessible

SPECIAL MICROCONTROLLER FEATURES:

100,000 erase/write cycle Enhanced Flash program memory typical

1,000,000 erase/write cycle Data EEPROM memory typical

Data EEPROM Retention > 40 years

Self-reprogrammable under software control

In-Circuit Serial Programming™ (ICSP™) via two pins

Single-supply 5V In-Circuit Serial Programming

Watchdog Timer (WDT) with its own on-chip RC oscillator for reliable operation

Programmable code protection

Power saving Sleep mode

Selectable oscillator options

In-Circuit Debug (ICD) via two pins

CMOS TECHNOLOGY:

Low-power, high-speed Flash/EEPROM technology

Fully static design

Wide operating voltage range (2.0V to 5.5V)

Commercial and Industrial temperature ranges

Low-power consumption

PAACET

23


CONCLSUION

During the first phase of our project we were able to finish the speech recognition

part and a part of the prosthetic control board. The construction of the arm and interfacing

the servo motors are the major task before us for the second phase of our project.

PAACET

24


REFERENCE:

Tele-Operated Anthropomorphic Arm and Hand Design

Namal A. Senanayake, Khoo B. How, and Quah W. Wai

Controlling a prosthetic arm with a throat microphone

Elena Mainardi, Angelo Davalli

Development of a prosthetic arm: experimental validation with the user and an

adapted software

V. Artigue, G. Thomann

Report

Documents

basic block

electrically

voice recognition

prosthetic

voice recognition

electric prostheses

emg signals

prosthetic