A Helping Hand in Communication The future of communication for the hearing and vision impaired Mr Iain Murray Dr Euan Lindsay.

A Helping Hand in Communication

The future of communication for the hearing and vision

impairedMr Iain MurrayDr Euan Lindsay

A day in the life of John

• John is – A student– Shares a house with two friends– Enjoys shopping– Keeps in touch with friends and family– Totally Blind

Colours

• Inexpensive• Accurate

(enough)• Long battery life• Compact

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Currency Identifier

• Australian Polymer notes are of a similar size– Hard to distinguish denomination if totally blind

• Utilises the unique pattern in the clear window

A Secure Dual Channel Wireless Headset for Multi-user

Environments• Dual channel, two-

way, high quality, low power infrared headset prototype which has been field tested by the ABWA and certified.

Current Form Ideal Form

Braille• Not a direct correlation to text

– Similar to shorthand

• Consists of 6 dots that may be embossed on both sides

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Curtin University Brailler (CUB)

• Perkins Brailler– Mechanical only

• Mountbatten Brailler– Electronic, very old

technology

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

QuickTime™ and aMPEG-4 Video decompressor

are needed to see this picture.

Curtin University Brailler

Braille Scanner

• Aimed at Sighted non-Braille Readers

• Converts Braille to Expanded Text in real time

QuickTime™ and aNone decompressor

are needed to see this picture.

Cisco Network Academy Program

• The Networking Academy program is an e-learning model that delivers Web-based educational content, online testing, student performance tracking, and instructor training and support, as well as hands-on labs. (Cisco,2002)

Cisco Network Academy Program (CNAP)

• 10,312 Academies– 1229 in Asia-Pacific

• 162 Countries– 25 Countries in Asia Pacific– 97,488 students in Asia Pacific

(as of 23rd April 2006)

Classmates

Cisco Network Academy Program

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

Current Access Methods

• For low vision users.– Screen enlargement.

• Zoomtext, Magic.

– Screen review programs.• Jaws, Slimware, Artic.• Two output methods - speech and Braille displays.

– Screen review software must rely on text output.

• Tactile Graphics– PIAF

• Printed Braille and Audio recordings

Network Dominoes

Tactile Printing

Whiteboard

• Cheap camera– <$150

• Portable– Multiple use of

laptop QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

iNetSim

• Simulates a network• Allows for trials of configurations without expensive

equipment• Works with voice output

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

What about John’s Text Books?

• Daisy Digital Talking Book (DTB) compatible

• Full Indexing• Annotations

– Text– Audio– Video

• High speed capture– 90 minutes in 4

minutes

• DRM enabled

What about Tony?

QuickTime™ and aMPEG-4 Video decompressor

are needed to see this picture.

Sign Language

• Common for hearing impairment• Uses Vision to replace sound• Many types

– Auslan– ASL– BSL

But what if you don’thave vision either?

A Demonstration

Some Obvious Challenges…

• Need an interpreter

• Need a fluent interpreter

• Slow to start with

… and some not obvious ones

• One-to-one

• In-person

• Slang / Dialects

An Alternative

Removes Barriers

• Communicate with non-signers– Through their choice of interface

• Direct interaction, without an interpreter

• Can talk to people in other rooms / cities / countries

• Multiple people at once• Recording

The Hand is A Complex System

The Current Hand

QuickTime™ and aMPEG-4 Video decompressor

are needed to see this picture.

All good

QuickTime™ and aMPEG-4 Video decompressor

are needed to see this picture.

The challenges ahead…

• Servomotors instead of air muscles

• Artificial Intelligence to learn trajectories for the hand signs– Learn by mimicking

More challenges

• Communication protocol– Correct data vs On-Time data

• Social / Cultural Implications– “SMS” speak?

The Team(in no particular order)

• Staff– Chris Moore– Serge Mokroous– Kieren Eaton– Leigh Harrison

• Students– Andrew Pasquale, Simon Dunn, Ryan

Williams, Carl Blair, James Hope, Leong Chen Chew, Ben Farrugia, Ivar Siewert, Peter Scarfe

Thank You