Pardon Me, Your Computer’s Showing Using speech to speed and streamline desktop computing Kimberly Patch President, Redstart Systems SpeechTek West February.

Pardon Me, Your Computer’s ShowingUsing speech to speed and streamline desktop computing

Kimberly PatchPresident,Redstart Systems

SpeechTek WestFebruary 22, 2007

Keys to Using Speech to Speed and Streamline Computer Control

• Minimizing steps

• Making commands easy to remember

• Enabling combinations

Human-Machine Grammar • Follows the way the brain works

• Designed for controlling a computer

Keys to Using Speech to Speed and Streamline Computer Control

• Minimizing steps

Tap existing knowledge, but don’t let experience confine new methods of communication.

11 steps

Retrieving a PictureKeyboard and Mouse Steps

Click InsertClick PictureClick From FileClick Look InClick Program FilesClick Redstart SystemsClick Utter CommandClick DemoClick Bird PicturesLeft ArrowEnter

Think outside the silicon box

11 steps

Retrieving a PictureKeyboard/Mouse Style in the Real World

Stand upWalk to the file cabinetOpen the third drawerGo to the "B" sectionFind the Bird folderOpen the Bird folderFind the Pictures subfolderFind the Redstart Singing pictureTake it out of the folderPut in my handLet go

Efficient Speech # 1 Efficient Speech # 2

Under i p fBird Pictures Folder1 Right Enter

Redstart Singing FileAll Copy to Word New

3 steps 2 steps

Retrieving a Picture

Keyboard and Mouse

Click Outlook Express iconClick MessageClick New MessageType “David, Linda”Tab KeyType “Eric”Tab KeyType “Paris trip”Tab KeyType “David, Linda,”EnterEnter

Address Email and Add Subject, First Line

12 steps

Keyboard and Mouse Efficient Speech

Click Outlook Express iconClick MessageClick New MessageType “David, Linda”Tab KeyType “Eric”Tab KeyType “Paris trip”Tab KeyType “David, Linda,”EnterEnter

Express David Linda CC KimParis trip1 Tab

12 steps 3 steps

Address Email and Add Subject, First Line

Accommodating vs. Direct

Accommodating vs. Direct

Keys to Using Speech to Speed and Streamline Computer Control

• Minimizing steps

• Making commands easy to remember

How do you remember commands?

Two Fry

Counter Trey Right

Unit 26, Code 11-31, 13th and Vine

Delta 265, Clear to Land, Runway 3-0

Chair Dance

Teeth Grow

Blues 2

Human-Human Human-Computer

Two FryCounter Trey Right Delta 265, Clear to land, Runway 3-0Chair DanceBlues 2

Word OpenBackspace Left DeleteBird Pictures FolderWindow 0 by 20Window Close No

Efficiently Worded Instruction

Important Factors

• How the brain processes words

Important Factors

• How the brain processes words

• Memory chunking

Important Factors

• How the brain processes words

• Memory chunking

• Six degrees of separation

Human-Machine Grammar

• No synonyms

Human-Machine Grammar

• No synonyms

• Logical rules to minimize wording possibilities

Human-Machine Grammar

• No synonyms

• Logical rules to minimize wording possibilities

• Follows the way the brain uses language

UC Top 60 Command Words (plus numbers and labels)

All CapsAnother ByCapCheckClearCompoundCopyCutGoGraphInsert

MenuMessageMicrophoneMouseNewNopePasteRedoScreen SecondsShortSite

SpeechSpellThisTouch Touch 2Tray 1-20 UnderUndoVolumeWin(dow)WordWords

<0-200><1st-20th><screen labels>

Paired words

Left – RightLefts – RightsBefore – AfterBefores – AftersUp – DownUps – DownsLine – Line UpLines – Line UpsOn – OffOpen – CloseTop – BottomMax(imize) – Min(imize)

Most Common HMG Rules

• Match words used for a command as closely as possible with what the command does

• Be consistent

• Eliminate synonyms

• Follow the way people naturally adjust language to fit a situation

• Follow the order of events

Common Commands

Speech On

Line Copy

3 Before

Window Close

Screen Clear

Keys to Using Speech to Speed and Streamline Computer Control

• Minimizing steps

• Making commands easy to remember

• Enabling combinations

Computer Command Steps

Computer Command Steps

Computer Command Steps

Carnegie Mellon Study

74% of users prefer a structured rather than natural language approach to speech recognition

Speech Graffiti Vs. Natural Language: Assessing the User Experience, Stephanie Tomko and Roni Rosenfeld, Carnegie Mellon University www.cs.cmu.edu/~usi/papers/HLT04.pdf

Command Language Based on Efficient Human Process

• Minimal steps

Command Language Based on Efficient Human Process

• Minimal steps

• Easy-to-remember commands

Command Language Based on Efficient Human Process

• Minimal steps

• Easy-to-remember commands

• Combinations

Beyond the Keyboard and Mouse

• Speech links

• Document access and navigation

• Favorites-like lists

• Virtual calculator

Word Help – Insert a Special Character

Table of Contents Search

Function 1Click Table of ContentsClick Working with TextClick Symbols, Characters…"Click "Insert a symbol or…"Click CloseClick CloseClick Close

Function 1Special CharactersClick "Insert a symbol or..."Click "Insert a special… "Click CloseClick CloseClick Close

8 steps 7 steps

Beyond the Keyboard and Mouse

• Speech links

• Document access and navigation

• Favorites-like lists

• Virtual calculator

[email protected]

www.redstartsystems.com

• Human-Machine Grammar

• Papers and presentations

• Utter Command – speech interface software that works the way your brain does