Adaptive Input — Breaking Development Conference, San Diego

Jason Grigsby • @grigs • cloudfour.com

Adaptive Inputhttp://www.flickr.com/photos/cobalt/7217055290/

Tweets: @grigs_talksSlides: http://bit.ly/grigs-2013-07-22

http://www.flickr.com/photos/cobalt/7217055290/

http://www.flickr.com/photos/cdm/51747860/

http://www.flickr.com/photos/rheaney/4397863376

It started with TVs.

Designing for a 10-foot UI is very different.

http://www.flickr.com/photos/chrisbartow/5835428673

Larger text and fewer words.

Make up, down, left, right directions clear.

http://images.dailytech.com/nimage/29122_large_amazon_prime_screen_5.jpg

How do we know what is a TV?

This is HDTV.

This is HDTV.1980 px

1080

px

Resolution does not define the optimal

experience.

Next came responsive web apps.

https://twitter.com/freediverx/status/354698695041744896

How do I make this responsive?How do I make this responsive?

mobiledesktop

THE ART OF WEB DEVELOPMENTTHE ART OF WEB DEVELOPMENT

Web widgetsTHE ART OF WEB DEVELOPMENTTHE ART OF WEB DEVELOPMENT

Mobile widgets

It’s not that we’re technically incapable, but adapting a phone UI to a tablet UI is not so dissimilar from trying to automatically adapt desktop UI to a phone. They are fundamentally different platforms with different usability considerations, and something that makes sense on phones may or may not belong on tablets.

—Todd Anglin, Kendo UI

http://www.kendoui.com/blogs/teamblog/posts/12-09-11/universal_mobile_apps_with_html5_and_kendo_ui.aspx

Sometimes it’s hard to envision a responsive version.

http://demos.kendoui.com/web/grid/editing.html

http://www.flickr.com/photos/jesuspresley/384080245/

We want people to be productive…

and stay in the zone.

http://www.flickr.com/photos/raccatography/8038855203

http://www.flickr.com/photos/shantellmartin/4543010568

Which seems very different from playing on an iPad.

For both the TV…

and the desktop web app…

input matters much more than screen size.

The grid is important to support d-pad interaction.

http://images.dailytech.com/nimage/29122_large_amazon_prime_screen_5.jpg

http://www.flickr.com/photos/royalsapien/2387707860

And keyboard and mouse are what we envision work is.http://www.flickr.com/photos/royalsapien/2387707860

http://www.flickr.com/photos/hellogeri/6154034099/

Two years ago at BDConf, Jeremy talked about how…

http://www.flickr.com/photos/60415054@N00/14301113/

we told ourselves that the web was…

http://www.flickr.com/photos/60415054@N00/14301113/

http://www.flickr.com/photos/60415054@N00/14301113/

640 px

480

px

640 px

480

px

1024 px

768

px

http://www.flickr.com/photos/adactio/6153481666/

http://www.flickr.com/photos/adactio/6153481666/

Then mobile came and made us realize…

that it was a collective hallucination all along.

http://www.flickr.com/photos/garibaldi/303085857/

The web never had a fixed canvas.

http://www.flickr.com/photos/paulocarrillo/124755065/

Even our tools perpetuate the lie.

http://www.flickr.com/photos/69797234@N06/7203485148/

We’ve made tremendous progress.

But there is another collective hallucination.

http://www.flickr.com/photos/garibaldi/303085857/

=

=

Supports hover and pointer events.

Keyboard and touch.

Even the iPhone can have a keyboard.

Are these laptops or tablets?

Desktop computer with 23” touch screen

Luke nailed it.

http://static.lukew.com/unified_device_design.png

We can no longer make assumptions about input based on screen size or form factor.

And we probably never should have.

http://www.flickr.com/photos/cblue98/7254221968

Input represents a bigger challenge than screen size.http://www.flickr.com/photos/cblue98/7254221968

http://www.flickr.com/photos/taedc/9278192929

And it is changing more rapidly than ever before.

http://www.flickr.com/photos/taedc/9278192929

So let’s take a closer look…

Let’s start with futuristic input.

http://www.flickr.com/photos/jdhancock/3714748769/

http://uncyclopedia.wikia.com/wiki/File:Man_yelling_at_computer.JPG

VOICE

http://uncyclopedia.wikia.com/wiki/File:Man_yelling_at_computer.JPG

VOICE

http://www.98ps.com/viewnews-15222.html

Siri gets all of the hype…

http://www.98ps.com/viewnews-15222.html

but both Microsoft and Google have compelling voice input in their products.

Explore the web on your TV with Internet Explorer for Xbox One. Use your voice to browse your favorite sites with ease. Use Bing to find the best of the web. Or use Xbox SmartGlass on your phone or tablet to control your experience.

http://www.xbox.com/en-US/xboxone/what-it-does

Xbox 360 Voice Control

http://www.youtube.com/watch?v=xsfVJxakDa8

How would web pages change if we had voice control?

Google voice search

You can use speech recognition too.

http://www.google.com/intl/en/chrome/demos/speech.htmlhttp://www.moreawesomeweb.com/demos/speech_translate.html

Web Speech API Specification19 October 2012Editors:

Glen Shires, Google Inc.Hans Wennborg, Google Inc.

Please refer to the errata for this document, which may include some normative corrections.

Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech APICommunity Group under the W3C Community Final Specification Agreement (FSA). A human-readablesummary is available.

AbstractThis specification defines a JavaScript API to enable web developers to incorporate speech recognition andsynthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and touse speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows webpages to control activation and timing and to handle results and alternatives.

Status of This DocumentThis specification was published by the Speech API Community Group. It is not a W3C Standard nor is it onthe W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)other conditions apply. Learn more about W3C Community and Business Groups.

All feedback is welcome.

Table of Contents1 Conformance requirements2 Introduction3 Use Cases4 Security and privacy considerations5 API Description5.1 The SpeechRecognition Interface5.1.1 SpeechRecognition Attributes5.1.2 SpeechRecognition Methods5.1.3 SpeechRecognition Events5.1.4 SpeechRecognitionError5.1.5 SpeechRecognitionAlternative5.1.6 SpeechRecognitionResult5.1.7 SpeechRecognitionList5.1.8 SpeechRecognitionEvent5.1.9 SpeechGrammar5.1.10 SpeechGrammarList5.2 The SpeechSynthesis Interface5.2.1 SpeechSynthesis Attributes5.2.2 SpeechSynthesis Methods5.2.3 SpeechSynthesisUtterance Attributes5.2.4 SpeechSynthesisUtterance Events5.2.5 SpeechSynthesisEvent Attributes5.2.6 SpeechSynthesisVoice5.2.7 SpeechSynthesisVoiceList6 Examples6.1 Speech Recognition Examples6.2 Speech Synthesis ExamplesAcknowledgmentsReferences

1 Conformance requirementsAll diagrams, examples, and notes in this specification are non-normative, as are all sections explicitly marked

Web Speech API Specification19 October 2012Editors:

Glen Shires, Google Inc.Hans Wennborg, Google Inc.

Please refer to the errata for this document, which may include some normative corrections.

Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech APICommunity Group under the W3C Community Final Specification Agreement (FSA). A human-readablesummary is available.

AbstractThis specification defines a JavaScript API to enable web developers to incorporate speech recognition andsynthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and touse speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows webpages to control activation and timing and to handle results and alternatives.

Status of This DocumentThis specification was published by the Speech API Community Group. It is not a W3C Standard nor is it onthe W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)other conditions apply. Learn more about W3C Community and Business Groups.

All feedback is welcome.

Table of Contents1 Conformance requirements2 Introduction3 Use Cases4 Security and privacy considerations5 API Description5.1 The SpeechRecognition Interface5.1.1 SpeechRecognition Attributes5.1.2 SpeechRecognition Methods5.1.3 SpeechRecognition Events5.1.4 SpeechRecognitionError5.1.5 SpeechRecognitionAlternative5.1.6 SpeechRecognitionResult5.1.7 SpeechRecognitionList5.1.8 SpeechRecognitionEvent5.1.9 SpeechGrammar5.1.10 SpeechGrammarList5.2 The SpeechSynthesis Interface5.2.1 SpeechSynthesis Attributes5.2.2 SpeechSynthesis Methods5.2.3 SpeechSynthesisUtterance Attributes5.2.4 SpeechSynthesisUtterance Events5.2.5 SpeechSynthesisEvent Attributes5.2.6 SpeechSynthesisVoice5.2.7 SpeechSynthesisVoiceList6 Examples6.1 Speech Recognition Examples6.2 Speech Synthesis ExamplesAcknowledgmentsReferences

1 Conformance requirementsAll diagrams, examples, and notes in this specification are non-normative, as are all sections explicitly marked

Web Speech API Specification19 October 2012Editors:

Glen Shires, Google Inc.Hans Wennborg, Google Inc.

Please refer to the errata for this document, which may include some normative corrections.

Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech APICommunity Group under the W3C Community Final Specification Agreement (FSA). A human-readablesummary is available.

AbstractThis specification defines a JavaScript API to enable web developers to incorporate speech recognition andsynthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and touse speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows webpages to control activation and timing and to handle results and alternatives.

Status of This DocumentThis specification was published by the Speech API Community Group. It is not a W3C Standard nor is it onthe W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)other conditions apply. Learn more about W3C Community and Business Groups.

All feedback is welcome.

Table of Contents1 Conformance requirements2 Introduction3 Use Cases4 Security and privacy considerations5 API Description5.1 The SpeechRecognition Interface5.1.1 SpeechRecognition Attributes5.1.2 SpeechRecognition Methods5.1.3 SpeechRecognition Events5.1.4 SpeechRecognitionError5.1.5 SpeechRecognitionAlternative5.1.6 SpeechRecognitionResult5.1.7 SpeechRecognitionList5.1.8 SpeechRecognitionEvent5.1.9 SpeechGrammar5.1.10 SpeechGrammarList5.2 The SpeechSynthesis Interface5.2.1 SpeechSynthesis Attributes5.2.2 SpeechSynthesis Methods5.2.3 SpeechSynthesisUtterance Attributes5.2.4 SpeechSynthesisUtterance Events5.2.5 SpeechSynthesisEvent Attributes5.2.6 SpeechSynthesisVoice5.2.7 SpeechSynthesisVoiceList6 Examples6.1 Speech Recognition Examples6.2 Speech Synthesis ExamplesAcknowledgmentsReferences

1 Conformance requirementsAll diagrams, examples, and notes in this specification are non-normative, as are all sections explicitly marked

Web Speech API Specification19 October 2012Editors:

Glen Shires, Google Inc.Hans Wennborg, Google Inc.

Please refer to the errata for this document, which may include some normative corrections.

Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech APICommunity Group under the W3C Community Final Specification Agreement (FSA). A human-readablesummary is available.

AbstractThis specification defines a JavaScript API to enable web developers to incorporate speech recognition andsynthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and touse speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows webpages to control activation and timing and to handle results and alternatives.

Status of This DocumentThis specification was published by the Speech API Community Group. It is not a W3C Standard nor is it onthe W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)other conditions apply. Learn more about W3C Community and Business Groups.

All feedback is welcome.

Table of Contents1 Conformance requirements2 Introduction3 Use Cases4 Security and privacy considerations5 API Description5.1 The SpeechRecognition Interface5.1.1 SpeechRecognition Attributes5.1.2 SpeechRecognition Methods5.1.3 SpeechRecognition Events5.1.4 SpeechRecognitionError5.1.5 SpeechRecognitionAlternative5.1.6 SpeechRecognitionResult5.1.7 SpeechRecognitionList5.1.8 SpeechRecognitionEvent5.1.9 SpeechGrammar5.1.10 SpeechGrammarList5.2 The SpeechSynthesis Interface5.2.1 SpeechSynthesis Attributes5.2.2 SpeechSynthesis Methods5.2.3 SpeechSynthesisUtterance Attributes5.2.4 SpeechSynthesisUtterance Events5.2.5 SpeechSynthesisEvent Attributes5.2.6 SpeechSynthesisVoice5.2.7 SpeechSynthesisVoiceList6 Examples6.1 Speech Recognition Examples6.2 Speech Synthesis ExamplesAcknowledgmentsReferences

1 Conformance requirementsAll diagrams, examples, and notes in this specification are non-normative, as are all sections explicitly marked

Web Speech API Specification19 October 2012Editors:

Glen Shires, Google Inc.Hans Wennborg, Google Inc.

Please refer to the errata for this document, which may include some normative corrections.

Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech APICommunity Group under the W3C Community Final Specification Agreement (FSA). A human-readablesummary is available.

AbstractThis specification defines a JavaScript API to enable web developers to incorporate speech recognition andsynthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and touse speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows webpages to control activation and timing and to handle results and alternatives.

Status of This DocumentThis specification was published by the Speech API Community Group. It is not a W3C Standard nor is it onthe W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)other conditions apply. Learn more about W3C Community and Business Groups.

All feedback is welcome.

Table of Contents1 Conformance requirements2 Introduction3 Use Cases4 Security and privacy considerations5 API Description5.1 The SpeechRecognition Interface5.1.1 SpeechRecognition Attributes5.1.2 SpeechRecognition Methods5.1.3 SpeechRecognition Events5.1.4 SpeechRecognitionError5.1.5 SpeechRecognitionAlternative5.1.6 SpeechRecognitionResult5.1.7 SpeechRecognitionList5.1.8 SpeechRecognitionEvent5.1.9 SpeechGrammar5.1.10 SpeechGrammarList5.2 The SpeechSynthesis Interface5.2.1 SpeechSynthesis Attributes5.2.2 SpeechSynthesis Methods5.2.3 SpeechSynthesisUtterance Attributes5.2.4 SpeechSynthesisUtterance Events5.2.5 SpeechSynthesisEvent Attributes5.2.6 SpeechSynthesisVoice5.2.7 SpeechSynthesisVoiceList6 Examples6.1 Speech Recognition Examples6.2 Speech Synthesis ExamplesAcknowledgmentsReferences

1 Conformance requirementsAll diagrams, examples, and notes in this specification are non-normative, as are all sections explicitly marked

Web Speech API Specification19 October 2012Editors:

Glen Shires, Google Inc.Hans Wennborg, Google Inc.

Please refer to the errata for this document, which may include some normative corrections.

Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech APICommunity Group under the W3C Community Final Specification Agreement (FSA). A human-readablesummary is available.

AbstractThis specification defines a JavaScript API to enable web developers to incorporate speech recognition andsynthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and touse speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows webpages to control activation and timing and to handle results and alternatives.

Status of This DocumentThis specification was published by the Speech API Community Group. It is not a W3C Standard nor is it onthe W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)other conditions apply. Learn more about W3C Community and Business Groups.

All feedback is welcome.

Table of Contents1 Conformance requirements2 Introduction3 Use Cases4 Security and privacy considerations5 API Description5.1 The SpeechRecognition Interface5.1.1 SpeechRecognition Attributes5.1.2 SpeechRecognition Methods5.1.3 SpeechRecognition Events5.1.4 SpeechRecognitionError5.1.5 SpeechRecognitionAlternative5.1.6 SpeechRecognitionResult5.1.7 SpeechRecognitionList5.1.8 SpeechRecognitionEvent5.1.9 SpeechGrammar5.1.10 SpeechGrammarList5.2 The SpeechSynthesis Interface5.2.1 SpeechSynthesis Attributes5.2.2 SpeechSynthesis Methods5.2.3 SpeechSynthesisUtterance Attributes5.2.4 SpeechSynthesisUtterance Events5.2.5 SpeechSynthesisEvent Attributes5.2.6 SpeechSynthesisVoice5.2.7 SpeechSynthesisVoiceList6 Examples6.1 Speech Recognition Examples6.2 Speech Synthesis ExamplesAcknowledgmentsReferences

1 Conformance requirementsAll diagrams, examples, and notes in this specification are non-normative, as are all sections explicitly marked

Web Speech API Specification19 October 2012Editors:

Glen Shires, Google Inc.Hans Wennborg, Google Inc.

Please refer to the errata for this document, which may include some normative corrections.

Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech APICommunity Group under the W3C Community Final Specification Agreement (FSA). A human-readablesummary is available.

AbstractThis specification defines a JavaScript API to enable web developers to incorporate speech recognition andsynthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and touse speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows webpages to control activation and timing and to handle results and alternatives.

Status of This DocumentThis specification was published by the Speech API Community Group. It is not a W3C Standard nor is it onthe W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)other conditions apply. Learn more about W3C Community and Business Groups.

All feedback is welcome.

Table of Contents1 Conformance requirements2 Introduction3 Use Cases4 Security and privacy considerations5 API Description5.1 The SpeechRecognition Interface5.1.1 SpeechRecognition Attributes5.1.2 SpeechRecognition Methods5.1.3 SpeechRecognition Events5.1.4 SpeechRecognitionError5.1.5 SpeechRecognitionAlternative5.1.6 SpeechRecognitionResult5.1.7 SpeechRecognitionList5.1.8 SpeechRecognitionEvent5.1.9 SpeechGrammar5.1.10 SpeechGrammarList5.2 The SpeechSynthesis Interface5.2.1 SpeechSynthesis Attributes5.2.2 SpeechSynthesis Methods5.2.3 SpeechSynthesisUtterance Attributes5.2.4 SpeechSynthesisUtterance Events5.2.5 SpeechSynthesisEvent Attributes5.2.6 SpeechSynthesisVoice5.2.7 SpeechSynthesisVoiceList6 Examples6.1 Speech Recognition Examples6.2 Speech Synthesis ExamplesAcknowledgmentsReferences

1 Conformance requirementsAll diagrams, examples, and notes in this specification are non-normative, as are all sections explicitly marked

Web Speech API Specification19 October 2012Editors:

Glen Shires, Google Inc.Hans Wennborg, Google Inc.

Please refer to the errata for this document, which may include some normative corrections.

Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech APICommunity Group under the W3C Community Final Specification Agreement (FSA). A human-readablesummary is available.

AbstractThis specification defines a JavaScript API to enable web developers to incorporate speech recognition andsynthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and touse speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows webpages to control activation and timing and to handle results and alternatives.

Status of This DocumentThis specification was published by the Speech API Community Group. It is not a W3C Standard nor is it onthe W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)other conditions apply. Learn more about W3C Community and Business Groups.

All feedback is welcome.

Table of Contents1 Conformance requirements2 Introduction3 Use Cases4 Security and privacy considerations5 API Description5.1 The SpeechRecognition Interface5.1.1 SpeechRecognition Attributes5.1.2 SpeechRecognition Methods5.1.3 SpeechRecognition Events5.1.4 SpeechRecognitionError5.1.5 SpeechRecognitionAlternative5.1.6 SpeechRecognitionResult5.1.7 SpeechRecognitionList5.1.8 SpeechRecognitionEvent5.1.9 SpeechGrammar5.1.10 SpeechGrammarList5.2 The SpeechSynthesis Interface5.2.1 SpeechSynthesis Attributes5.2.2 SpeechSynthesis Methods5.2.3 SpeechSynthesisUtterance Attributes5.2.4 SpeechSynthesisUtterance Events5.2.5 SpeechSynthesisEvent Attributes5.2.6 SpeechSynthesisVoice5.2.7 SpeechSynthesisVoiceList6 Examples6.1 Speech Recognition Examples6.2 Speech Synthesis ExamplesAcknowledgmentsReferences

1 Conformance requirementsAll diagrams, examples, and notes in this specification are non-normative, as are all sections explicitly marked

Web Speech API Specification19 October 2012Editors:

Glen Shires, Google Inc.Hans Wennborg, Google Inc.

Please refer to the errata for this document, which may include some normative corrections.

Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech APICommunity Group under the W3C Community Final Specification Agreement (FSA). A human-readablesummary is available.

AbstractThis specification defines a JavaScript API to enable web developers to incorporate speech recognition andsynthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and touse speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows webpages to control activation and timing and to handle results and alternatives.

Status of This DocumentThis specification was published by the Speech API Community Group. It is not a W3C Standard nor is it onthe W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)other conditions apply. Learn more about W3C Community and Business Groups.

All feedback is welcome.

Table of Contents1 Conformance requirements2 Introduction3 Use Cases4 Security and privacy considerations5 API Description5.1 The SpeechRecognition Interface5.1.1 SpeechRecognition Attributes5.1.2 SpeechRecognition Methods5.1.3 SpeechRecognition Events5.1.4 SpeechRecognitionError5.1.5 SpeechRecognitionAlternative5.1.6 SpeechRecognitionResult5.1.7 SpeechRecognitionList5.1.8 SpeechRecognitionEvent5.1.9 SpeechGrammar5.1.10 SpeechGrammarList5.2 The SpeechSynthesis Interface5.2.1 SpeechSynthesis Attributes5.2.2 SpeechSynthesis Methods5.2.3 SpeechSynthesisUtterance Attributes5.2.4 SpeechSynthesisUtterance Events5.2.5 SpeechSynthesisEvent Attributes5.2.6 SpeechSynthesisVoice5.2.7 SpeechSynthesisVoiceList6 Examples6.1 Speech Recognition Examples6.2 Speech Synthesis ExamplesAcknowledgmentsReferences

1 Conformance requirementsAll diagrams, examples, and notes in this specification are non-normative, as are all sections explicitly marked

Web Speech API Specification19 October 2012Editors:

Glen Shires, Google Inc.Hans Wennborg, Google Inc.

Please refer to the errata for this document, which may include some normative corrections.

Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech APICommunity Group under the W3C Community Final Specification Agreement (FSA). A human-readablesummary is available.

AbstractThis specification defines a JavaScript API to enable web developers to incorporate speech recognition andsynthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and touse speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows webpages to control activation and timing and to handle results and alternatives.

Status of This DocumentThis specification was published by the Speech API Community Group. It is not a W3C Standard nor is it onthe W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA)other conditions apply. Learn more about W3C Community and Business Groups.

All feedback is welcome.

Table of Contents1 Conformance requirements2 Introduction3 Use Cases4 Security and privacy considerations5 API Description5.1 The SpeechRecognition Interface5.1.1 SpeechRecognition Attributes5.1.2 SpeechRecognition Methods5.1.3 SpeechRecognition Events5.1.4 SpeechRecognitionError5.1.5 SpeechRecognitionAlternative5.1.6 SpeechRecognitionResult5.1.7 SpeechRecognitionList5.1.8 SpeechRecognitionEvent5.1.9 SpeechGrammar5.1.10 SpeechGrammarList5.2 The SpeechSynthesis Interface5.2.1 SpeechSynthesis Attributes5.2.2 SpeechSynthesis Methods5.2.3 SpeechSynthesisUtterance Attributes5.2.4 SpeechSynthesisUtterance Events5.2.5 SpeechSynthesisEvent Attributes5.2.6 SpeechSynthesisVoice5.2.7 SpeechSynthesisVoiceList6 Examples6.1 Speech Recognition Examples6.2 Speech Synthesis ExamplesAcknowledgmentsReferences

1 Conformance requirementsAll diagrams, examples, and notes in this specification are non-normative, as are all sections explicitly marked

Special thanks to Eric Bidelmanhttp://moreawesomeweb.com

Speech Recognition API Support

?

Gestures?

Amazing, but too new to know what, if anything, this technology

will mean for the web.

Let’s come back from the future and look at something much Dumber.Dumber

Dumber

Dumber

-pad remote controlsD

-pad remote controlsD

-pad remote controlsD

http://www.flickr.com/photos/stewc/6669743035/

http://www.flickr.com/photos/stewc/6669743035/

TVs browsers that support d-pad, send arrow key events.

http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/

If then

http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/

is undetectable. This is a recurring theme for input.

Sensors and camera

http://www.flickr.com/photos/retrocactus/2170677056

Sensors and camera

Camera

http://www.flickr.com/photos/retrocactus/2170677056

GPS

http://www.flickr.com/photos/3dking/149450434

GPS

GeoLocation

http://www.flickr.com/photos/3dking/149450434

Gyroscope & Accelerometer

http://www.flickr.com/photos/chrisjagers/4694134078

Back to today’s problems.

http://www.flickr.com/photos/chrisjagers/4694134078

Hover state No hover state

Hover state

Typing easier for many

No hover state

Typing often more difficult

Higher precision with mouse means smaller targets possible

Hover state

Less precise than mouse and requires larger touch targets

Typing easier for many

No hover state

Typing often more difficult

Higher precision with mouse means smaller targets possible

Hover state

Less precise than mouse and requires larger touch targets

Typing easier for many

No hover state

Typing often more difficult

Right clicking and “power” tools Single and multi-touch gestures

http://www.flickr.com/photos/28096801@N05/5012309802

I got this. Detect touch.http://www.flickr.com/photos/28096801@N05/5012309802

Whatever you may think, it currently isn't possible to reliably detect whether or not the current device has a touchscreen, from within the browser.

—Stu Cox, You Can’t Reliably Detect a Touch Screen

http://www.stucox.com/blog/you-cant-detect-a-touchscreen/

Chrome would like to turn on touch events by default.

https://code.google.com/p/chromium/issues/detail?id=159527https://docs.google.com/a/cloudfour.com/presentation/d/1-n1qyzewpagREbzW2zm0wOalq33UhbtbSkWf9mEdly8/edit#slide=id.gc2d80e5b_171

Detect a mouse? Not reliably.

Surely we can detect a keyboard?

Surely we can detect a keyboard?

NOPE

Input is dynamic.

Input is dynamic.

Boris Smus’s experiments responding to input.

Boris Smus’s experiments responding to input.

http://www.flickr.com/photos/lyza/7382235106

Maybe we need to be more zen about input.

After poking at this problem for a few weeks, my conclusion is: every desktop UI should be designed for touch now. When any desktop machine could have a touch interface, we have to proceed as if they all do.

—Josh Clark

http://globalmoxie.com/blog/desktop-touch-design.shtml

http://ie.microsoft.com/testdrive/ieblog/2011/Sep/20_TouchInputforIE10andMetrostyleApps_1.pnghttp://www.w3.org/TR/pointerevents/

http://blog.webplatform.org/2013/02/pointing-toward-the-future/

New Pointer Events spec normalizes touch and mouse

Pointer Events builds on the DOM event model to offer a new way to handle input

on the web, enabling developers to build

touch-first experiences that work with mouse, pen, and other pointing devices as

well…They are also designed from the ground up to allow modern browsers to

accelerate the touch-surface performance,

leading to a smoother user experience.

What about those who won’t let go of their “power” interfaces?

http://www.flickr.com/photos/ecos/4092571213/

http://www.flickr.com/photos/scarygami/5689980135/

One option: give them a choice.

Gmail display density settings

Couch Mode+ See allCentric TV’s videos / Recently viewed / Related videos

Th Dream Experience - …Uploaded 2 years ago

More Of The Dream Exp…Uploaded 2 years ago

The Dream Experience -…Uploaded 2 years ago

The Dream Experience …Uploaded 2 years ago

The Love King Breaks It…Uploaded 2 years ago

PROMOTED

War Paint for TreesFrom Lincoln Motor Company

TM + © 2013 Vimeo, LLC. All rights reserved. Terms • Privacy • Copyright • Cookies • Site Map Made with k in NYC. Language: English

Join Log In Create Watch Upload

3 CommentsDan Thil2 months ago

craigblaze7 months ago

Denis Rozenkov1 year ago

4

4

4

4

L Like this video S Share this videoF Full screen M More videos

26 LIKES

+ See all

RELATED COLLECTIONS

Aloe Blacc • Stones Throw

ALSO CHECK OUTMore stuff from Centric TV

399 Videos

1 Like

5 Channels

9 Following

NEED HELP?If you have questions about what’s on this page, lookhere first: Help / FAQKeyboard Shortcuts

SOULSESSIONS // Aloe Blaccfrom Centric TV PROPRO 2 years ago NOT YET RATEDNOT YET RATED

This conversation is missing your voice. Please join Vimeo or log in.

visualhybrid 2 years ago

Man, totally hypnotic song..

A.A. 1 year ago

The opening vocalise makes me shiver! Great voice and interpretation and the rest of theband really swings!

rossangeles 1 year ago

Directed, shot and edited by Ross Harris.

Vimeo

About Vimeo

Vimeo Blog

Vimeo Guidelines

Developers

Jobs

Help

Help Center

Basics

Video School

FAQ

Forums

Upgrade

Vimeo Plus

Vimeo PRO

Vimeo for Business

UploadUpload multiple videos atfaster speeds with ournew uploader.

Did you know?Hubnut lets you show multiple videos in arow, so people can keep watching. Andwatching. And watching.

1

3 Follow Stats 9 Switch to HTML5 Player

Search s

[ ]

– VIDEO

SVimeo Couch Mode

Couch Mode+ See allCentric TV’s videos / Recently viewed / Related videos

Th Dream Experience - …Uploaded 2 years ago

More Of The Dream Exp…Uploaded 2 years ago

The Dream Experience -…Uploaded 2 years ago

The Dream Experience …Uploaded 2 years ago

The Love King Breaks It…Uploaded 2 years ago

PROMOTED

War Paint for TreesFrom Lincoln Motor Company

TM + © 2013 Vimeo, LLC. All rights reserved. Terms • Privacy • Copyright • Cookies • Site Map Made with k in NYC. Language: English

Join Log In Create Watch Upload

3 CommentsDan Thil2 months ago

craigblaze7 months ago

Denis Rozenkov1 year ago

4

4

4

4

L Like this video S Share this videoF Full screen M More videos

26 LIKES

+ See all

RELATED COLLECTIONS

Aloe Blacc • Stones Throw

ALSO CHECK OUTMore stuff from Centric TV

399 Videos

1 Like

5 Channels

9 Following

NEED HELP?If you have questions about what’s on this page, lookhere first: Help / FAQKeyboard Shortcuts

SOULSESSIONS // Aloe Blaccfrom Centric TV PROPRO 2 years ago NOT YET RATEDNOT YET RATED

This conversation is missing your voice. Please join Vimeo or log in.

visualhybrid 2 years ago

Man, totally hypnotic song..

A.A. 1 year ago

The opening vocalise makes me shiver! Great voice and interpretation and the rest of theband really swings!

rossangeles 1 year ago

Directed, shot and edited by Ross Harris.

Vimeo

About Vimeo

Vimeo Blog

Vimeo Guidelines

Developers

Jobs

Help

Help Center

Basics

Video School

FAQ

Forums

Upgrade

Vimeo Plus

Vimeo PRO

Vimeo for Business

UploadUpload multiple videos atfaster speeds with ournew uploader.

Did you know?Hubnut lets you show multiple videos in arow, so people can keep watching. Andwatching. And watching.

1

3 Follow Stats 9 Switch to HTML5 Player

Search s

[ ]

– VIDEO

SVimeo Couch Mode

Couch Mode+ See allCentric TV’s videos / Recently viewed / Related videos

Th Dream Experience - …Uploaded 2 years ago

More Of The Dream Exp…Uploaded 2 years ago

The Dream Experience -…Uploaded 2 years ago

The Dream Experience …Uploaded 2 years ago

The Love King Breaks It…Uploaded 2 years ago

PROMOTED

War Paint for TreesFrom Lincoln Motor Company

TM + © 2013 Vimeo, LLC. All rights reserved. Terms • Privacy • Copyright • Cookies • Site Map Made with k in NYC. Language: English

Join Log In Create Watch Upload

3 CommentsDan Thil2 months ago

craigblaze7 months ago

Denis Rozenkov1 year ago

4

4

4

4

L Like this video S Share this videoF Full screen M More videos

26 LIKES

+ See all

RELATED COLLECTIONS

Aloe Blacc • Stones Throw

ALSO CHECK OUTMore stuff from Centric TV

399 Videos

1 Like

5 Channels

9 Following

NEED HELP?If you have questions about what’s on this page, lookhere first: Help / FAQKeyboard Shortcuts

SOULSESSIONS // Aloe Blaccfrom Centric TV PROPRO 2 years ago NOT YET RATEDNOT YET RATED

This conversation is missing your voice. Please join Vimeo or log in.

visualhybrid 2 years ago

Man, totally hypnotic song..

A.A. 1 year ago

The opening vocalise makes me shiver! Great voice and interpretation and the rest of theband really swings!

rossangeles 1 year ago

Directed, shot and edited by Ross Harris.

Vimeo

About Vimeo

Vimeo Blog

Vimeo Guidelines

Developers

Jobs

Help

Help Center

Basics

Video School

FAQ

Forums

Upgrade

Vimeo Plus

Vimeo PRO

Vimeo for Business

UploadUpload multiple videos atfaster speeds with ournew uploader.

Did you know?Hubnut lets you show multiple videos in arow, so people can keep watching. Andwatching. And watching.

1

3 Follow Stats 9 Switch to HTML5 Player

Search s

[ ]

– VIDEO

S

Couch Mode+ See allCentric TV’s videos / Recently viewed / Related videos

Th Dream Experience - …Uploaded 2 years ago

More Of The Dream Exp…Uploaded 2 years ago

The Dream Experience -…Uploaded 2 years ago

The Dream Experience …Uploaded 2 years ago

The Love King Breaks It…Uploaded 2 years ago

PROMOTED

War Paint for TreesFrom Lincoln Motor Company

TM + © 2013 Vimeo, LLC. All rights reserved. Terms • Privacy • Copyright • Cookies • Site Map Made with k in NYC. Language: English

Join Log In Create Watch Upload

3 CommentsDan Thil2 months ago

craigblaze7 months ago

Denis Rozenkov1 year ago

4

4

4

4

L Like this video S Share this videoF Full screen M More videos

26 LIKES

+ See all

RELATED COLLECTIONS

Aloe Blacc • Stones Throw

ALSO CHECK OUTMore stuff from Centric TV

399 Videos

1 Like

5 Channels

9 Following

NEED HELP?If you have questions about what’s on this page, lookhere first: Help / FAQKeyboard Shortcuts

SOULSESSIONS // Aloe Blaccfrom Centric TV PROPRO 2 years ago NOT YET RATEDNOT YET RATED

This conversation is missing your voice. Please join Vimeo or log in.

visualhybrid 2 years ago

Man, totally hypnotic song..

A.A. 1 year ago

The opening vocalise makes me shiver! Great voice and interpretation and the rest of theband really swings!

rossangeles 1 year ago

Directed, shot and edited by Ross Harris.

Vimeo

About Vimeo

Vimeo Blog

Vimeo Guidelines

Developers

Jobs

Help

Help Center

Basics

Video School

FAQ

Forums

Upgrade

Vimeo Plus

Vimeo PRO

Vimeo for Business

UploadUpload multiple videos atfaster speeds with ournew uploader.

Did you know?Hubnut lets you show multiple videos in arow, so people can keep watching. Andwatching. And watching.

1

3 Follow Stats 9 Switch to HTML5 Player

Search s

[ ]

– VIDEO

SVimeo Couch Mode

Vimeo Couch Mode

The key benefit of this approach:

You’re designing for user need not for a specific form factor or input.

http://www.flickr.com/photos/raver_mikey/504815463

Progressive Input?

Graph from Chapter 1 of Adaptive Web Design by Aaron Gustafson http://easy-readers.net/books/adaptive-web-design/

Graph from Chapter 1 of Adaptive Web Design by Aaron Gustafson http://easy-readers.net/books/adaptive-web-design/

Progressive enhancement contains a value judgment

Who are we to judge what input is better?

http://www.flickr.com/photos/fensterbme/4783366926

We need to learn to adapt.

http://www.flickr.com/photos/cdm/147947664/

So that’s the puzzle that has been haunting me.

http://www.flickr.com/photos/sravishankar/3460495

http://www.flickr.com/photos/wwworks/1384952210

Thank You!Special thanks to Luke Wroblewski, Eric Bidelman and Flickr users for

generously sharing their photos under creative commons license.