Telco: Voice-Command Personal Agent Service with AWS Cloud (MBL202) | AWS re:Invent 2013

© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.

Mick Etoh, Ph.D. November 13, 2013

DOCOMO CLOUD

1

Friday, November 15, 13

Who am I?

2


Who am I?• Mick Etoh, SVP, NTT DOCOMO

2


Who am I?• Mick Etoh, SVP, NTT DOCOMO• Role: R&D Strategy

2


Who am I?• Mick Etoh, SVP, NTT DOCOMO• Role: R&D Strategy • Expertise:

2



Pattern Recognition & Machine Learning, Signal Processing & Multimedia, Mobile Networks

2




Fun:

2




Fun:Road Bike, Fly Fishing, and Wine Lover

2


Who is DOCOMO?

3


Who is DOCOMO?

• $61 million subscribers in Japan

3


Who is DOCOMO?

• $61 million subscribers in Japan• $45 billion revenue (2013)

3


Who is DOCOMO?


Next to China Mobile, Verizon, and AT&T

3


Who is DOCOMO?


Next to China Mobile, Verizon, and AT&T

• “i-mode” innovator (1999)

3


Revenue Structure (FY2012)

Device

ServiceCloud

PacketNetwork

VoiceNetwork

Feature Phone EraVoice Stage Data Stage

4


Revenue Structure (FY2012)

Device

ServiceCloud

PacketNetwork

VoiceNetwork

Smart Phone

Voice: $13 Billion

Data: $19 Billion

Service $5.5 Billion

Feature Phone EraVoice Stage Data Stage

4


Wireless Carrier + Service Provider

5


DOCOMO CLOUD Services

6


DOCOMO CLOUD Services

docomo Phonebook docomo Mail Photo Collection databox dmarket

Schedule and Memo ToruCa Hanashite Hon'yaku Utsushite Hon'yaku dmenu

Mail-Honyaku-Concier Shabette-Concier docomo DriveNet i-concier Sugotoku Containt

6


Service Delivery Platform RENEWED

7


Service Delivery Platform RENEWED

DOCOMO CLOUD7


Enterprises using Public Cloud in Japan

8



2.5% (2011)Yano Research

8



7.3% (2012)Yano Research

8


DOCOMO as a Big AWS User

9


DOCOMO as a Big AWS UserFour-DigitEC2 Instances

9


A Telco Story About Launching Voice-Command Personal Agent Service with AWS Cloud

10


Siri

11


“Shabette-Concier” Voice agent service

12


“Shabette-Concier” Voice agent service

• Launched Mar. 1, 2012• Over 40 services in it• Including chatting• 10 million users

Shabette

Voice

=

Concier

Concierge

=How may I help you?

12


13


13


14


14


15


15


An Entrance to Services

16


Train transfer / Last train alarm / Weather alarm / Traffic information / Receipt search / Map search / Weather search / Image search / QA search / Movie search / Music search / News search / Twitter search / Game search / App search / Book/Comic search / Area guide / Browser search / Artist search / TV program search / Dictionary / Horoscope / Result search for soccer game / Result search for baseball game / Result search for golf tournament / Shopping / Calculate calorie of food / Sightseeing search / Call taxi / Knowledge of life / Q&A for DOCOMO / Phone call / Launch camera / Timer / Alarm / Scedular / Memo / Music player / Email / i-Concirge / i-Channel / DOCOMO Map / Disaster kit / Ir communication / Post Twitter / Photo collection / Launch Galary / Translation / Cheat sheet / User support / Flight mode / Day and time / Terminal information / Display setting / Keyboard and input / Storage setting / Language setting / Location service setting / Application listup / Security setting / Sound setting / User dictionary / Wi-Fi setting / Wall paper setting / Battery setting / Communication service setting

An Entrance to Services

37search types

22applications

20functions

16


Monetization

Just say, wanna buy!“d-shopping” : real marketplace

17


Monetization

Just say, wanna buy!“d-shopping” : real marketplace

I want to buywater!

How may I help you?

OK !

17


18


“Status Quo”Shrine

18



“99.9999%”Temple

18



“99.9999%”Temple

Office of Inspirations

18



“99.9999%”Temple


WannabeSkunkworks

18


19


In Status Quo Shrine, so many superstitions....

19


Is this ‘Cloud’?

In Status Quo Shrine, so many superstitions....

19


In 99.9999% Temple,

20


In 99.9999% Temple,Belief: High Availability Forever

20


In 99.9999% Temple,

Downtime per Year 31.5 seconds

Belief: High Availability Forever

20


Three Major Concerns to Public Cloud

Security

Performance

Switching Cost

21


At Office of Inspirations

22


At Office of Inspirations

Make a Magic Lamp right now

with Genie (2009)

22


In Wannabe Skunkworks (2010)

23



Maybe, it’s high time to make

a voice agent.But, as a stealth product.

23





Yep, we have lot of expertise & data for speech recognition.

23






Just out of curiosity,

I’d like to use Public Cloud.

23






Just out of curiosity,

I’d like to use Public Cloud.

23


Distributed Speech Recognition (2006)

First Implementation of ETSI AURORA Project (2000-2003)16 Khz Sampling MFCC feature vectors --> 5.6 Kbps

24


Distributed Speech Recognition (2006)

First Implementation of ETSI AURORA Project (2000-2003)16 Khz Sampling MFCC feature vectors --> 5.6 Kbps

Mobile Terminal

FeatureExtraction

Result

Encode

Server

Decode Rec.Engine

Dictionary

24


Data Mining Team(2009)

In Wannabe Skunkworks

25


Data Mining Team(2009)

Web 2.0 Believers

In Wannabe Skunkworks

25


The paradigm shift from ‘Search’

26


The paradigm shift from ‘Search’

Microsoft Bing

Google Voice Search

26


A Small Team in Wannabe Skunkworks (2010-2012)

☹

AgileDevelopment

Core engineer

Core engineer Developmentpromoter

Developmentpromoter

Product owner

27


A Small Team in Wannabe Skunkworks (2010-2012)

☹

AgileDevelopment

Core engineer

Core engineer Developmentpromoter

Developmentpromoter

Product owner

27


Basic Architecture 2010

28


Basic Architecture 2010(inspired by Microsoft Bing, and Google Voice Search)

28


Basic Architecture 2010

Logging

VoiceRecognition

TaskRecognition

Logging

Voicetext text contents

ServiceProviders’ DB

contents

text

Text to speech

(inspired by Microsoft Bing, and Google Voice Search)

28


5/2011Stealth Product

Data Center

（Hybrid with other Cloud）

History of Voice Agent

29



Data Center


History of Voice Agent3/2012

Mar.～

Version 1

PublicCloud 29



Data Center



June

AWSNC-reg

3/2012

Mar.～

Version 1

PublicCloud 29



Data Center



June

AWSNC-reg

Sept.

11/2012

AWSTokyo-reg.

Version 2

3/2012

Mar.～

Version 1

PublicCloud 29


Speech

Speech Data

Language Data

Machine Learning Machine Learning

AcousticModel

Lexicon LanguageModel

Speech Recognition I want...

Text

Voice Recognition

30


BELIEF: More data usually beats better algorithms.

Speech

Speech Data

Language Data

Machine Learning Machine Learning

AcousticModel

Lexicon LanguageModel

Speech Recognition I want...

Text

Voice Recognition

30


Task Recognition and Service Flow

Find a good Italian restaurant in Palo Alto

Abstractor

Featureextractor

MC-SVMClassifierTokenizer

Query ext.

I want...

Text

Contents

Restaurant

Task CorpusLexiconDictionary

Abstraction Dictionary

31


Task Recognition and Service Flow

PRINCIPLE: Machine learning enhances the service quality. A better quality service acquires more data.

ServiceLauncher

Search Engine A

Search Engine BSearch Engine C

Find a good Italian restaurant in Palo Alto

Abstractor

Featureextractor

MC-SVMClassifierTokenizer

Query ext.

I want...

Text

Contents

Restaurant

Task CorpusLexiconDictionary

Abstraction Dictionary

31


Version 1 Implementation (Mar. 2012)

☹

Started with two local cloud providers in Japan, and soon faced two difficulties:

32



☹

A

LB

B


32



☹

A

LB

B


• Scalability in server count

32



☹

A

LB

B


• Scalability in server count• Inflexible monthly

payment32


Scrambling•Mar 2012 0.5M accesses/day

•Apr 2012 1.5M accesses/day

•Jul 2012 2.5M accesses/day

33


Scrambling•Mar 2012 0.5M accesses/day

•Apr 2012 1.5M accesses/day

•Jul 2012 2.5M accesses/day

33


Moved to AWS (June, 2012)

34



Scale!

34


Technology Platform Leader

Innovative Ecosystem

Global User Footprint

AWS Partners


Scale!

34


System Architecture (June 2012)

Availability Zone #1

SmartPhone

ManagementServer

Log Server for VR Availability Zone #2

Voice Recognizer(VR) Task Recognizer(TR) Log management system

Same as AZ #1

TR Servers

ELB (across multiple zones) Tokenizer Access Log Servers

Availability Zone #3

VPC

VR Servers

LBELB(across multiple zones)

ELB(across multiple zones)

35


Killer Design Pattern: Multi-Data Center

Availability Zone #1 Availability Zone #2

LBLB

VR servers

Route 53

m2.4xlarge x ~300 !!

HAProxy

Voice

VPC

VR servers

Voice Recognition Part

36


DOCOMO Skunkworks: Cloud Natives• For another service,

adopted 21 out of 48 design patterns.

• Among them, typically used ‘Queuing Chain’ and ‘Scale Out’ Pa4erns.

37


Other Crucial Design Patterns

38



•Scheduled Scale Out

38




38




•Clone Server

38


Traffic Pattern

0:00 1:00 2:00 3:00 4:00 5:00 6:00 7:00 8:00 9:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00

Normal traffic

39


Traffic Pattern

0:00 1:00 2:00 3:00 4:00 5:00 6:00 7:00 8:00 9:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00

Normal traffic

39


Bursty traffic for which Auto-Scale failsTV commercial causes a spike traffic.

40


0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23


30x traffic

# of servers

40


0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23


30x traffic

# of servers

20:38 20:40 20:43 20:45

10min

40


Unexpected TrafficScheduled

–TV Commercial–Introduction by TV program

fortunately pre-scheduled!

Suddenly happened–Earthquake–New Feature Release

burst

burst, but gradually tapering

41


Prepare for Launching Servers

Refresh AMI every day–VR model has been changed everyday

Launch AMI for bursty traffic

–10 min - 30 min to launch•Loading VR model is taking time....

42


More intelligence

24hours

43


More intelligence

Optimized server deployment using Elastic Map Reduce (EMR) for pattern analysis

24hours

43


Moved twice • Apr 2012–Start to plan for moving-out to AWS

• Jun 2012–Moved to AWS Northern California

• Tokyo did not have enough instances at that time

• Deploy mixture instance types (c1.xlarge and m2.4xlarge)

• Sep 2012–Moved to AWS Tokyo

• Unified to m2.4xlarge–Improved latency

44


“99.9999%”Temple



DOCOMOSkunkworks

October, 2012

(not ‘Wannabe’ anymore)

45


Kid, your game is over.

“99.9999%”Temple



DOCOMOSkunkworks

October, 2012


45



Use our stable system.

“99.9999%”Temple



DOCOMOSkunkworks

October, 2012


45




Move onquickly! “99.9999%”

Temple



DOCOMOSkunkworks

October, 2012


45




Move onquickly!

Catch meif you can SCALE!

“99.9999%”Temple



DOCOMOSkunkworks

October, 2012


45


In DOCOMO Skunkworks (2013)

46



They are SLOW and pricy. Keep Restless

improvement and our lean-startup culture.

46





Yep, let’s continue our system

improvement on the fly.46






improvement on the fly.

Hey, leader,how about multi-lingual

translation services?

46






improvement on the fly.

Hey, leader,how about multi-lingual

translation services?

46


Now, we have …

• 10 million terminals pre-installed• More than 4M distinct user accesses• 1M accesses / day

17-Nov-16 27-Nov-16 7-Dec-16 17-Dec-16 27-Dec-16 6-Jan-17 16-Jan-17 26-Jan-17 5-Feb-17 15-Feb-17 25-Feb-17 7-Mar-17 17-Mar-17 27-Mar-17 6-Apr-17 16-Apr-17 26-Apr-17 6-May-17 16-May-17 26-May-17 5-Jun-17 15-Jun-17 25-Jun-17 5-Jul-17 15-Jul-17 25-Jul-17 4-Aug-17 14-Aug-17 24-Aug-1717-Nov-12Number of Speech

47


Other Tips•Instance Management✓Simple DB as a key-value

store

•Log management✓Enormous logs from servers✓Direct write to S3

•Multi-AZ✓HA and act-act configuration

48


Thanks to the people who gave great lessons to ‘DOCOMO Skunkworks.’

49



in changing our development style to

49



in changing our development style to“Deploy first, think later,”

49




49




in bringing us a high performance culture, and

49





49





in becoming “Cloud Natives.”

49


Please give us your feedback on this presentation

As a thank you, we will select prize winners daily for completed surveys!

MBL202 Thank You

50


Telco: Voice-Command Personal Agent Service with AWS Cloud (MBL202) | AWS re:Invent 2013

Technology

etsi aurora

status quo

gave great

good italian

mobile networks

lingual translation

local cloud

google voice