Telco: Voice-Command Personal Agent Service with AWS Cloud (MBL202) | AWS re:Invent 2013

Mick Etoh, Ph.D. November 13, 2013

DOCOMO CLOUD

Friday, November 15, 13

Who am I?

Who am I?• Mick Etoh, SVP, NTT DOCOMO

Who am I?• Mick Etoh, SVP, NTT DOCOMO• Role: R&D Strategy

Who am I?• Mick Etoh, SVP, NTT DOCOMO• Role: R&D Strategy • Expertise:

Pattern Recognition & Machine Learning, Signal Processing & Multimedia, Mobile Networks

Fun:Road Bike, Fly Fishing, and Wine Lover

Who is DOCOMO?

• $61 million subscribers in Japan

Who is DOCOMO?

• $61 million subscribers in Japan• $45 billion revenue (2013)

Who is DOCOMO?

Next to China Mobile, Verizon, and AT&T

Who is DOCOMO?

Next to China Mobile, Verizon, and AT&T

• “i-mode” innovator (1999)

Revenue Structure (FY2012)

Device

ServiceCloud

PacketNetwork

VoiceNetwork

Feature Phone EraVoice Stage Data Stage

Revenue Structure (FY2012)

Device

ServiceCloud

PacketNetwork

VoiceNetwork

Smart Phone

Voice: $13 Billion

Data: $19 Billion

Service $5.5 Billion

Feature Phone EraVoice Stage Data Stage

Wireless Carrier + Service Provider

DOCOMO CLOUD Services

docomo Phonebook docomo Mail Photo Collection databox dmarket

Schedule and Memo ToruCa Hanashite Hon'yaku Utsushite Hon'yaku dmenu

Mail-Honyaku-Concier Shabette-Concier docomo DriveNet i-concier Sugotoku Containt

Service Delivery Platform RENEWED

DOCOMO CLOUD7

Enterprises using Public Cloud in Japan

2.5% (2011)Yano Research

7.3% (2012)Yano Research

DOCOMO as a Big AWS User

DOCOMO as a Big AWS UserFour-DigitEC2 Instances

A Telco Story About Launching Voice-Command Personal Agent Service with AWS Cloud

“Shabette-Concier” Voice agent service

• Launched Mar. 1, 2012• Over 40 services in it• Including chatting• 10 million users

Shabette

Concier

Concierge

=How may I help you?

An Entrance to Services

Train transfer / Last train alarm / Weather alarm / Traffic information / Receipt search / Map search / Weather search / Image search / QA search / Movie search / Music search / News search / Twitter search / Game search / App search / Book/Comic search / Area guide / Browser search / Artist search / TV program search / Dictionary / Horoscope / Result search for soccer game / Result search for baseball game / Result search for golf tournament / Shopping / Calculate calorie of food / Sightseeing search / Call taxi / Knowledge of life / Q&A for DOCOMO / Phone call / Launch camera / Timer / Alarm / Scedular / Memo / Music player / Email / i-Concirge / i-Channel / DOCOMO Map / Disaster kit / Ir communication / Post Twitter / Photo collection / Launch Galary / Translation / Cheat sheet / User support / Flight mode / Day and time / Terminal information / Display setting / Keyboard and input / Storage setting / Language setting / Location service setting / Application listup / Security setting / Sound setting / User dictionary / Wi-Fi setting / Wall paper setting / Battery setting / Communication service setting

An Entrance to Services

37search types

22applications

20functions

Monetization

Just say, wanna buy!“d-shopping” : real marketplace

Monetization

Just say, wanna buy!“d-shopping” : real marketplace

I want to buywater!

How may I help you?

“Status Quo”Shrine

“99.9999%”Temple

Office of Inspirations

“99.9999%”Temple

WannabeSkunkworks

In Status Quo Shrine, so many superstitions....

Is this ‘Cloud’?

In Status Quo Shrine, so many superstitions....

In 99.9999% Temple,

In 99.9999% Temple,Belief: High Availability Forever

In 99.9999% Temple,

Downtime per Year 31.5 seconds

Belief: High Availability Forever

Three Major Concerns to Public Cloud

Security

Performance

Switching Cost

At Office of Inspirations

Make a Magic Lamp right now

with Genie (2009)

In Wannabe Skunkworks (2010)

Maybe, it’s high time to make

a voice agent.But, as a stealth product.

Yep, we have lot of expertise & data for speech recognition.

Just out of curiosity,

I’d like to use Public Cloud.

Just out of curiosity,

I’d like to use Public Cloud.

Distributed Speech Recognition (2006)

First Implementation of ETSI AURORA Project (2000-2003)16 Khz Sampling MFCC feature vectors --> 5.6 Kbps

Distributed Speech Recognition (2006)

First Implementation of ETSI AURORA Project (2000-2003)16 Khz Sampling MFCC feature vectors --> 5.6 Kbps

Mobile Terminal

FeatureExtraction

Result

Encode

Server

Decode Rec.Engine

Dictionary

Data Mining Team(2009)

In Wannabe Skunkworks

Data Mining Team(2009)

Web 2.0 Believers

In Wannabe Skunkworks

The paradigm shift from ‘Search’

Microsoft Bing

Google Voice Search

A Small Team in Wannabe Skunkworks (2010-2012)

AgileDevelopment

Core engineer

Core engineer Developmentpromoter

Developmentpromoter

Product owner

A Small Team in Wannabe Skunkworks (2010-2012)

AgileDevelopment

Core engineer

Core engineer Developmentpromoter

Developmentpromoter

Product owner

Basic Architecture 2010

Basic Architecture 2010(inspired by Microsoft Bing, and Google Voice Search)

Basic Architecture 2010

Logging

VoiceRecognition

TaskRecognition

Logging

Voicetext text contents

ServiceProviders’ DB

contents

Text to speech

(inspired by Microsoft Bing, and Google Voice Search)

5/2011Stealth Product

Data Center

（Hybrid with other Cloud）

History of Voice Agent

Data Center

History of Voice Agent3/2012

Mar.～

Version 1

PublicCloud 29

Data Center

AWSNC-reg

3/2012

Mar.～

Version 1

PublicCloud 29

Data Center

AWSNC-reg

11/2012

AWSTokyo-reg.

Version 2

3/2012

Mar.～

Version 1

PublicCloud 29

Speech

Speech Data

Language Data

Machine Learning Machine Learning

AcousticModel

Lexicon LanguageModel

Speech Recognition I want...

Voice Recognition

BELIEF: More data usually beats better algorithms.

Speech

Speech Data

Language Data

Machine Learning Machine Learning

AcousticModel

Lexicon LanguageModel

Speech Recognition I want...

Voice Recognition

Task Recognition and Service Flow

Find a good Italian restaurant in Palo Alto

Abstractor

Featureextractor

MC-SVMClassifierTokenizer

Query ext.

I want...

Contents

Restaurant

Task CorpusLexiconDictionary

Abstraction Dictionary

Task Recognition and Service Flow

PRINCIPLE: Machine learning enhances the service quality. A better quality service acquires more data.

ServiceLauncher

Search Engine A

Search Engine BSearch Engine C

Find a good Italian restaurant in Palo Alto

Abstractor

Featureextractor

MC-SVMClassifierTokenizer

Query ext.

I want...

Contents

Restaurant

Task CorpusLexiconDictionary

Abstraction Dictionary

Version 1 Implementation (Mar. 2012)

Started with two local cloud providers in Japan, and soon faced two difficulties:

• Scalability in server count

• Scalability in server count• Inflexible monthly

payment32

Scrambling•Mar 2012 0.5M accesses/day

•Apr 2012 1.5M accesses/day

•Jul 2012 2.5M accesses/day

Scrambling•Mar 2012 0.5M accesses/day

•Apr 2012 1.5M accesses/day

•Jul 2012 2.5M accesses/day

Moved to AWS (June, 2012)

Scale!

Technology Platform Leader

Innovative Ecosystem

Global User Footprint

AWS Partners

Scale!

System Architecture (June 2012)

Availability Zone #1

SmartPhone

ManagementServer

Log Server for VR Availability Zone #2

Voice Recognizer(VR) Task Recognizer(TR) Log management system

Same as AZ #1

TR Servers

ELB (across multiple zones) Tokenizer Access Log Servers

Availability Zone #3

VR Servers

LBELB(across multiple zones)

ELB(across multiple zones)

Killer Design Pattern: Multi-Data Center

Availability Zone #1 Availability Zone #2

VR servers

Route 53

m2.4xlarge x ~300 !!

HAProxy

VR servers

Voice Recognition Part

DOCOMO Skunkworks: Cloud Natives• For another service,

adopted 21 out of 48 design patterns.

• Among them, typically used ‘Queuing Chain’ and ‘Scale Out’ Pa4erns.

Other Crucial Design Patterns

•Scheduled Scale Out

•Clone Server

Traffic Pattern

0:00 1:00 2:00 3:00 4:00 5:00 6:00 7:00 8:00 9:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00

Normal traffic

Traffic Pattern

0:00 1:00 2:00 3:00 4:00 5:00 6:00 7:00 8:00 9:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00

Normal traffic

Bursty traffic for which Auto-Scale failsTV commercial causes a spike traffic.

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

30x traffic

# of servers

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

30x traffic

# of servers

20:38 20:40 20:43 20:45

Unexpected TrafficScheduled

–TV Commercial–Introduction by TV program

fortunately pre-scheduled!

Suddenly happened–Earthquake–New Feature Release

burst, but gradually tapering

Prepare for Launching Servers

Refresh AMI every day–VR model has been changed everyday

Launch AMI for bursty traffic

–10 min - 30 min to launch•Loading VR model is taking time....

More intelligence

24hours

More intelligence

Optimized server deployment using Elastic Map Reduce (EMR) for pattern analysis

24hours

Moved twice • Apr 2012–Start to plan for moving-out to AWS

• Jun 2012–Moved to AWS Northern California

• Tokyo did not have enough instances at that time

• Deploy mixture instance types (c1.xlarge and m2.4xlarge)

• Sep 2012–Moved to AWS Tokyo

• Unified to m2.4xlarge–Improved latency

“99.9999%”Temple

DOCOMOSkunkworks

October, 2012

(not ‘Wannabe’ anymore)

Kid, your game is over.

“99.9999%”Temple

DOCOMOSkunkworks

October, 2012

Use our stable system.

“99.9999%”Temple

DOCOMOSkunkworks

October, 2012

Move onquickly! “99.9999%”

Temple

DOCOMOSkunkworks

October, 2012

Move onquickly!

Catch meif you can SCALE!

“99.9999%”Temple

DOCOMOSkunkworks

October, 2012

In DOCOMO Skunkworks (2013)

They are SLOW and pricy. Keep Restless

improvement and our lean-startup culture.

Yep, let’s continue our system

improvement on the fly.46

improvement on the fly.

Hey, leader,how about multi-lingual

translation services?

improvement on the fly.

Hey, leader,how about multi-lingual

translation services?

Now, we have …

• 10 million terminals pre-installed• More than 4M distinct user accesses• 1M accesses / day

17-Nov-16 27-Nov-16 7-Dec-16 17-Dec-16 27-Dec-16 6-Jan-17 16-Jan-17 26-Jan-17 5-Feb-17 15-Feb-17 25-Feb-17 7-Mar-17 17-Mar-17 27-Mar-17 6-Apr-17 16-Apr-17 26-Apr-17 6-May-17 16-May-17 26-May-17 5-Jun-17 15-Jun-17 25-Jun-17 5-Jul-17 15-Jul-17 25-Jul-17 4-Aug-17 14-Aug-17 24-Aug-1717-Nov-12Number of Speech

Other Tips•Instance Management✓Simple DB as a key-value

•Log management✓Enormous logs from servers✓Direct write to S3

•Multi-AZ✓HA and act-act configuration

Thanks to the people who gave great lessons to ‘DOCOMO Skunkworks.’

in changing our development style to

in changing our development style to“Deploy first, think later,”

in bringing us a high performance culture, and

in becoming “Cloud Natives.”

Please give us your feedback on this presentation

As a thank you, we will select prize winners daily for completed surveys!

MBL202 Thank You

Telco: Voice-Command Personal Agent Service with AWS Cloud (MBL202) | AWS re:Invent 2013

etsi aurora

status quo

gave great

good italian

mobile networks

lingual translation

local cloud

google voice

Technology

(SOV203) Understanding AWS Storage Options | AWS re:Invent.....

AWS re:Invent re:Flection - Spot Pricing

AWS Re:Invent - Optimizing Costs with AWS

(SPOT301) AWS Innovation at Scale | AWS re:Invent 2014

AWS re:Invent 2016: Large-scale AWS Migrations (ENT204)

Navigating AWS re:Invent 2015

Migrating My.T-Mobile.com to AWS (ENT214) | AWS re:Invent...

AWS Re:Invent - Securing HIPAA Compliant Apps in AWS

AWS re:Invent 2016 Fast Forward

Feedback on AWS re:invent 2016

20151207 AWS re:invent 2015 ReCap

AWS re:invent 2016 후기

2014 AWS Re:Invent sharing

Understanding AWS Storage Options (STG101) | AWS re:Invent.....

Overview of Windows on AWS (CPN206) | AWS re:Invent 2013

Aws re:Invent 2016 Security Follow Up AWS Organizations