A dive into Microsoft Strategy on Machine Learning, Chat Bot, and Artificial Intelligence by SeokJin Han, 20170329

A Dive into Microsoft Strategy on Machine

Learning, Chat Bot, and Artificial Intelligence

SeokJin Han

Microsoft

Businesses will require ROI

from AI

• Investment increased 10 times recent 5

years (2011-2016), but commercial cases are

limited

• Drastic changes of views last 2 years

(AI: from enemies to partners)

Faster development on

Conversational Interface

• Game-changing innovations

(AI learns human languages)

• Natural language search from Google and

Bing, DeepText from Facebook (Personal

Pattern Recognition), Changes on Chat

Bots/Digital Assistants/Messenger Apps

Designs evolve to increase

Credibility of AI

• Reflects onto AI design the knowledge on

how human earns credibility between

people

• AI NLP integrated with Communication

components such as tone, emotion, timing,

visual perception, and word selection

Begin discussion on how AIs

will talk to each other

• Protocols between AIs

• How to evade collision between AI

systems operating as silos

• Consider collisions between AI systems of

different purposes

Imbedded bias will be a big

blocker for AI dev

• Cases from Google/Microsoft

• Gender, Racial imbalance

• Different sources of bias

• Training data, user interactions, lack

of diversity, conflicting purposes

InteractionsComputer – Computer

Human – Computer

Human – Human

5 predictions for artificial intelligence in 2017, Stuart Frankel, CEO, Narrative Science

AI discussion in “2017”?

Digital Transformation

Microsoft dedication to AI

• AI and Research group – Organizational

change

• Microsoft Research

• Information Platform Group

• Cortana Engineering

• “Democratizing AI”

• “Partnership on AI”(NPO) – founding

member

• Aggressive investments for Cloud based

Machine Learning, Cognitive Services, Bots

• Most diversified AI portfolio in the market

Agent Applications Services Infrastructure

Cortana Office 365

Dynamics 365

Cortana Intelligence

• Bot Framework

• Cognitive Services

• Cognitive Toolkit

• Azure Machine

Learning

Azure N Series

FPGA

Platform

Approach

Microsoft AI Portfolio

Machine Learning at Microsoft

• Clutter in Office 365Spam Filtering, Infer.Net probability model

• Power BIData visualization in Natural Language

• CortanaVoice recognition/synthesis, Intent/entity extraction

• KinectBehavior recognition from infra-red images

• HololensAugmented Reality

• Windows Phone KeyboardsEmphasizes keys to pick using spell correction history

• Windows TabletEnhances touch recognition quality

• OneNoteEnhances handwriting recognition quality

• Windows Boot time reductionReads frequently used apps in advance

• Microsoft BandHigher measurement accuracies using cheaper sensors

• XBox GamesAI, Ranking System

• Bing / SharePointSearch

• OneDriveAutomatic image tagging / categorization

• Skype TranslatorReal time bi-directional translation

• Project AdamImage recognition : Recognize different dog breeds,

identify toxic plants + more

Cosmos/Scope

Big Data Services running Microsoft

Stored data : 3 EB+

Cluster size : 10 thousands+ nodes

# of machines : 100 thousands

Analyzed data : 150 PB+ / day

Internal analysts : thousands

Analytics jobs : 100s thousands / day

SMSG

Live

STB Commerce RiskLCA

Cortana Intelligence Suite

Transform data into intelligent action

Intelligence

Dashboard /

Visualization

Info Mgmt Big Data Store Machine Learning /

Advanced Analytics

CortanaIoT Hub

Event Hub

HDInsight

(Hadoop and

Spark)

Stream

Analytics

Data Intelligence Action

People

Automated Systems

Apps

Web

Mobile

Bots

Bot

FrameworkSQL Data

WarehouseData Catalog

Data Lake

Analytics

Data Factory Machine

Learning

Data Lake

StoreCognitive

Services

Power BI

Data

Sources

Apps

Sensors

and

devices

Data

FUTURE PROOF ARCHITECTURE

Azure

API

Management

Backend Services

Data sources

Apps

Sensors and devices

Event Hub

IoT Hub

Machine Learning

HDInsight(Apache Spark)Storage

Power BIStream Analytics

SQL Data Warehouse

Azure Data Factory & Azure Data Catalog

Data Lake StoreData Lake Analytics

SQL Server Integration Services

R ServicesStreamInsights Analytics Platform

System

Reporting Services, Analysis Services,

Mobile Report

Microsoft R ServerMicrosoft Office

Cognitive Services

Bot Framework

Cortana

PolyBase

Po

lyB

ase Pu

bli

sh &

Co

nsu

me

Demo

• Cortana Intelligence Gallery

Rolls-Royce case studyhttps://customers.microsoft.com/en-US/story/rollsroycestory

Rolls-Royce demohttp://rolls-royce.azurewebsites.net/#/fleetlocation

Solutions – Predictive Maintenance for Aerospacehttps://gallery.cortanaintelligence.com/Solution/Predictive-Maintenance-for-Aerospace-4

Tutorial – Simulating phenotypes from genomic datahttps://gallery.cortanaintelligence.com/Experiment/Simulating-phenotypes-from-genomic-data-2

https://github.com/Azure/Cortana-Intelligence-Gallery-Content/tree/master/Resources/Phenotype-Prediction

Solutions – Vehicle Telemetry (IoT)https://gallery.cortanaintelligence.com/Solution/Vehicle-Telemetry-Analytics-9

https://docs.microsoft.com/en-us/azure/machine-learning/cortana-analytics-playbook-vehicle-telemetry

Advanced Analytics Cycle

Act: Score,

Visualize

Deploy Apps,

Services &

Visualizations

Measure

Preparation Modeling

Feature &

Algorithm

Selection

Model Testing &

Validation

Models

Visualizations

Ingest

Profile

Explore

Visualize

Transform

Cleanse

Denormalize

Prepare Model

OperationalizeModels

Visualizations

Data prep and exploration

Statistical analysis

Predictive models

Evaluating models

Input1 Input2 … Actual Predicted

• Classification example – Confusion Matrix

Classification vs Regression

Azure Machine Learning

Machine Learning

Cloud BI

(Power BI)

On-premise 대시보드(SQL Server 2016

Reporting Services)

1. Data Ingestion 2. Experiment

(Build and

evaluate models)

3. Deploy as web services

다양한 지원 Data set • Plain text (.txt)• Comma-separated values (CSV) • Tab-separated values (TSV) • OData values• SVMLight data (.svmlight)• Attribute Relation File Format (.arff) • Zip file (.zip)• R object or workspace file (.RData)

클라우드 BLOB/테이블 저장소(Azure Blob /Table Storage)

Hive 쿼리(HDInsight)

클라우드 PaaS형 DB

(Azure SQL DB)

1) 데이터 셋 업로드2) 클라우드 데이터 원본에 직접 연결

클라우드 BLOB/테이블 저장소(Azure Blob /Table Storage)

Hive 쿼리(HDInsight)

클라우드 PaaS형 DB

(Azure SQL DB)

실험 결과 데이터 셋 저장

웹 서비스로 배포

4. Consume ML models

잘 만들어진 분석 모델의 API화(타 비즈니스 앱에서 사용하기 위해)

On-premise Excel BI

서비스 API 키를 사용하여어플리케이션에서 API를호출하여 JSON 형태의결과 값 직접 사용

C#, Python 등 다양한언어로 API 호출 가능

2) 시각화

1) 비즈니스 어플리케이션에서 활용

Azure BLOB Storage에API 호출 결과(배치) 데이터 집합 저장

실험 결과 데이터 셋 또는 API 호출 결과 데이터셋을 시각화

[웹 서비스 관리 화면][2) 클라우드 직접 연결 방식 : 쿼리 입력 가능] [실험 수행 화면]

1) 모델 API 활용한 비즈니스 앱 개발2) 결과 데이터를 활용한 시각화

실험에 사용할데이터 전송

Demo

• Azure Machine Learning

Simple example : Linear Regression

Predictive Maintenance examplehttps://gallery.cortanaintelligence.com/Experiment/Predictive-Maintenance-Step-2A-of-3-train-and-evaluate-regression-models-2

Evaluate Model - Metrics Reportedhttps://msdn.microsoft.com/library/azure/927d65ac-3b50-4694-9903-20f6c1672089https://docs.microsoft.com/en-us/azure/machine-learning/machine-learning-evaluate-model-performancehttps://blogs.msdn.microsoft.com/andreasderuiter/2015/02/09/using-roc-plots-and-the-auc-measure-in-azure-ml/

JupyterNotebook to explore dataset

Excel Add-in for Azure MLhttps://blogs.technet.microsoft.com/machinelearning/2015/09/01/excel-add-in-for-azure-ml/

Operationalizing R with AML

Machine Learning cases – Mfg/Services

Publish Category Customer Use Case

2015/12

Utility /Electricity production prediction

ServusNetBefore : Previous system offered farm level production prediction using daily weather forecast. Now : Using cloud based end-to-end solution, covers more plant types, supports global scale multiple farm portfolios

2015/10Service/

MarketingOpenField

OpenField is an innovative data mgmt company which provides solutions for elite soccer clubs/sports/concert halls. Now : Contextual marketing solutions provide Ticket Purchase prediction, No-show prediction and maximize profits

2015/09Finance/Predictive

MaintenanceDiebold

Before : Unplanned downtime is a big loss and causes revenue drop, sudden repair cost, customer dissatisfactionNow : With advanced IoT technologies, they now can monitor machines periodically / continuously and predict failures before they happen.

2015/06Utility/

Demand Forecasting

GenscapeGenscape provides data and intelligence services in energy industry. Piloted Demand Forecasting model developments.

2015/03Utility/

Workload prediction

eSmart Systems

eSmarts developed S/W for Smart Grid / Meters for Norwegian utility customers. Predicts energy workload from minimum scale(meter-level) to higher, forecast where will be the bottleneck, use results for optimization algorithms to automatically rebalance workloads.

2014/09Utility/

Smart Building

Carnegie Mellon Univer

sity

Carnegie Mellon Univ uses Azure and PI System™ (by OSIsoft, a global ISV with Microsoft) to maintain buildings and reduce energy cost. Now CMU leverages Azure Machine Learning to improve failure detection, diagnosis, and optimize operations.

2014/08Manufacturing/

Predictive Maintenance

ThyssenKrupp

ThyssenKrupp Elevator focuses on service stability as competitive edge. With IoT and Machine Learning, ThyssenKrupp provides unique premium services including predictive maintenance at its core.

Publish Category Customer Use Cases

2015/12Service/

HRRussell Reynold

s Associates

Before : Hire candidates search is labor-centric and requires manual analysis with query writing on DatabaseNow : Microsoft Big Data and Advanced Analytics technologies enabled Machine Learning based Candidate Recommendations based on structured/unstructured data

2015/11Retail/Deep

LearningCoco-Cola

A case where Coca-Cola Company and Universal McCann used Microsoft Deep Learning technologies to perform cutting edge marketing campaign

2015/08Healthcare/Diagnosis

OptolexiaUsing eye movement tracking data and analytics engine built using cloud based Microsoft Azure Machine Learning, Optolexia offers far faster tool to diagnose Dyslexia(난독증) at school.

2015/06Public/Churn

Analytics

Tacoma Public School

Used Churn Analytics approach to predict students with high probability to quit. A public school in Tacoma, WA, dramatically improved its understanding of student behavior and could act upon the insights discovered using Machine Learning.

2015/05Service(Resea

rch)/Marketing

MendeleySocial document platform provider for researchers, built models to predict key users, performed email target marketing, and expanded its user base.

2015/04Healthcare/

Demand Forecast

Gaffey Healthcare

Used Azure ML to build predictive models and integrate with AlphaCollector, providing hospitals with insights how long it will take for insurance companies to pay claims, and help determine whether a human collector is needed to accelerate the claim payment process. Helped customers improve cash flows and reduce operational costs.

2015/02Healthcare/Diagnosis

Aerocrine

Before : Aerocrine’s monitoring tools are effectively used to diagnose and cure Asthma(천식), but very sensitive to small changes in the environment.Now : Using cloud based analytic solutions to improve diagnostic stability, helping millions of Asthma patients WW.

2014/12Retail/

MarketingJJ Food Service

Customers want products they like to be already in the shopping cart (Personalized recommendation). Customers of JJ Food Service are experiencing this whenever the make orders on web and over phone. Enabled by Azure Machine Learning and Dynamics.

2014/12Retail/

MarketingPier 1 Imports

Pier 1 Imports wanted to be connected to customers with data insights. Evaluated Predictive Analytics solutions and chose cloud based Azure Machine Learning and Power BI.

Look out for the cases outside your industry.

Many approaches are applicable across industries.

1. Which area to drive Digital Transformations through data analytics

2. What procedures/tools/algorithms to take

Machine Learning cases – Others

Why Chatbots are disrupting UX

Potential of Chatbots : “ability to individually and contextually communicate one-to-many”

1. 1 to Many communication

• Emails, Social Media are examples, but they are not personal.

2. Individual communication

• Personalization in 1-to-many communication is a recent consideration.

• Best way today is to use programmatic advertising but this requires efforts and know-how.

3. Contextual communication

• This happens whenever you talk to someone. Most people do this unconsciously.

Why Chatbots are rising as a new type of UX

Most people are consistently using messaging apps

• Average owned apps: 27. Daily using 4-6.Keep using 3% In 30 days

• 2.5 billion people owns 1+ messaging apps.3.6 billion expected (50% of WW population) in years.

Do not try to invent new appsto bring in customers.

Instead, offer your services in already popular messaging apps.

• Bots are UX, Conversations as a Platform (CaaP)

• Contents are important as well: From simple information delivery to actionable insights

1.Microsoft R • Statistical Analysis, Data Preparation, Predictive Modeling

Big Data • Hadoop, Spark, Data Lake Analytics

Machine Learning • Predictive Analysis, Deep Learning

Cognitive Services • Image Recognition, Natural Language Understanding

Bot Framework • Dev Framework, Different service channels

Technologies around Bots

Reference Architecture for Bots

Extended Scenarios

Big Data Analytics

Spark on HDInsight

Data Lake Analytics

Real Time Processing

Stream Analytics

Personalized Offer

Machine Learning

SQL Server R Services

On-premises Integration

SQL Server

Data Management Gateway

Visualization enabled

Power BI Embedded

Developing apps that understand human

• Face, image, emotion recognition

• STT/TTS, voice recognition/identification

• Intent/Entity understanding, sentiment/topic recognition, spell check

• Complex task processing, knowledge exploration

• Bing search functionalities integration(Web, auto-complete, image/video/news search)

Intelligence

Cortana

Bot

Framework

Cognitive

Services

Demo

• Cognitive Services Live, Intelligent Kiosk

Bots – wherever you have conversations

Intelligence

Cortana

Bot

Framework

Cognitive

Services

• Bot Connector : Register your own Bots, configure channels, publish on Bot Directory. Connect your Bots to SMS, Office 365 emails, Skype, Slack, Twitter, Facebook, Telegram and more.

• Bot Development SDK: Open source SDK available at GitHub. Offer every tool required for Bot development based on Node.js and C#.

• Bot Directory : A public place where you can publish your own Bots.

Enterprise Meeting Assistant

ATTEN

DESS

STA

RT T

IME

DU

RA

TIO

N

LOC

ATIO

N

Pls schedule a meeting for my team on the

next Tuesday morning with Yong at 13F

User Input

MY TEAM

IS A

LIST OF

PEOPLE

NEXT TUESDAY

MORNING

IS A

DATE

TIME

Yong

IS A

PEOPLE

NAME

13F

IS A

LOCATION

NAME

BOOK A MEETING

IS AN

INTENION

• Resolve Attendees

Create Active Directory query for “my team”

FIND “PEOPLE REPORT

TO ME” IN

ACTIVE DIRECTORY• Slots for Book Meeting

“Book a meeting” is an intention to book meeting

Yohn C. Jingtian J. Wenhao H. Lei F.

Filter related people by name contains “Yong”

• Link to Entities

Yong Rui Yong Liu

Filter people by relationship to me

Yong Rui

Using AI + HI to Complete Tasks

Conversational Entity Extraction

Response suggestion

AI for Business

Provides customers with more personal and natural ways to interact with businesses

Adds AI to business processes and connect Insights to Actions

Use insights hidden in data from in/out of company, understand customers and develop businesses

Demo

• Skype Bots, [email protected], LUIS, QnA Maker

Democratizing AI

To empower every person and organization to achieve more with AI

Thanks!

Ask questions :SeokJin Han ([email protected])