Bot Framework & Azure cognitive service簡介

BoT

BoT

智慧服務

儀錶板 & 資料視覺化

巨量資料儲存 機器學習跟分析

行動

People

Automated Systems

Apps

Web

Mobile

Bots

Cortana

Bot

Framework

Cognitive

Services

Power BI

資訊管理

Event Hubs

Data Catalog

Data Factory

HDInsight

(Hadoop and

Spark)

Stream

Analytics

智慧服務

Data Lake

Analytics

Machine

Learning

SQL Data

Warehouse

Data Lake

Store

資料來源

應用程式

感知器

與裝置

資料

IoT Hub

Bot Framework

1

2

3

4

5

6

7

8

9

10

每月熱門APP資料來源 AppAnnie 2015

1bn

100m100m

300m

1bn總註冊使用者(行動裝置及電腦2016)

217m

在智慧型的互動模式中, 人跟機器可以透過自然人類語言做順暢的溝通

…

…

• 能發掘出語言中提到的人事時地物(Contextual

awareness)

• 記住跟每個人的聊天歷程

• 推敲出語句背後要表達的用意(Intention)

• 能了解互動當時的人跟情境, 並能有敏感度的回

應

發展對話機器人的挑戰

微軟正在建造最全面的工具及服務給對話機器人(Bot)應用

產品與服務 | 個性

機器學習APIs:

電腦視覺, 語音, 語言, 知識, 搜尋, 翻譯

接在熟悉的平台上利用即時語音, 影像, 訊息, 群組訊息去接觸客戶

包括開發對話機器人的SDK, 讓對話機器人更聰明的感知服務與機器學習

建造一次, 發佈到多個平台並容易被搜尋使用

提供開源(Open Source)的SDK讓開發者更容易建置對話機器人, 數分鐘內就可以建立完成

連接對話機器人到所有主流APP社交平台, 可以接觸數十億使用者

讓對話機器人容易被使用者搜尋及找到

www.botframework.com

https://github.com/Microsoft/BotBuilder

Cognitive Services

Microsoft

感知服務

讓您的 Apps 了解圖片與影片的內容

讓您的 Apps 過濾雜訊可以辨識說話者是誰? 並了解用戶的意圖

文字語意分析，讓 Apps 了解用戶文字的意涵

彙整來自互聯網，學術研究與您自己所累積的知識與資料

透過 Bing API 存取數十億網頁, 圖片, 影片與新聞

microsoft.com/cognitive

Intelligent Services Made Easy

語音時間長短 < 15 秒 < 2 分鐘

轉換成果n-best 可供選擇

句子停頓處提供最佳選擇

********* Final N-BEST Results *********

[0] Confidence=Normal Text="450 six St San Francisco."

[1] Confidence=Normal Text="For 50 six St San Francisco."

[2] Confidence=Normal Text="456th St San Francisco."

[3] Confidence=Normal Text="450 six St in San Francisco."

[4] Confidence=Normal Text="456 St San Francisco."

語音辨識模型 Voice Recognition Modes

Short Form Long Form

450 6th St. San Francisco

支援 REST API 與特定平台函式庫

real-time streaming

LUIS

了解對方在講些什麼使用預設Bing 或 Cortana 模型或是使用自己的模型

理解人類鍵入之文字

提供互動式模型訓練工具漸少標示關鍵實體成本

呼叫內容自動保存，可日後持續訓練與學習

支援五種語言 (英文,中文, 義大利文, 法文, 西班牙文)

語言理解模型

microsoft.com/cognitive

LUIS (語言理解智能服務) - 讓機器聽得懂!• 創建企業專屬的語言理解模型:

示範例句

• 請問這裡是否有賣XXX(暖暖包)?!

模型訓練

訓練成果

• 來一句新的：這裡還有沒有雷神巧克力?!

*screenshots show actual search results on bing.com

https://bingapis.azure-api.net/v5/search?q=nasa

{

“_type”: “SearchResponse”,

“queryContent”: {…},

“webPages”: {…},

“news”: {…},

“images”: {…},

“videos”: {…},

“relatedSearches”: {…},

“rankingResponse”: {…}

}

{

“answerType”:”WebPages”,

“resultIndex”:0,…

},

{

“answerType”:”News”,

“resultIndex”:1,…

}

Ranking Response

Search Response

Web

Results

Deep

Links(1st Algo)

News

Results

Image

Results

Video

Results

Related

Searches

https://bingapis.azure-api.net/v5/images/search?q=shuttle+launch

• 支援多種過濾條件(尺寸, 版權, 色彩, 過濾成人內容 …等)• Image insights (相似圖片，相關搜尋… 等)

source: nasa.gov

https://bingapis.azure-api.net/v5/videos/search?q=viral+videos

• 支援多種過濾條件 (影片解析度,長度 … 等)• 影片預覽圖片

https://bingapis.azure-api.net/v5/news/search?q=cuba

• 新聞分類• 豐富之metadata (美國區域)

source: youtube.com

*screenshots show actual search results in bing.com

source: cnn.com

AI 服務架構Technology Storyboard

Social Media

Customer

Insights

Productivity Cloud

Smart

Agents

Skype

Dynamics

Marketing/AXCRM Online

PowerAppsCognitive

Services

Personal Graph

Office Graph

Bing

PowerBI

Office 365

Sharepoint

Knowledge Capture

Knowledge Sharing

Customer Support Case Management

Conversational AI Coaching

Intelligent Cloud

App Insights

HDInsight

Azure Machine

LearningData Factory

Data Lake

Stream

AnalyticsEvent Hubs

HDStormAzure SQL DB

Notification

Hubs

Microsoft

Social

Engagement

More Personal

Computing

Customer

Windows 10

Cortana

In-Store

Online

(web, chat)

Devices

Digital

Signage/

Kiosk

Equipment

Insights

Usage

Insights

Real-time contextual

data Weather, Events

& environmental

conditions

Real-Time network,

infrastructure &

customer telemetry

Customer Service

Engineering Knowledge Capture

Knowledge Sharing

Technical Case Management

Cognitive

Services

Give your solutions

a human side

Cognitive

Services

Give your solutions

a human side

Vision

JSON:{"language": "en","orientation": "Up","regions": [{"boundingBox": "41,77,918,440","lines": [{"boundingBox": "41,77,723,89","words": [{"boundingBox": "41,102,225,64","text": "LIFE"},{"boundingBox": "356,89,94,62","text": "IS"},{"boundingBox": "539,77,225,64","text": "LIKE"}

. . .

"faceRectangle": {"width": 193, "height": 193, "left": 326, "top": 204}

“scores": { "anger": 5.182241e-8,

"contempt": 0.0000242813,

"disgust": 5.621025e-7,

"fear": 0.00115027453,

"happiness": 1.06114619e-8,

"neutral": 0.003540177,

"sadness": 9.30888746e-7,

"surprise": 0.9952837}

"faceRectangle": {"width": 193, "height": 193, "left": 326, "top": 204}

…

"attributes": { "age": 42, "gender": "male", "headPose": { "roll": "8.2", "yaw": "-37.8", "pitch": "0.0" }}

Speech

Language

Language understanding models

Knowledge

Search

Bing web searchIntelligent search to your apps with the ability to comb billions of webpages, images, videos, and news with a single API callRetrieve web documents indexed by Bing and narrow the results down with filters such as by answer type and freshness

Bing image searchA variety of image search options, from trending images to detailed insightsScour the web for images and get results that include thumbnail and full image URLs, publishing website, image metadata, related images, and more

Bing video searchA variety of advanced video search features, including trending videos, price, and other useful metadataFind videos from across the web and get responses that provide useful metadata including creator, encoding format, video size and quality, and source view count

Bing news searchTurn any app into a news desk with world news grouped and filtered by topic, local news, and metadata you can mine

Bing auto suggestQuery completion suggestions capabilities, so users can type less and get to what they want faster

[