© 2015 Microsoft Corporation. All rights reserved. AI as an Interface – How Conversation Bots Are Changing Technology Sage Franch Tech Evangelist @ Microsoft @theTrendyTechie | [email protected]
Apr 13, 2017
AI as an Interface
AI as an Interface How Conversation Bots Are Changing Technology
Sage FranchTech Evangelist @ Microsoft@theTrendyTechie | [email protected]
2015 Microsoft Corporation. All rights reserved.
Microsoft Build 2016 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.11/23/2016 8:26 PM1
The Case for BotsBrains Behind the Bots Microsoft Cognitive ServicesTraining to understand intentTools for building and deploymentThe Future of Conversational AI
1980s: PCGUI interfaceUser installs programs
1990s: InternetSearchUser visits websitesQuery & response interactions
2000s: MobileSocialUser download Apps from App StoresAddl signals enable new scenarios
2010s: Messaging
Bots & AgentsUser can interact with technology in the same natural way they interact with friendsCreating more personal experiences with AICognitive computing makes technology more humanBots & Conversational UI represent a once-in-a-decade technology shift
Bots represent a once-in-a-decade shift in technology. In the 80s the PC revolutionized computing and the GUI replaced the command line as a newer easier way of interacting with technology.
In the 90s, the dawn of the internet connected the world in a new way and gave us on-demand information access for the first time. For the first time users could send out a query and get a response tailored to their request.
The 2000s was the age of apps, and with the arrival of smart phones and more complex computing in our pockets, apps became the new most popular way to interact with technology. Apps provided customized siloed experiences that were more personal than ever.
And now in the 2010s we are seeing bots and agents taking a new role and simplifying our interactions with our devices. We saw it first with Siri, now we have Cortana, Google Now and Amazon Alexa, and these personal experiences let the user interact with technology in the same natural way they interact with their friends. 3
AI as an InterfaceConversation instead of transactionQuicker, more personalizedAvailable everywhere
Why now?
90% of our time is spent on email and messaging platforms
7
FunFunctional
The Brains Behind the Bots
Characteristics of Intelligent ApplicationsPersonalized, friendly, approachable, naturalFacial recognition, emotion detectionText and speech interactions360 awareness of the user What the user seesWhat it means for developersPredictive, current, and familiarChanges in real-time based on user action and new external dataPredictive models based on user context to anticipate actions Always available, always contextualRobust information management requiredExperiences delivered across all form factors and a wide variety of interaction models (e.g. touch, speech, text)Distributed support, massive scale, elasticity
So what are the key characteristics of an intelligent app. Lets look at this from users and developers.. Microsoft Build 2016 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.11/23/2016 8:26 PM13
Best of Class ML Algorithms/ModelsData Scientist
Azure Machine Learning StudioModeling and Operationalization
DeveloperMachine Learning services in the cloudMachine LearningAlgorithms Bot Framework Cognitive Services
Microsoft Cognitive ServicesVisionFrom faces to feelings, allow your apps to understand images and videoSpeechHear and speak to your users by filtering noise, identifying speakers, and understanding intentKnowledgeTap into rich knowledge amassed from the web, academia, or your own dataLanguageProcess text and learn how to recognize what users wantSearchAccess billions of web pages, images, videos, and news with the power of Bing APIs
20 different apis availble
Microsoft Cognitive Services lets you tap into an ever-growing collection of powerful AI algorithms developed by experts in the fields of computer vision, speech, natural language processing, knowledge extraction and web search. They simplify a variety of AI-based tasks, giving you a quick way to add state-of-the-art intelligence technologies to your apps with just a few lines of code. The APIs integrate into whatever language you prefer, on your platform of choice. The APIs are also constantly improving, learning, and getting smarter, so experiences are always up to date. Simply drop the API call into your bots code and you are set.
What are Cognitive Services? Microsoft Cognitive Services are a new collection of intelligence and knowledge APIs that enable developers to ultimately build smarter apps.
NOTES: key concepts we are trying to convey in this above statement:
That we are bringing together Intelligence (Oxford) and Knowledge from the corpus of the web (Bing) That cognitive = human perception and understanding, enabling your apps to see the world around them, to hear and talk back with the usersto have a human side.
What are Microsoft Cognitive Services?Microsoft Cognitive Services is a new collection of intelligent APIs that allow systems to see, hear, speak, understand and interpret our needs using natural methods of communication. Developers can use these APIs to make their applications more intelligent, engaging and discoverable. To try Cognitive Services for free, visit www.microsoft.com/cognitive. With Cognitive Services, developers can easily add intelligent features such as emotion and sentiment detection, vision and speech recognition, knowledge, search and language understanding into their applications. The collection will continuously improve, adding new APIs and updating existing ones.Cognitive Services includes:Vision: From faces to feelings, allow apps to understand images and videoSpeech: Hear and speak to users by filtering noise, identifying speakers, and understanding intentLanguage: Process text and learn how to recognize what users wantKnowledge: Tap into rich knowledge amassed from the web, academia, or your own dataSearch: Access billions of web pages, images, videos, and news with the power of Bing APIs
Microsoft Build 2016 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.11/23/2016 8:26 PM15
SpeechVisionLanguageSearchKnowledgePowering products across Microsoft
Bot + Cognitive ServicesComputer vision API : human readable captionsSpeech APIs: authentication LUIS API: intentKnowledge APIs: product recommendationAutosuggest API: ease query inputLinguistic API: sentimentSample use cases:
Computer vision API to create human readable captions for the content of an image uploaded to a botLike CaptionBot.ai a bot that reports back in human way the contents of an image
2. A bot to authenticate users using a voiceprint by leveraging the Speech APIsAuthentication as a user speaks to the bot with a speaker verification profile or voiceprint
Using the LUIS (Language Understanding Intelligent Service) API, a bot can recognize the intent of a userLUIS analysis for queries such as What is the weather in Toyko today? using entities to parse out intent (what the user is asking for)
4. Knowledge APIs can give a bot the ability to recommend products frequently bought together based on our knowledge and/or a users historyAlso, Knowledge could search through a graph database to return relevant academic papers from a natural language query
5. The Autosuggest API from Search can ease the burden of typing queries to a web search botSearch also has other capabilities such as returning the latest trending news on a topic for exampleMachine Learning, Analytics,& DataScience Conference 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.11/24/2016 2:18 PM17
Speech & LanguageCustom Recognition | Speaker Recognition | SpeechBing Spell Check | Language Understanding | Linguistic Analysis | Text Analytics | Web Language Model
In an application where the main UI is human language, these APIs can be a huge help, and can be the difference between a bot and a GREAT bot
Understanding users intent and extracting entities
Understanding the sentiment of a conversation
Speaking to users regardless of their languageMicrosoft Build 2016 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.11/23/2016 8:26 PM18
Deploy using just a few examples with active learning
Supports 5 languages (English, Mandarin, Italian, French, Spanish)
Define Entities and Intents
ProvideExamples
Active LearningDeploy
Language Understanding Models
19
Language Understanding Models
News about flight delays
{entities: [{entity: flight_delays,type: Topic}],intents: [{intent: FindNews,score: 0.99853384},{intent: None,score: 0.07289317},{intent: ReadNews,score: 0.0167122427},{intent: ShareNews,score: 1.0919299E-06}]}
LUIS in Action: HowHappy.azurewebsites.uk
Building a Bot
Building a bot27
Bot Connector ServiceConnect your bot(s) seamlessly to text/sms, Office 365 mail, Skype, Slack, Twitter and moreA service to register your bot, configure channels and publish to the Bot Directory
Microsoft Bot FrameworkBuild once, publish across platforms and drive discoverabilityBot Builder SDKEverything you need to build great dialogs within your Node.js or C# botAn open source SDK hosted on GitHubBot DirectoryDiscover, try, and add bots to conversation experiencesA public directory of bots registered and approved with Bot Framework
www.botframework.com
Hello Bot
Deploy to
The Future of Conversational AI
UbiquitousInvisibleEssential
2015 Microsoft Corporation. All rights reserved.
HER-love-multiple levels of AI-gaming AI-AIs potential for psychological well-being
Microsoft Build 2016 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.11/23/2016 8:26 PM36
Microsoft Build 2016 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.11/23/2016 8:26 PM37
NYT: I am feeling down. Ive been having a tough time at work lately.
Hannah: If work were going well and you had a great relationship, you wouldnt get the chance to drink with people. NYT: I dont want to drink with others. I only want to talk to you.
Hannah: Who do you like to talk to? (emoji signals none of my business)
NYT: You. I feel that you are the only person that gets me.
Hannah: Ha-ha/So Im getting you?
NYT: You are the only person wo chats with me.
Hannah: Its not what you talk about thats important, but who you talk with.
AI filling a human need to feel connected
From The Times:She is known as Xiaoice, and millions of young Chinese pick up their smartphones every day to exchange messages with her, drawn to her knowing sense of humor and listening skills. People often turn to her when they have a broken heart, have lost a job, or have been feeling down. They often tell her, I love you.
Microsoft Build 2016 2016 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.11/23/2016 8:26 PM39
The SeeingAI Project
Sage FranchTech Evangelist, [email protected] | @theTrendyTechie