AI: Mobile Apps That Understands Your Intention When You Typed

Mobile Apps thatUnderstandsYour Intention WhenYou TypedMarvin HengTwitter : @hmhengBlog : http://hmheng.azurewebsites.netGithub: https://github.com/hmheng

{ } LUIS

Pre-requisites• Installed Visual Studio 2017 for Windows with Xamarin

Cross-Platform component & all necessary platform-specific SDK. (Download a FREE Community Version here)

• An Azure account (Follow this to create a free one!)

• Create a cross platform mobile app with .NETStandard 2.0(Learn to how to create one here)

• Created a Language Understanding Intelligent Service (LUIS) (Learn how here, it is just simple as ABC)

Let’s create a LUIS subscription @ portal.azure.com

Setup LUIS at Azure Portal & luis.ai1. Login with your Azure account @ portal.azure.com, click “+New” and search for LUIS.

1b1a

Setup LUIS at Azure Portal & luis.ai2. Select “Language Understanding Intelligent Service”.

2

Setup LUIS at Azure Portal & luis.ai3. Let’s hit “Create” to create a subscription for staging & production.

3

Setup LUIS at Azure Portal & luis.ai4. Enter your preferred name & the rest let it as default while I select free pricing for this demo. Then click “Create”.

4

Setup LUIS at Azure Portal & luis.ai5. Now, we head to www.luis.ai with what we created previously in tutorial @ “Together We Can Make World Smarter with LUIS”.

Setup LUIS at Azure Portal & luis.ai6. Before we publish it to production, we need to click “Add Key” to add a new key.

6

Setup LUIS at Azure Portal & luis.ai7. Select your subscription that was just created at portal.azure.com and click “Add Key”. A key should be generated.

7

Setup LUIS at Azure Portal & luis.ai8. Please take note of the Key in 8a, and app ID underlined in light blue while endpoint in purple (Optionally obtained from Dashboard) at 8b. We will need these information at later step.

8a 8b

Now, do a trick to let your mobile app understand whatever you type!

1. Follow this tutorial to create a Xamarin.Forms app here.

Code App to Understands What You Typed

2. Replace the code in ContentPage.Content of MainPage.xamlwith following code.

…<ContentPage.Content>

<StackLayout><Entry x:Name="txtMessage" Text="Command Here" /><Button x:Name="btnConnect" Text="Send" /><Label x:Name="lblIntentLabel" Text="Intent:" /><Label x:Name="lblIntent" Text="" /><Label x:Name="lblEntitiesLabel" Text="Entities:" /><Label x:Name="lblEntities" Text="" />

</StackLayout></ContentPage.Content>

…

Code App to Understands What You Typed

2

Code App to Understands What You Typed3. Next, we will need to get some sample code from LUIS’s documentation and do some changes for Xamarin.

Code App to Understands What You Typed4. Copy the code from MakeRequest function.

4

5. Paste it in MainPage.xaml.cs & make it an event function. This function will call LUIS api & get the results from LUIS API.

public async void MakeRequest(object sender, EventArgs e){

var client = new HttpClient();var queryString = HttpUtility.ParseQueryString(string.Empty);

// This app ID is for a public sample app that recognizes requests to turn on and turn off lightsvar luisAppId = “<Your App Id>";var subscriptionKey = “<Your App Key>";

// The request header contains your subscription keyclient.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", subscriptionKey);

// The "q" parameter contains the utterance to send to LUISqueryString["q"] = txtMessage.Text;

// These optional request parameters are set to their default valuesqueryString["timezoneOffset"] = "0";queryString["verbose"] = "false";queryString["spellCheck"] = "false";queryString["staging"] = "false";

var uri = "https://southeastasia.api.cognitive.microsoft.com/luis/v2.0/apps/" + luisAppId + "?" + queryString;var response = await client.GetAsync(uri);

var strResponseContent = await response.Content.ReadAsStringAsync();}

Code App to Understands What You Typed

5

6. Replace the ID, Key & endpoint region which you can obtain from the previous page.

public async void MakeRequest(object sender, EventArgs e){

var client = new HttpClient();var queryString = HttpUtility.ParseQueryString(string.Empty);

// This app ID is for a public sample app that recognizes requests to turn on and turn off lightsvar luisAppId = “<Your App Id>";var subscriptionKey = “<Your App Key>";

// The request header contains your subscription keyclient.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", subscriptionKey);

// The "q" parameter contains the utterance to send to LUISqueryString["q"] = txtMessage.Text;

// These optional request parameters are set to their default valuesqueryString["timezoneOffset"] = "0";queryString["verbose"] = "false";queryString["spellCheck"] = "false";queryString["staging"] = "false";

var uri = "https://southeastasia.api.cognitive.microsoft.com/luis/v2.0/apps/" + luisAppId + "?" + queryString;var response = await client.GetAsync(uri);

var strResponseContent = await response.Content.ReadAsStringAsync();}

Code App to Understands What You Typed

6a

6b

7. Create a new Folder “Models” and a Class “LuisResponse.cs”

Code App to Understands What You Typed

7

8. Add in the following codes to the newly created class –LuisResponse.cs for deserializing the response from LUIS later.

public class LuisResponse{

public string query { get; set; }public TopScoringIntent topScoringIntent { get; set; }public List<Intent> intents { get; set; }public List<Entity> entities { get; set; }

}

public class TopScoringIntent{

public string intent { get; set; }public double score { get; set; }

}

public class Intent{

public string intent { get; set; }public double score { get; set; }

}

public class Value{

public string timex { get; set; }public string type { get; set; }public string value { get; set; }

}

….

Code App to Understands What You Typed

7

….

public class Resolution{

public List<Value> values { get; set; }}

public class Entity{

public string entity { get; set; }public string type { get; set; }public int startIndex { get; set; }public int endIndex { get; set; }public double score { get; set; }public Resolution resolution { get; set; }

}

9. Add following lines in white after the last line of MakeRequest function.

…var strResponseContent = await response.Content.ReadAsStringAsync();

try{

lblIntent.Text = "";lblEntities.Text = "";LuisResponse luisResponse = JsonConvert.DeserializeObject<LuisResponse>(strResponseContent);if (luisResponse != null){

if (luisResponse.topScoringIntent != null){

lblIntent.Text = luisResponse.topScoringIntent.intent;}

if(luisResponse.entities.Count() > 0){

foreach (var entities in luisResponse.entities){

lblEntities.Text += entities.entity + "(" + entities.type + ")\n";}

}}

}catch(Exception ex){

Console.WriteLine(ex.ToString());}

…

Code App to Understands What You Typed

9

10. Now, we have to tell btnConnect to fire MakeRequestfunction when the button is being clicked.

public MainPage(){

InitializeComponent();btnConnect.Clicked += MakeRequest;

}

Code App to Understands What You Typed

10

11. Let’s compile and run.

Code App to Understands What You Typed

12. Type your sentence, eg. “Meet Marvin at Starbucks tomorrow”, you should get the intention & entities involved.

Code App to Understands What You Typed

12

Congratulation!You’ve Unlocked The Very First LUIS App Challenge!

Mobile Apps thatUnderstandsYour Intention WhenYou TypedMarvin HengTwitter : @hmhengBlog : http://hmheng.azurewebsites.netGithub: https://github.com/hmheng

{ } LUIS