Understanding User Feedback using Sentiment Analyzer & … · Understanding User Feedback using Sentiment Analyzer & Analytics Automation ROHIT SINGHAL

Unde rs ta nding Us e r Fe e dba ck us ingS e ntime nt Ana lyze r & Ana lytics Automa tion

ROHIT SINGHAL

© 2018 eBay. All rights reserved.2

• I am Rohit Singhal

• QA Engineer @eBay, DE

• ~5 years in testing domain

@rohsin29

S e s s ion Expe cta tions ?

© 2018 eBay. All rights reserved.4** Image Source: Google

© 2018 eBay. All rights reserved.5** Image Source: Google

Te s t S e tup vs Re a l Us e rs !

© 2018 eBay. All rights reserved.7

Test Lab Production

© 2018 eBay. All rights reserved.8

iOS 8 (0.20%) iOS 9 (1.57%)

iOS 10 (15.88%) iOS 11 (34.3%) iOS 12

(55.5%)

iOS Distribution Android Distribution

**Source: david-smith.org/iosversionstats & Google

JellyBean (2.2) Kitkat (7.8) Lollipop (18.2)

MarshMallow (21.6%) Nougat (29.3) Oreo (17.2%)

Fe e dba ck S ource s

© 2018 eBay. All rights reserved.10

© 2018 eBay. All rights reserved.11

© 2018 eBay. All rights reserved.12 **Source: Play Store

S ome Examples

© 2018 eBay. All rights reserved.13

Feature Requests

**Source: Play Store

Ana lytics a nd Da ta S ource s

© 2018 eBay. All rights reserved.15 ** Image Source: Google

Analytics is a collection, reporting, and analysis of application‘s data. The focus is on identifying measures based on

your organizational and user goals and using the data to determine the success or failure of those goals and to drive

strategy and improve the user’s experience.

And how track that?

© 2018 eBay. All rights reserved.16 ** Image Source: Google

© 2018 eBay. All rights reserved.17** Image Source: Google

Ma nua l Approa ch

© 2018 eBay. All rights reserved.19

Visit Social Platforms

(Google Play Store, App

Store, FB, Twitter etc)

Figure out what Users are

saying (Negative, Neutral)

Generate Meaningful Data

PROBLEM STATEMENT - 1

© 2018 eBay. All rights reserved.20

Manual Testing for

Analytics for every

Story/Card

Regression - Testing

Analytics with User

Journey

Validating Analytics on

Release Candidate – As a

User Journey

PROBLEM STATEMENT - 2

Automa te d Approa ch

© 2018 eBay. All rights reserved.22

Fetch Daily Reviews for

Android App (Google Play

Store)

Segregate Feedback

(Negative & Neutral)

Generate Meaningful Daily

Reports

Release Generate Repeat

(with Continuos Integration)

SOLUTION - 1

© 2018 eBay. All rights reserved.23

FEATURE FILES

Events and Values to be

Validated

Live Execution on Test and

Staging Environments

SOLUTION - 2

De mo

© 2018 eBay. All rights reserved.25

© 2018 eBay. All rights reserved.26

© 2018 eBay. All rights reserved.27

© 2018 eBay. All rights reserved.28

© 2018 eBay. All rights reserved.29

Analytics Feature

© 2018 eBay. All rights reserved.30

Baseline Events

Cha lle ng e s Fa ce d

© 2018 eBay. All rights reserved.32

• Integrating this complete process and running it daily on CI to generate

meaningful reports

• Figuring out the language for the feedbacks as our mobile application is used in

multiple regions (WIP)

• Storing and separating the feedback based on date range

Te ch S ta ck (Ope n S ource )

© 2018 eBay. All rights reserved.34

• Cucumber

• Ruby, python

• Watir

• MITM Proxy

• AYLIEN Sentiment Analysis

Ne xt S te ps

© 2018 eBay. All rights reserved.36

• Introduce ML to make reporting more robust and more specific to an app’s functionality

• Use Google APIs to fetch reviews rather than using Selenium

• Fetching reviews from other Social Platforms like Twitter, Facebook, etc.

• Running end-to-end BDD test for analytics on all platforms

© 2018 eBay. All rights reserved.37

Thank You

@rohsin29

rohsin02

https://github.com/rohsin02/Sentiment-Analysis-Play-Store-Reviews