Analytics at Motorola: Motorola journey to enable self-serve analytics that leverage Google tools and API

motorola confidential

Analytics at Motorola

Motorola journey to enable self-serve analytics that leverage Google tools and API

http://www.gatewayanalyticsnetwork.com/b125-home

Dr Patrick DeglonDirector of Engineering, Analytics Area Tech LeadMotorola Mobility [email protected] pdeglon

http://www.gatewayanalyticsnetwork.com/b122-home


What my friends think I do

What my mum thinks I do

What the Execs think I do

What I think I do What the Execs wants me to do

What I really do

About Patrick


...

Overview of Motorola Business

About UsMotorola exists to invent, build and deliver the best mobile devices on the planet, improving the lives of millions of people.


1928 1936 1943 1955 1969World’s first

portable

FM two-way

radio

Motorola introduced

Police Cruiser Radio Receiver

World’s first high-power transistor

in commercial production

First words from moon relayed via Motorola

radio

Motorola: 80+ YEARS OF INNOVATION

1947 1963World’s first

truly rectangular

color TV tube

1983 19961991World’s first commercial handheld

cellular phone

DynaTAC 8000X

weighed 28-ounces

(794 grams)

World’s first GSM cellular

system

World’s first dual-mode

cellular phone

The 3.1 ounce (88 grams) StarTac©

wearable cellular phone is the

world’s smallest and lightest

1990World’s first

HDTV technical standard

1973Demonstrated prototype of the DynaTAC

portable cellular system

World's first handset , iDEN i1000plus, to

combine a digital phone, two-way radio, Internet microbrowser,

e-mail, fax and two-way messaging

1999 2000World’s first

general packet radio service

(GPRS) wireless phone for always on

Internet access

2004 2006 2009Iconic RAZR V3 wireless

phone introduced

MING smart phone recognizes

10,000+ handwritten characters

from Chinese alphabet

20122002World’s first

wireless cable modem gateway

introduced

Motorola DROID #1 on

Time’s Top Ten of 2009

2013 2014Launch MotoX, Moto G

Fast upgrades

Galvin Manufacturing Corp

Moto EMoto 360


• Overview of Analytics at Motorola

• Daily Email: Reporting with Google Spreadsheet + BigQuery

• Moto Insights: Reporting with App Engine + BigQuery

• Drive Insights: Reporting with App Engine + BigQuery + Google

Analytics + Google Drive

• Big Feed: ETL with App Engine & Big Query

Agenda


How do we see Analytics at Motorola?

Provide easy access to data & insights to drive the business towards our goal*:

Raw Data

Business Datasets

Visualization

Driving Insights & Recommendation

* Motorola exists to invent, build and deliver the best mobile devices on the planet, improving the lives of millions of people.

Enablers:● Self-serve with low barrier

to data and know-how ● High data quality & trust

● Deep understanding of customer & device behavior

● Feedback loop to improve user experience & device performance

● Strong data stewardship


Our Big Data Environment

Instrumentation with user consent

Android Settings Motorola Privacy ☐ Help Improve Motorola Products (On/Off) Moto Care (On/Off)

MotorolaCloud

Insights

Google Platforms

App Engine (large scale web servers)Cloud Storage (large file storage)Data Store (large real-time storage)Big Query (public implementation of Dremel)Compute Engine (virtual Linux servers)…

Legacy

New


Google: A galaxy of open APIs and Tools…

App EngineBig Query

Compute Engine

Users Cloud Messaging

Analytics

Maps

Charts

Data Store

Cloud Storage

Cloud SQL

Drive

Docs

Translate

Predict

Mail

Data Store

Task Queue

Memcache

URL Fetch

Cloud End Point

Channel

Java

Python

Go

Mail and many more ...


• Big Query: Instrumentation

• Big Query: Server logs

• Big Query: SalesForce

• Big Query: External data

• Google Analytics

• Google Spreadsheets & CSV

Raw Data Business Datasets Visualization

• BigFeed (internal tool build on App Engine to manage BigQuery ETL)

• Compute Engine (ETL CRON jobs)

• Google Spreadsheets & Docs

• Tableau widget through a Java web server

• Moto Insights (internal tool bridging BigQuery with Google Charts)

• Drive Insights (next generation of Moto Insights with Google Drive)

Data flows for Analytics








Agenda


Daily Activations Report

How to provide a global source of truth and

provide insights from data points?

Existing Situation

- Numerous (conflicting) sources of truth- Too many variations of same data cube- “Table in your face” approach- No global business definition- No curation of manually entered data - Report accessible on an internal portal (through VPN)- No mobile form factor


Simplified Business Flow

Data Flow

Motorola Factory# Shipments

Distribution Channels# Sales

First Usage# Activations

InsightsGoogle BigQuery

MotorolaCloud


Final deliverable

Illustrative Simulation


• Assume sales follow a diffusion S-shape, i.e.

Description of the illustrative simulation

Marketing Word of mouth

ΔN

Nmax

N

• Add random noise to theoretical daily activations (Poisson)

• Simulated daily activations (sales) for United States, Canada, Brazil, India, Russia, China, Germany and United Kingdom with various launch date per region

ΔN = a (Nmax - N) + b N (Nmax - N)


Step 1: Create a backbone table

SELECTCAL_DT,CountryFROM

ON A.Dummy=B.DummyWHERE

B.CAL_DT>=A.Launch_Date

motorola.com:sandbox:demo.backbone:INNER JOIN (SELECT Country, CASE WHEN Country IN ('United States','Canada') THEN '2013-08-01' WHEN Country IN ('Brazil','Russia','India','China') THEN '2013-10-01' ELSE '2013-12-01' END AS Launch_Date, GDP_USD/1e7 AS Scale, 1 AS DummyFROM [motorola.com:sandbox:pdeglon.countries]WHERE Country IN ('United States','Canada','Brazil','Russia','India', 'China','Germany','United Kingdom')) AS B

(SELECT CAL_DT, 1 AS Dummy FROM [motorola.com:sandbox:pdeglon.calendar]) AS A

X

BA

CK

UP


Step 2: Calculate KPI value over time

SELECTCAL_DT,Country, ‘Phone 123’ AS Model,

INTEGER(Scale*EXP(-POW( -150,2)/2/POW(75,2))/(75*SQRT(2*PI())) ) AS Daily_ActivationsFROM

[motorola.com:sandbox:demo.backbone]

motorola.com:sandbox:demo.baseline:

DATEDIFF(TIMESTAMP(CAL_DT),TIMESTAMP(Launch_Date))

...

Normal Distribution:

BA

CK

UP


Step 3: Add Random Noise

SELECTCAL_DT,Model, Country,INTEGER(

Daily_Activations + SQRT(Daily_Activations) *SQRT(-2*LN(RAND()))*COS(2*PI()*RAND())

) AS Daily_ActivationsFROM

[motorola.com:sandbox:demo.baseline]

motorola.com:sandbox:demo.simulation:

Normal (Gaussian) Random Number (mu=0, sigma=1)

(pseudo) Poisson distribution for N=Daily_activation

BA

CK

UP


Step 4: Final Pivot for report

SELECTCAL_DT,SUM(Daily_Activations) AS Total,SUM(CASE WHEN Country IN ('United States','Canada') THEN Daily_Activations ELSE 0 END) AS NA,SUM(CASE WHEN Country IN ('Brazil','Russia','India','China') THEN Daily_Activations ELSE 0 END) AS BRIC,SUM(CASE WHEN Country IN ('Germany','United Kingdom') THEN Daily_Activations ELSE 0 END) AS EU,SUM(CASE WHEN Country='United States' THEN Daily_Activations ELSE 0 END) AS UnitedStates,SUM(CASE WHEN Country='Canada' THEN Daily_Activations ELSE 0 END) AS Canada,SUM(CASE WHEN Country='Brazil' THEN Daily_Activations ELSE 0 END) AS Brazil,SUM(CASE WHEN Country='Russia' THEN Daily_Activations ELSE 0 END) AS Russia,SUM(CASE WHEN Country='India' THEN Daily_Activations ELSE 0 END) AS India,SUM(CASE WHEN Country='China' THEN Daily_Activations ELSE 0 END) AS China,SUM(CASE WHEN Country='Germany' THEN Daily_Activations ELSE 0 END) AS Germany,SUM(CASE WHEN Country='United Kingdom' THEN Daily_Activations ELSE 0 END) AS UnitedKingdomFROM[motorola.com:sandbox:demo.simulation]WHERECAL_DT<CURRENT_DATE()GROUP BY 1ORDER BY 1 DESC

BA

CK

UP


Google Spreadsheet personalization


Script Editor: App Script language (JS style)


Example for adding a new menu item


Query the data: SQL pivot


Running the query on BigQuery


Parsing the results


Pivot data sheet


Aggregate into a Summary sheet


Email template sheet (HTML + spreadsheet formula)


Preparing email: concatenate all cells


Adding a chart to the email


Sending the email through Gmail


Helping user understand Signal vs Noise and Main Drivers


Celebrating with Key Milestones


Final output


Automating the process with Triggers


Schedule process & QA

Refresh Data (H-3)

Automated QA Checks

Area Leads check the numbers

Alert Email to Operation Center

Errors

Preliminary Email (H-2)

No Errors

Final Emails (H)

No Complains

Revisit QA rules

Complains

Correct Data & RerunCorrect Data & Rerun








Agenda


Moto Insights

How to provide insights to executive-on-the-go with a

robust system where new report take seconds to create?

Existing Situation

- Require VPN- New Report take weeks to create- New features take months to develop- Team spend more time resolving defects than developing new features & reports


Demo Moto Insights

Moto InsightsApp Engine

Data Source:Big Query

ReportMeta Data:Datastore

Tracking:Datastore

Users Access:Google Users

(email) +Datastore (role)

Moto InsightsWeb portal

Moto Insights Android App


Responsive Design

responsive design

Front-end based on Bootstrap CSS and AngularJS


Report meta-data


Chart meta-data


All Google chart types


Example with the Illustrative Simulation








Agenda


Drive Insights

How to promote self serve and democratize Analytics within the

company while maintaining quality (data & insights) as well as managing Big Data access?

Existing Situation

- Reports are produce by a centralized team- Insight needs are rapidly changing- In the eyes of our internal customers, long lead time on report evolution

Key assumption: Business community can become SQL literate with the help of trainings, tutorials and documentation on a Data Wiki (Confluence)


Demo Drive Insights (Moto Insights V2)

Drive InsightsApp Engine

Data Source:Big Query

Data Source:Google Analytics

iFrame Source:Tableau Server

iFrame Source:Google DocumentsReport

Meta Data:Google Drive

(Text file with JSON)

Report Meta Data:Datastore

(Report copy & usage tracking)

Users Access Control:

Google Users + Drive Sharing

Google Drive

Moto Insights Android App

Drive InsightsWeb portal


New document type in your Google Drive


Report meta-data


Chart meta-data


Google Chart Types


SQL Editor (CodeMirror)


Report document in your Google Drive


Example of the Illustrative Simulation


Sharing through Google Drive

42…so what?

Answer to the Ultimate Question of Life, The Universe, and Everything

Example: Campaign MeasurementCampaigns

• Campaign Id• Campaign Name• Time range• Set of Countries• Set of Products

KPI

• Date• Country• Product• KPI[]

X

Trend

• Campaign Id• Date• Total of KPI[]

Summary

• Campaign Id• Campaign Name• Impact Measurement[]• Statistical Error[]

Time Series

Analysis

Extract of a Drive Insights reportCampaign Window Real data on

truncated axis

Campaign Cycle








Agenda


BigFeed: in-house App Engine ETL tool

How to address complexity, transformation and partitioning?

Existing Situation

- Queries across the entire dataset- Required more complicated queries which limited self serve…- Became inefficient as the volume grew….- Lacked comprehensive partitioning scheme


“BigFeed is an internal application that provides API and services to build, run and monitor execution of BigQuery queries”

• Provides a generic framework to develop specific datasets quickly• Focus is on Query

• Provides URL based access to run BQ to BQ ETL• Platform agnostic• Can run feeds on demand

What is BigFeed?


Why Do a BigFeed?

1. Simplicity - enable simpler queries for the user community. Foster self serve2. Performance 3. Cost → Scanning less data 4. Need to run a query repetitively → Ongoing analysis5. Single point of optimization








Agenda


Q&A

?Dr Patrick DeglonDirector of Engineering, Analytics Area Tech LeadMotorola Mobility [email protected] pdeglon

Analytics at Motorola: Motorola journey to enable self-serve analytics that leverage Google tools and API

Data & Analytics

motorola radio motorola

motorola motorola journey

motorola confidential

motorola droid

overview of motorola

data insights

motorola daily email

google drive data flows