Big data analytics and building intelligent applications

1confidentialFlytxt. All rights reserved. 30 June 201530 June 2015©

Architecting IntelligenceBig Data Analytics and

Building Intelligent Applications


Our discussion today

Big Data Analytics and Intelligent Applications• The puzzle, the hype, the customer?

• Man-machine collaboration

State of the Art• Practical AI, Machine Learning, Data Mining

• Data Science, Data Games

What does the future guarantee?• Physics, Networks and Computation

• New computation models?


Big Data, the Meme


Big (Data Analytics) Distraction


Big (Data) Crowd


Start unraveling the complexity

What do I want to communicate that currently requires a significant amount of time and energy to analyze, interpret, and share?• Stuart Frankel, “Data Scientists Don’t Scale”, Harvard Business Review,

May 2015

What economic value will my customer gain from Big Data?


FlytxtOur vision is to create >10% measurable economic value for Mobile Enterprises through Big Data Analytics

Flytxt’s solutions create incremental revenues from new and existing sources, optimize margins and enhance customer experience

Dutch company with corporate office in Dubai, global delivery centres in India and regional presence in Mexico City, Johannesburg, Singapore, Dhaka and Nairobi.

Sample text

Awards and Recognitions


Benefits delivered to customers

PartnersOperators

Proven across many Countries, Brands and Logos

Brands

IIT DELHI

4% Increase

in Gross Revenue

30%Growth

in Mobile Money users

10%Growth

in Data Users

105%Increase

in Special offer Sales

300%Increase

in Store Footfall

25%Dropin Churn


Big Data Technology Architecture – the Flytxt example


Our discussion today (Part 2)



State of the Art• AI, Machine Learning, Data Mining





Practical AI: Personalized Driving

Navigator suggest alternative route due to traffic congestion

System identifies primary/secondary driver

Bluetooth connection to Car

Systems

Car system connects with

database to access unique ID info

Infotainment settings are

modified (language

preference, radio station…)

Navigator presents favorite destinations

Connected office identifies next meeting happens in 10 min and offers re-scheduling


Practical AI: Pattern Classification in Location Analytics

6/30/2Copyright © 2012 12

Automatic classification of venues / routesbased on their features

Each venue/route is represented by a setof features

Labeled examples corresponding tovarious venue types / route types whichrepresent classes

Learn a decision boundary that separatesthe classes & then make predictions


Customer MarketingProgram Product Rol

Infra-structure

Location

Descriptive

Exploratory

Heuristic

Predictive

Prescriptive

Visualization

CLV Monitoring

Opportunity Identification

Behavioral Variations

Action Prediction

Personalized Recommendation

Effectiveness Measurement

Program Reach Analysis

Business impact Analysis

Outcome Forecasting

Impact Optimization

Product Popularity Monitoring

Product Promotion Analysis

Product Association

Profitability Simulation

Product Promotion Recommendation

Business Health Monitoring

KPI Impact Analysis

Business Impact measurement

Impact Forecasting

Yield Optimization

Utilization Monitoring

Challenge Identification

Cost Benefit Analysis

Event Prediction

Optimization Recommendation

Geo-Spatial Reporting

Location Affinity Analysis

Location- Behavior Association

Location based Forecasting

Location based Recommendation

Roots of Practical AI: Analytics Models built by Data Scientists


Data Sciences: State of the Art

KDD Cup: organized by ACM Special Interest Group on Knowledge Discovery and Data Mining

2010: Predict student performance on mathematical problems from Intelligent Tutoring System logs

2011: Recommending Music Items based on the Yahoo! Music Dataset

2012: Predict which information sources one user might follow in Weibo (Chinese “twitter”)

2013: Determine whether an author has written a given paper

2014: Predict funding requests that deserve an A+ (for DonorsChoose.org)

2015: Predict student dropout on a Massive Open Online Course platform (XuetangX)


Our discussion today (Part 3)



State of the Art?• AI, Machine Learning, Data Mining





Memoirs from the Past: Hilbert’s Program

In 1900, David Hilbert, a very influential universal mathematician, announced a grand search for a complete and consistent set of axioms for all mathematics

In 1931, Kurt Gödel announced his discovery of the Incompleteness Theorem: There will always be statements about the natural numbers that are true, but that are unprovable within the system

Hilbert probably dedicated his life trying to prove his hypothesis, which Gödel proved cannot be true!

However, Gödel’s work inspired Alan Turing and Alonzo Church, and in 1936, they mathematically defined “computation”


The future AI platform is a network!

Courtesy: Maulik Kamdar, Stanford University


Future AI agents

AI agents will compute; with data that gets generated on many devices

2025: 100 billion connected devices, 175 zeta bytes of data per year (Huawei)

Data volumes will grow faster than any network or computer can be sized

How will you scale the AI of tomorrow?


Practical AI: Moving data, moving code

Code must meet data to compute – code moves and/or data does, across a (wireless) network

History: All data moved to where the code was

Near past: Parallel and distributed computation – partition code & data

Now: (approximately) Move code to where the data is (Hadoop etc)

Future: Determine the code-data match and optimize movement?• Is there is a computational model for this?


Physics, Networks, Computation – immutable laws

Energy dissipation in radiation (Gauss’s / Coulomb’s Laws)• Low energy reception implies higher decoding error (Shannon’s Limit)• How fast can memory-to-memory transfers happen?

Capacity of a wireless network is constrained by interference (e.g. see Gupta & Kumar, 2000)• Spectrum (# channels) available will remain finite• Channel allocations will be dynamic, but how fast can two interfering pairs

find free channels?

Are there limits to local computation? (e.g. see works by Ning Xie, Shai Vardi)• Moving code or data implies “local” processing• How much AI can be computed, and at what cost?


Example: High-dimension Clustering

6/30/2 Co21

Basic machine learning algorithm to group nodes (users, people, devices) by state (behavior)• Each node produces a vector describing current state

• Nodes are clustered together by some measure of vector similarity

“Moving code” distributed implementations available today (on Hadoop/Spark)

Future: Rate of change of state will outpace speeds of computation and communication

Is the solution hierarchical, is the paradigm divide and conquer?

How will network & algorithm design and implementation change?• Can all clustering problems be solved “locally”?


Discussion summary

Big Data Analytics and Intelligent Applications• Build for customer value, build simple solutions

State of the Art• Practical AI and Data Sciences

What does the future guarantee?• Need to scale AI compute: Data generation rates faster than compute

/ communication rates


Thank Youwww.flytxt.com

http://www.flytxt.com/

http://www.flytxt.com/

Big data analytics and building intelligent applications

Technology

data mining data science

data games

data users

big data crowd

big data analytics distraction

data scientists dont

art ai

machine learning