Knowledge poweredaudienceengagement public

Shifting Paradigms: Knowledge Powered Audience Engagement

Ying Li

Chief Data Scientist, Parllay

Ying Li


Data Science and Computational Advertising




“…make online advertising

even more relevant and

useful by using additional (to

search) information about the

websites people visit. …”







“… find precisely

the consumer you

are looking for”








the consumer you

are looking for”

“… focusing your campaign

on … audience … interested

in the areas of your products

and services.








the consumer you

are looking for”

“… focusing your campaign

on … audience … interested

in the areas of your products

and services. hundreds of millions of users

billions of impressions

terabytes of raw data

Observational Analysis of Audience Behavior

Observational Analysis of Audience Behavior 12/4/10 17:53 pondora

12/23/10 18:43 linkedin

12/23/10 18:55 complaint free world

12/23/10 19:27 isee practice test

12/23/10 19:47 free isee practice tests

11/20/10 19:16 univercity village

12/29/10 7:04 eric schadt

12/18/10 17:37 in search of excellence

12/29/10 20:02 road conditions

12/29/10 20:02 seattle traffic

12/11/10 20:27 vineyard crest bellevue

11/30/10 17:37 eric schadt

12/21/10 22:19 eric schist eqtl

12/21/10 22:19 eric schmidt qtl

12/22/10 4:52 eric schist


12/22/10 4:53 +eric schist

12/22/10 4:54 +eric schist

12/22/10 4:55 eqtl

12/22/10 4:59 eqtl schist

12/22/10 5:29 rosetta inpharmatics closing

12/9/10 5:04 overlake school

12/12/10 21:00

jo ann

12/12/10 21:02

craft supplies

12/27/10 16:12

bellevue square hours

12/20/10 17:52

yahoo labs

12/20/10 19:13

harrah's

12/20/10 19:14

harrah's entertainment

12/20/10 22:59

delicious

12/28/10 23:27

zune download

1/3/11 18:25 psalm 90

1/3/11 18:28 bible in one year chart

11/22/10 14:54

schoolreport.org

1/2/11 23:10 automatic labeler

1/3/11 6:05 origami dragon

12/27/10 6:25 能权力

12/27/10 6:25 百普力

12/27/10 6:27 enteral nutritional suspension


12/23/10 18:43 linkedin










11/30/10 17:37 eric schadt





12/22/10 4:53 +eric schist

12/22/10 4:54 +eric schist

12/22/10 4:55 eqtl




12/12/10 21:00

jo ann

12/12/10 21:02

craft supplies

12/27/10 16:12


12/20/10 17:52

yahoo labs

12/20/10 19:13

harrah's

12/20/10 19:14


12/20/10 22:59

delicious

12/28/10 23:27

zune download

1/3/11 18:25 psalm 90


11/22/10 14:54

schoolreport.org



12/27/10 6:25 能权力

12/27/10 6:25 百普力


Easy to predict

• Interests

Music

Science

School

Craft

• Language

English, Chinese

• Religion

Christianity


12/23/10 18:43 linkedin










11/30/10 17:37 eric schadt





12/22/10 4:53 +eric schist

12/22/10 4:54 +eric schist

12/22/10 4:55 eqtl




12/12/10 21:00

jo ann

12/12/10 21:02

craft supplies

12/27/10 16:12


12/20/10 17:52

yahoo labs

12/20/10 19:13

harrah's

12/20/10 19:14


12/20/10 22:59

delicious

12/28/10 23:27

zune download

1/3/11 18:25 psalm 90


11/22/10 14:54

schoolreport.org



12/27/10 6:25 能权力

12/27/10 6:25 百普力


Easy to predict

• Interests

Music

Science

School

Craft

• Language

English, Chinese

• Religion

Christianity

Possible but harder to infer

• Applying for school

• Craft supply for Origami

• Nordstrom Quarterly Shopper –

through data enrichment


12/23/10 18:43 linkedin










11/30/10 17:37 eric schadt





12/22/10 4:53 +eric schist

12/22/10 4:54 +eric schist

12/22/10 4:55 eqtl




12/12/10 21:00

jo ann

12/12/10 21:02

craft supplies

12/27/10 16:12


12/20/10 17:52

yahoo labs

12/20/10 19:13

harrah's

12/20/10 19:14


12/20/10 22:59

delicious

12/28/10 23:27

zune download

1/3/11 18:25 psalm 90


11/22/10 14:54

schoolreport.org



12/27/10 6:25 能权力

12/27/10 6:25 百普力


Easy to predict

• Interests

Music

Science

School

Craft

• Language

English, Chinese

• Religion

Christianity

Hard to predict more details

• Science discipline = Data Mining

• Music genre = U2

• School = private school for son







12/23/10 18:43 linkedin










11/30/10 17:37 eric schadt





12/22/10 4:53 +eric schist

12/22/10 4:54 +eric schist

12/22/10 4:55 eqtl




12/12/10 21:00

jo ann

12/12/10 21:02

craft supplies

12/27/10 16:12


12/20/10 17:52

yahoo labs

12/20/10 19:13

harrah's

12/20/10 19:14


12/20/10 22:59

delicious

12/28/10 23:27

zune download

1/3/11 18:25 psalm 90


11/22/10 14:54

schoolreport.org



12/27/10 6:25 能权力

12/27/10 6:25 百普力


Easy to predict

• Interests

Music

Science

School

Craft

• Language

English, Chinese

• Religion

Christianity










Very possible to get wrong

• Gambler


12/23/10 18:43 linkedin










11/30/10 17:37 eric schadt





12/22/10 4:53 +eric schist

12/22/10 4:54 +eric schist

12/22/10 4:55 eqtl




12/12/10 21:00

jo ann

12/12/10 21:02

craft supplies

12/27/10 16:12


12/20/10 17:52

yahoo labs

12/20/10 19:13

harrah's

12/20/10 19:14


12/20/10 22:59

delicious

12/28/10 23:27

zune download

1/3/11 18:25 psalm 90


11/22/10 14:54

schoolreport.org



12/27/10 6:25 能权力

12/27/10 6:25 百普力


Easy to predict

• Interests

Music

Science

School

Craft

• Language

English, Chinese

• Religion

Christianity











• Gambler


12/23/10 18:43 linkedin










11/30/10 17:37 eric schadt





12/22/10 4:53 +eric schist

12/22/10 4:54 +eric schist

12/22/10 4:55 eqtl




12/12/10 21:00

jo ann

12/12/10 21:02

craft supplies

12/27/10 16:12


12/20/10 17:52

yahoo labs

12/20/10 19:13

harrah's

12/20/10 19:14


12/20/10 22:59

delicious

12/28/10 23:27

zune download

1/3/11 18:25 psalm 90


11/22/10 14:54

schoolreport.org



12/27/10 6:25 能权力

12/27/10 6:25 百普力


Easy to predict

• Interests

Music

Science

School

Craft

• Language

English, Chinese

• Religion

Christianity











• Gambler


12/23/10 18:43 linkedin










11/30/10 17:37 eric schadt





12/22/10 4:53 +eric schist

12/22/10 4:54 +eric schist

12/22/10 4:55 eqtl




12/12/10 21:00

jo ann

12/12/10 21:02

craft supplies

12/27/10 16:12


12/20/10 17:52

yahoo labs

12/20/10 19:13

harrah's

12/20/10 19:14


12/20/10 22:59

delicious

12/28/10 23:27

zune download

1/3/11 18:25 psalm 90


11/22/10 14:54

schoolreport.org



12/27/10 6:25 能权力

12/27/10 6:25 百普力


Easy to predict

• Interests

Music

Science

School

Craft

• Language

English, Chinese

• Religion

Christianity











• Gambler

Impossible to predict any future interests

• Tech Hobbyist

Data, Math, Algorithmic Engines



Tech Hobbyist


• Find the audience

Tech Hobbyist



Tech Hobbyist



Tech Hobbyist


• Find the audience • Expand the audience

Tech Hobbyist


• Find the audience • Expand the audience

Tech Hobbyist


• Find the audience • Expand the audience • Validate the audience

Tech Hobbyist


• Find the audience • Expand the audience • Validate the audience

Tech Hobbyist


• Find the audience • Expand the audience • Validate the audience • Forecast inventory

Tech Hobbyist


• Find the audience • Expand the audience • Validate the audience • Forecast inventory • Optimize inventory

Tech Hobbyist


• Find the audience • Expand the audience • Validate the audience • Forecast inventory • Optimize inventory • Test: A/B or multivariate

Tech Hobbyist


• Find the audience • Expand the audience • Validate the audience • Forecast inventory • Optimize inventory • Test: A/B or multivariate • Analyze:

• In-segment or out-segment • Before and after

Tech Hobbyist




Tech Hobbyist




• Do it all at scale

Tech Hobbyist

Reason to Be Excited: Data of Engagement



12/12/10 21:00 jo ann

12/12/10 21:02 craft supplies

11/27/10 21:30 9633 vineyard crest

12/27/10 16:12 bellevue square hours

12/20/10 17:52 yahoo labs

12/20/10 19:13 harrah's

12/20/10 19:14 harrah's entertainment

12/20/10 22:59 delicious

12/28/10 23:27 zune download

1/3/11 18:25 psalm 90


11/22/10 14:54 schoolreport.org



12/27/10 6:25 能权力

12/27/10 6:25 百普力


Reason to Be Excited: Data of Engagement 12/4/10 17:53 pondora

12/23/10 18:43 linkedin










11/30/10 17:37 eric schadt





12/22/10 4:53 +eric schist

12/22/10 4:54 +eric schist

12/22/10 4:55 eqtl




12/12/10 21:00 jo ann



12/20/10 17:52 yahoo labs

12/20/10 19:13 harrah's


12/20/10 22:59 delicious


1/3/11 18:25 psalm 90





12/27/10 6:25 能权力

12/27/10 6:25 百普力



12/23/10 18:43 linkedin










11/30/10 17:37 eric schadt





12/22/10 4:53 +eric schist

12/22/10 4:54 +eric schist

12/22/10 4:55 eqtl




12/12/10 21:00 jo ann



12/20/10 17:52 yahoo labs

12/20/10 19:13 harrah's


12/20/10 22:59 delicious


1/3/11 18:25 psalm 90





12/27/10 6:25 能权力

12/27/10 6:25 百普力



12/23/10 18:43 linkedin










11/30/10 17:37 eric schadt





12/22/10 4:53 +eric schist

12/22/10 4:54 +eric schist

12/22/10 4:55 eqtl




12/12/10 21:00 jo ann



12/20/10 17:52 yahoo labs

12/20/10 19:13 harrah's


12/20/10 22:59 delicious


1/3/11 18:25 psalm 90





12/27/10 6:25 能权力

12/27/10 6:25 百普力



12/23/10 18:43 linkedin










11/30/10 17:37 eric schadt





12/22/10 4:53 +eric schist

12/22/10 4:54 +eric schist

12/22/10 4:55 eqtl




12/12/10 21:00 jo ann



12/20/10 17:52 yahoo labs

12/20/10 19:13 harrah's


12/20/10 22:59 delicious


1/3/11 18:25 psalm 90





12/27/10 6:25 能权力

12/27/10 6:25 百普力



12/23/10 18:43 linkedin










11/30/10 17:37 eric schadt





12/22/10 4:53 +eric schist

12/22/10 4:54 +eric schist

12/22/10 4:55 eqtl


New Opportunities with Connected Data

Location

Opinions

People Time


Location

Opinions

People Time


• Knowledge sources and inference

Location

Opinions

People Time


• Knowledge sources and inference • Audience graph

Location

Opinions

People Time



• Entity graph, Knowledge graph

Location

Opinions

People Time




• Semantified content • Wikipedia, Freebase, Khan Acadamy, …

Location

Opinions

People Time





Location

Opinions

People Time





• Real world context awareness

Location

Opinions

People Time





• Real world context awareness • Where you are, who you are with, what

resources are around you

Location

Opinions

People Time





• Real world context awareness • Where you are, who you are with, what

resources are around you

• Ability to close the loop • Dynamic Data

• Shared opinion

• Instant feedback

• From behavior observation to experimentation

Source: Adapted from Liebowitz, (2003)

The Wisdom Pyramid

Wisdom

Collective

application of

knowledge in

action

Knowledge

Experience, values, context applied to a

message

Information

A message meant to change receiver’s perception

Data

Discrete, objective facts about an event

Adding Value: • Contextualized

• Categorized

• Calculated

• Corrected

• Condensed

Adding Value: • Comparison

• Consequence

• Connections

• Conversations

Adding Value: • Action-oriented

• Measurable

efficiency

• Wiser decisions


The Wisdom Pyramid

Wisdom

Collective

application of

knowledge in

action

Knowledge


message

Information


Data



• Categorized

• Calculated

• Corrected

• Condensed


• Consequence

• Connections

• Conversations


• Measurable

efficiency

• Wiser decisions


The Wisdom Pyramid

Banner ads

Search Ads

Re-messaging

Wisdom

Collective

application of

knowledge in

action

Knowledge


message

Information


Data



• Categorized

• Calculated

• Corrected

• Condensed


• Consequence

• Connections

• Conversations


• Measurable

efficiency

• Wiser decisions


The Wisdom Pyramid

Banner ads

Search Ads

Re-messaging

Contextual ads

BT ads

Wisdom

Collective

application of

knowledge in

action

Knowledge


message

Information


Data



• Categorized

• Calculated

• Corrected

• Condensed


• Consequence

• Connections

• Conversations


• Measurable

efficiency

• Wiser decisions


The Wisdom Pyramid

Banner ads

Search Ads

Re-messaging

Contextual ads

BT ads

What would it

look like at this

stage?

Wisdom

Collective

application of

knowledge in

action

Knowledge


message

Information


Data



• Categorized

• Calculated

• Corrected

• Condensed


• Consequence

• Connections

• Conversations


• Measurable

efficiency

• Wiser decisions


The Wisdom Pyramid

Banner ads

Search Ads

Re-messaging

Contextual ads

BT ads

What would it

look like at this

stage?

Knowledge Powered

Audience Engagement

Key Components

Web

Structured Data

Unstructured Data

Data Sources

Key Components

Web

Structured Data

Unstructured Data

Data Sources

Ontology,

Schema

mapping

Classification

engine

Entity, facts,

events,

relationship

Sentiment,

opinion

Semantic Engines

Knowledge Extractors

Key Components

Web

Structured Data

Unstructured Data

Data Sources

Ontology,

Schema

mapping

Classification

engine

Entity, facts,

events,

relationship

Sentiment,

opinion

Semantic Engines


Key Components

Ontology

Entity Graph

User Graph

Dim 1

Int Attr1

Int Attr2

Bool Attr3

Dim 2

Int Attr1

Int Attr2

Bool Attr3

Int Attr 4

Dim 3

Int Attr1

Int Attr2

Dim 4

Int Attr1

Int Attr2

Bool Attr3

Char Attr4 Dim 5

Int Attr1

Int Attr2

Bool Attr3

Fact

Dim Keys

Int Meas1

Int Meas2

Float Meas3

Dimensional Index

Inference Engines

Web

Structured Data

Unstructured Data

Data Sources

Ontology,

Schema

mapping

Classification

engine

Entity, facts,

events,

relationship

Sentiment,

opinion

Semantic Engines


Key Components

Ontology

Entity Graph

User Graph

Dim 1

Int Attr1

Int Attr2

Bool Attr3

Dim 2

Int Attr1

Int Attr2

Bool Attr3

Int Attr 4

Dim 3

Int Attr1

Int Attr2

Dim 4

Int Attr1

Int Attr2

Bool Attr3

Char Attr4 Dim 5

Int Attr1

Int Attr2

Bool Attr3

Fact

Dim Keys

Int Meas1

Int Meas2

Float Meas3

Dimensional Index

Inference Engines

Web

Structured Data

Unstructured Data

Data Sources

Ontology,

Schema

mapping

Classification

engine

Entity, facts,

events,

relationship

Sentiment,

opinion

Semantic Engines


Content Recommendation

Engines

Key Components

Ontology

Entity Graph

User Graph

Dim 1

Int Attr1

Int Attr2

Bool Attr3

Dim 2

Int Attr1

Int Attr2

Bool Attr3

Int Attr 4

Dim 3

Int Attr1

Int Attr2

Dim 4

Int Attr1

Int Attr2

Bool Attr3

Char Attr4 Dim 5

Int Attr1

Int Attr2

Bool Attr3

Fact

Dim Keys

Int Meas1

Int Meas2

Float Meas3

Dimensional Index

Inference Engines

Web

Structured Data

Unstructured Data

Data Sources

Ontology,

Schema

mapping

Classification

engine

Entity, facts,

events,

relationship

Sentiment,

opinion

Semantic Engines



Engines

Key Components

Ontology

Entity Graph

User Graph

Dim 1

Int Attr1

Int Attr2

Bool Attr3

Dim 2

Int Attr1

Int Attr2

Bool Attr3

Int Attr 4

Dim 3

Int Attr1

Int Attr2

Dim 4

Int Attr1

Int Attr2

Bool Attr3

Char Attr4 Dim 5

Int Attr1

Int Attr2

Bool Attr3

Fact

Dim Keys

Int Meas1

Int Meas2

Float Meas3

Dimensional Index

Inference Engines

Web

Structured Data

Unstructured Data

Data Sources

Ontology,

Schema

mapping

Classification

engine

Entity, facts,

events,

relationship

Sentiment,

opinion

Semantic Engines



Engines

Feedback Loop

Key Components

Ontology

Entity Graph

User Graph

Dim 1

Int Attr1

Int Attr2

Bool Attr3

Dim 2

Int Attr1

Int Attr2

Bool Attr3

Int Attr 4

Dim 3

Int Attr1

Int Attr2

Dim 4

Int Attr1

Int Attr2

Bool Attr3

Char Attr4 Dim 5

Int Attr1

Int Attr2

Bool Attr3

Fact

Dim Keys

Int Meas1

Int Meas2

Float Meas3

Dimensional Index

Inference Engines

Entities and Semantics

Entities and Semantics

Entities and Semantics • Entity – of independent existence


Raspberry Pi

Pong

Lego

Erlang


Raspberry Pi

Pong

Lego

Erlang


• Semantics – assign meaning to data

Raspberry Pi

Pong

Lego

Erlang


• Semantics – assign meaning to data

Raspberry Pi

Pong

Lego

Erlang

Construction

toy

Game Arcade

game

Single-board

computer

Raspberry Pi

Pong

Lego

Erlang

Functional

language


• Semantics – assign meaning to data • Using statistics to associate meaning with concepts,

entities & user actions

• Use ontologies that describe them in a domain

Raspberry Pi

Pong

Lego

Erlang

Construction

toy

Game Arcade

game

Single-board

computer

Raspberry Pi

Pong

Lego

Erlang

Functional

language

Recommender systems

Recommender systems

User Profile &

contextual parameters

Community data

Product features

Knowledge Model

Recommender systems

Recommendation Engine

User Profile &


Community data

Product features

Knowledge Model

Recommender systems


Ranked product list User Profile &


Community data

Product features

Knowledge Model

Recommender systems




Community data

Product features

Knowledge Model

Recommender systems




Community data

Product features

Knowledge Model

Recommender systems




Community data

Product features

Knowledge Model

Recommender systems




Community data

Product features

Knowledge Model

Recommender systems




Community data

Product features

Knowledge Model

Recommender systems




Community data

Product features

Knowledge Model Construction

toy

Game Arcade

game

Single-board

computer

Raspberry Pi

Ying

Jon

Pong

Lego

Erlang

Functional

language

Recommender systems




Community data

Product features

Knowledge Model Construction

toy

Game Arcade

game

Single-board

computer

Raspberry Pi

Ying

Jon

Pong

Lego

Erlang

Functional

language

ML

Putting it together: discovery and anticipation

Construction

toy

Game Arcade

game

Single-board

computer

Raspberry Pi

Ying

Jon

Pong

Lego

Erlang

Functional

language


Construction

toy

Game Arcade

game

Single-board

computer

Raspberry Pi

Ying

Jon

Pong

Lego

Erlang

Functional

language


Construction

toy

Game Arcade

game

Single-board

computer

Raspberry Pi

Ying

Jon

Pong

Lego

Erlang

Functional

language


Robotics

Construction

toy

Game Arcade

game

Single-board

computer

Raspberry Pi

Ying

Jon

Pong

Lego

Erlang

Functional

language


Robotics

Robotics

Hardware

Construction

toy

Game Arcade

game

Single-board

computer

Raspberry Pi

Ying

Jon

Pong

Lego

Erlang

Functional

language


Robotics

Robotics

Hardware

Robot kit

Construction

toy

Game Arcade

game

Single-board

computer

Raspberry Pi

Ying

Jon

Pong

Lego

Erlang

Functional

language


Robotics

Robotics

Hardware

Arduino

Robot kit

Construction

toy

Game Arcade

game

Single-board

computer

Raspberry Pi

Ying

Jon

Pong

Lego

Erlang

Functional

language


Robotics

Robotics

Hardware

Arduino

Robot kit

Construction

toy

Game Arcade

game

Single-board

computer

Raspberry Pi

Ying

Jon

Pong

Lego

Erlang

Functional

language

Unsupervised Prediction


Robotics

Robotics

Hardware

Arduino

Robot kit

Construction

toy

Game Arcade

game

Single-board

computer

Raspberry Pi

Ying

Jon

Pong

Lego

Erlang

Functional

language

Unsupervised Prediction


In Summary

• Connected data

• Semantified content

• Feedback loop enabled machine learning

• Knowledge driven content recommendation

• Audience centric content curation

• At-scale continuous audience engagement

Thank you

Dr.Ying Li


Knowledge poweredaudienceengagement public

Technology

eric schist eqtl

eric schadt

eqtl schist

online advertising

eric schmidt qtl

parllay data science

websites people

search of excellence