Twitter Realtime Social Data @StartupFest

Realtime Social Data workshop @ startupfest

Sylvain Carle Senior Developer Advocate

@froginthevalley

2

Twitter is the pulse of the planet

@Sylvain: from Startup to Twitter

3

• Startup CTO (a few times around)

• REST + Streaming API building Needium

• Didn’t work so well (that’s another story)

• The tech stack was pretty awesome!

• Modern version of our 2011 idea today

• Colin @ElasticSearch, Sylvain @Twitter

• In the end, not so bad!

!

Twitter Data APIs

4

Twitter APIs deal both with the moment and the past

The pulse of the planet

5

• More than 241 million monthly active users • 500 million Tweets are sent per day, or 1 billion every ~2 days • 300+ billion Tweets have been sent since founding in 2006 • 76% of our active users are mobile users • 40% of our active users simply consume content on Twitter • Over 400 million monthly unique visitors to twitter.com • Twitter supports 35 different languages • 77% of Twitter accounts are outside the U.S.

Get started

6

!

Create you app: easyt.co/appsDo not re-invent the wheel: libs t.co/libs !

Create your App

7

Twitter Data APIs

8

The Past: REST API t.co/rest Optimize your calls, don’t get 429-ied !

The Moment: Streaming API t.co/streaming 1% Sample or Track Keywords !

Getting started

9

Twurl

10

!

t.co/twurl

Twurl

11

!

t.co/console

Hosebird (HBC)

12

!

t.co/hbc

Getting started

13

Think with Data

14

• Focus on 2nd order effects

• Derive value from the content/network • Catch emerging patterns: trends • Listen to the chatter: conversations • Flip it inside out: content from users • Broader perspective: analytics • Understand users: personalization

Think with Data

15

• Find one top level object to get started

• Users, Tweets, Timelines • Map this concept to your domain knowledge • Start with a measurable experience • Moebius virtuous circle: fed and feed data • Add your own layer of data for max value • Analyze collections + context, not single tweets

Building you taxonomy

16

• Human Insights (domain knowledge)

• Data Insights • Discovery mode (keyword inventory / baseline) • Machine Learning (co-occurrences, clustering) • Historical Indices • Noise cancellation • Detecting emerging trends

Taxonomy: Needium Locksmiths

17

• Human Insights (existing taxo + research) • http://en.wikipedia.org/wiki/Lock

• Data Insights • Discovery mode: swearing, types of issues • Machine Learning: “locked up” “locked out” • Historical Indices: evaluate opportunity by geo

• Noise cancellation: “lockin” “boat” “clock” • Detecting emerging trends: seasonal, time of day

Other Examples

18

• Dataminr - finding emerging news • Lithium - managing social at scale • Sprinklr - social experience management • Wayin - best brand stories by customers • Inside - most important stories, for you • More at t.co/partners

REST API 1.1 Resources

19

!

t.co/1.1

Streaming API Resources

20

!

t.co/1.1

Ramp up

21

• Start small, grow organically • Get your core data in and out (+ management) • Build your MVP Twitter integration • Measure, experiment, learn • Accelerate: rewrite, optimize, elevate • Pedal to the metal: promoted, hoses

Twitter Data APIs

22

!

Ask us @TwitterDev Ask in the forums t.co/discuss !

Twitter Data APIsStreaming API t.co/streaming REST API t.co/rest Create App t.co/appsTwurl t.co/twurl Open Source Libs t.co/libs Discussion forums t.co/discussAsk us @TwitterDev