From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

1/85

From publisher to platform:How the Guardian embraced the internetusing content, search, and Open Source

Stephen Dunn, Guardian News and [email protected], 25th May, 2011

Twitter: @cuica, @openplatform

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

2/85

1

From publisher to platformHow the Guardian embraced the Internet usingcontent, search, and Open Source

Stephen Dunn, Guardian News and Media

2

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

3/85

The publishing era

3

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

4/85

We started a longtime ago:

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

5/85

Swine flu

Keyword page

Twitter updates

Content partnerships

Audio

Video Open platform API

Live blogs

Comment

Mobile siteApps

Newspapers

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

6/85

To secure the financial and editorialindependence of the Guardian in perpetuity.To promote freedom in the press and liberal

journalism globally.

o secure the financial and editorial independenceof the Guardian in perpetuity

To promote freedom in the press and liberaljournalism globallyThursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

7/85

7

Open Web Principles

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

8/85

8

2009

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

9/85

A cool URI is one that does not change Tim Berners-Lee 1998

1.5 million resources redirected to new scheme

9

1. Permanent

http://www.flickr.com/photos/fstorr/

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

10/85

10

2. Addressable Resources are about something - ready for the

social web.

We live in the age of point-at-things (Coates 2005)

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

11/85

11

Multiple routes

to content

Tagging drives

discovery

3. Discoverable

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

12/85

12

4. Open

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

13/85

/technology/internet

/technology/all

/environment/climatechange

Example: The Hackable Guardian

http://www.guardian.co.uk/....

/rss

/rss

+business/globaleconomy/rss

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

14/85

Results...

14

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

15/85

15

3,750,000

7,500,000

11,250,000

15,000,000

18,750,000

22,500,000

26,250,000

30,000,000

Sep 2005 Oct 2006 Nov 2007 Dec 2008

U

nique

Users

Site traffic growthUnique Users

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

16/85

However...

16

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

17/85

17

1 Billion+

Internet

Users!

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

18/85

18

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

19/85

19

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

20/85

20

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

21/85

21

...How Istoppedworrying aboutmy website and

learned to lovethe wholeinternet.

Matt McAlister

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

22/85

22

The Open Strategy

OPEN IN

Bring in data and appsfrom the Internet

OPEN OUT

Enable partners tobuild applicationsusing Guardian

content and servicesfor other platforms

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

23/85

23

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

24/85

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

25/85

25

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

26/85

26

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

27/85

27

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

28/85

28

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

29/85

29

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

30/85

30

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

31/85

31

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

32/85

32

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

33/85

33

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

34/85

34

The Guardian alongside Al Jazeera was the one news source

that everybody on the streets in Tahrir - not just in Cairo but in

surrounding cities and major centers of revolutionary activity -

that people were talking about.

Jack Shenker

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

35/85

3522

The Open Strategy

OPEN IN

Bring in data and appsfrom the Internet

OPEN OUT

Enable partners tobuild applicationsusing Guardian

content and servicesfor other platforms

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

36/85

36

The Open Platform

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

37/85

37

The suite of services enablingpartners to build applications with

the Guardian

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

38/85

3822

OPEN IN

Bring in data and appsfrom the Internet

OPEN OUT

Enable partners tobuild applicationsusing Guardian

content and servicesfor other platforms

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

39/85

CONTENT APIA service for

selecting and

collecting

content from

the Guardianfor re-use

DATA STOREA directory of

useful data

curated by

Guardian

editors

POLITICS APIOpen database

of candidates,

voting records,

constituencies,

election results,live data on

election day

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

40/85

Mutualised news!

40

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

41/85

Mutualised news!

41

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

42/85

Mutualised news!

42

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

43/85

43

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

44/85

44

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

45/85

45

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

46/85

46

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

47/85

DATA STOREA directory of

useful data curatedby Guardian

editors

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

48/85

POLITICS APIOpen database ofcandidates, voting

records, constituencies,election results, livedata on election day

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

49/85

POLITICS APIOpen database ofcandidates, voting

records, constituencies,election results, livedata on election day

49

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

50/85

50

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

51/85

51

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

52/85

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

53/85

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

54/85

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

55/85

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

56/85

56

Open for Business

Thursday, 26 May 2011

3 Tiers of access

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

57/85

57

3 Tiers of access3 Revenue models

Keyless:Take our headlines. You keep associatedrevenues.

Approved: Take our full article content, but with anadvert. Guardian keeps ad revenue, you keep rest-of-

page revenue.

Bespoke:Take, reformat, augment our contentRevenue model to be negotiated. Combination ofMedia, Fees, Downloads.

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

58/85

58

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

59/85

59

What this means

Open Out: Developers can now access full content APIs ondemand with keys post-approved

Platform is positioned as a place to do business

So rapid scalability, reliability and performance are now core

requirements

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

60/85

OPEN IN

Bring in data andapps from the

internet

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

61/85

61

A framework forintegrating 3rd party

applications into

guardian.co.uk

MICROAPPS Simple REST/HTTPframework allows lightweightdevelopment

Applications proxied forperformance

Apps generally hosted in the

cloud, allows hot deploymentinto production

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

62/85

62

A framework forintegrating 3rd party

applications into

guardian.co.uk

MICROAPPS

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

63/85

What could I cook?

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

64/85

64

Bringing it together

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

65/85

65

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

66/85

App showcase

66

Thursday, 26 May 2011

From publisher to

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

67/85

67

From publisher to

latformSeeking massive growth, but no longer onlybroadcasting content on the website

User/partner engagement & contribution onJournalismdatasoftwareapplications

revenue and ads

Support developers and partners with data and APIs,need scalability, reliability, speed

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

68/85

68

Evolving thearchitecture

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

69/85

Oracle

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

70/85

Oracle

Why RDBMS?

5 years ago, fewer alternatives

Understand operations procedures

Can easily recruit DBAs / devs

Developer/ops tools

Business critical system: a safe choice

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

71/85

71

3,750,000

7,500,000

11,250,000

15,000,000

18,750,000

22,500,000

26,250,000

30,000,000

Sep 2005 Sep 2006 Sep 2007 Sep 2008

Unique

Users

Scaling trafficUnique Users

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

72/85

72

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

73/85

73

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

74/85

74

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

75/85

75

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

76/85

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

77/85

77

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

78/85

We chose Solr/Lucene

78

Can perform complex queries, including full-text search

We can change the schema with no downtime

Most queries are of similar cost

Scales very well horizontally

Just worked in the cloud

No strange control processes/engines

Developers just loved working with it!

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

79/85

79

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

80/85

RDBMS

80

Cloud, EC2

Api

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

81/85

8122

OPEN IN

Bring in data and appsfrom the Internet

OPEN OUT

Enable partners tobuild applicationsusing Guardian

content and servicesfor other platforms

What about Open In?

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

82/85

RDBMS

82

Apps

external hostingapp engine etc

Thursday, 26 May 2011

Core

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

83/85

Core

Cloud, EC2

OutIn

external hostingapp engine etc

rdbms

83

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

84/85

84

Thursday, 26 May 2011

8/6/2019 From Publisher To Platform: How The Guardian Embraced the Internet using Content , Search, and Open Source

85/85