Technology stack of social networks [MTS]

social networksThe technology stack of

Philipp Weber HDE MTS 2015.01.23

Table of Contents

● Overview

● Included networks

● The stacks

● Comments / Takeaway

*Some of the displayed numbers and

technologies reach back to the year of 2011.

**Rights to the registered Names / Logos /

Trademarks featured are property of and

belong to their respective owners.

Disclaimer

Facebook - 1.35 billion monthly active

users

Tumblr - 420 million users

Twitter - 284 million monthly active

users

LinkedIn - 300 million users

Google+ - 300 million monthly active

users

Included Networks

Frontend Backend

The stack

Databases Cache(Hosting)

Meta

Templating /

Javascript

Intermediate-

Layer

Programming

Languages

Server

specifics

Offline

processing

The stack - Facebook

● 864 million daily active

users

● 700 million mobile daily

active users

Facebook

Haystack

(Object-store)

Memcached

Facebook

markup

Language

PhpCassandra

Hadoop +

Hive

BigPipe

→ pagelets

Thrift

JavaVarnish

HTTP-accel.

HipHopVM -

Php-compiler

Frontend Backend Meta

The stack - Tumblr

● Microblog / social

networking website

● ~40k requests per second

● 15 billion page views per

month

● belongs to Yahoo

The stack - Tumblr

HBase

Redis

(Hosting)

Gearman and

Kestrel

PHP

HA-Proxy

Scala

Ruby

MySQL

Apache Kafka

Finagle

Frontend Backend Meta

The stack - Twitter

● 80% of Twitter active users

are on mobile

● Average of ~5750 Tweets/s

● Record 143.199 Tweets/s

Twitter

Flock Memcached(Hosting)Bootstrap

JVM

Finagle

Java

Netty

JavaScriptRedis

Hadoop+

Scalding

Scala

Frontend Backend Meta

User-Store

The stack - LinkedIn

● Leader in Professional

networking

● Opened their Network data

to busines-research in 2008

The stack - LinkedIn

Oracle Voldemort

Node.js

Java

GRails =

Java-Rails

Hadoop

Espresso Scala

JRuby

JavaScript

MySQLJQuery

Backbone.js

Frontend Backend Meta

The stack - Google+

● Build on Circle API

● Usually use special Google

technology and libraries

● Close interaction with

android planned

The stack - Google+

BigTable

Closure

Java

Custom JVM

GoogleFile

System

Closure C++

PythonJQuery

Frontend Backend Meta

Takeaway

So what can we learn from these examples?

Takeaways

#1 Embrace Open Source

Open Source systems benefit from university

research and a big community. Facebook

and Twitter release or found many of their

software in OpenSource projects.

Takeaways

#2 Scalability:

Prepare for scalability starting at the

foundation. Monolithic systems make rapid

growth impossible or error-proned.

Takeaways

#3 Discover new solutions:

Highscalability.com writes a very in-depth

technology-blog with interviews from tech-

giants and their experiences. Definitely worth

a read.

Thank you for

your Attention

Sources

Facebook

https://github.com/facebook

http://www.slideshare.net/meet.hak/facebook-technology-stack

http://newsroom.fb.com/company-info/

www.infoq.com/presentations/Scale-at-Facebook

Twitter

https://about.twitter.com/company

https://blog.twitter.com/2013/new-tweets-per-second-record-and-how

http://www.slideshare.net/caniszczyk/twitter-opensourcestacklinuxcon2013

Tumblr

http://www.theguardian.com/technology/2014/oct/23/tumblr-yahoo-revenues-2015-ads-nsfw

http://highscalability.com/blog/2012/2/13/tumblr-architecture-15-billion-page-views-a-month-and-harder.html

LinkedIn

https://press.linkedin.com/news-releases

https://engineering.linkedin.com/technology

https://www.youtube.com/watch?x-yt-ts=1421914688&x-yt-cl=84503534&v=OXN3wuHUBP0

Google+

http://www.usatoday.com/story/tech/2013/10/29/google-plus/3296017/

http://highscalability.com/blog/2011/7/12/google-is-built-using-tools-you-can-use-too-closure-java-ser.html