Welcome to criteo factory: 72k jobs per day

72K JOBS PER DAYW E L C O M E T O C R I T E O F A C T O R YH U G R E N N E S – S E P T E M B E R 3 0 2 0 1 5

Copyright © 2015 Criteo

2About me

Nicolas Helleringer

DevOps by nature

Engineering Manager

Site Reliability Engineering

@nhelleringer

3

Criteo ?Performance Display

Advertisement

27 offices

in the worldMulti

channels

1 billion

Internet users

reach

1600

employees

A behavior driven

recommendation algorithm

is the core of the company Founded

by 3

engineers

Technology is a huge driver

for the R&D and also for all the

company

C# basis SQL Server

Best relational DB on the market

Replication

Ramp up

on proven

technologies

› Good efficiency

› Known scaling

Ease of use

Microsoft

stack

Quick deployment

Homogeneous data layers

Online, offline and BI

Scaled for quite a long time

Investments in infrastructure

6

Technical insightsOf Machines and Men

540 engineers

1/3 of the

company

15k servers

3B display a day

1 million http

requests/sec

Scale of data

Machine Learning

Prediction

Hadoop

Rocks !Data repository

BI, Engine and

Research

Security - Kerberos

Try, failLearn, Repeat

Try at scale !

Be prepared to dig

into details

Test your use case

not a random

benchmark

HDFS raw

39 PiB

51 PiB

2 big clusters

2x 15k Cores

Hadoop at scale

1200 servers

700 servers

4 local clusters

240/320 Cores

1 pre prod cluster

1800 Cores

1 dev cluster

1200 Cores

Automation

Team of 4

Chef tooling

Gateways for users

Job scheduling

Slider ?

Resiliency

Compute power

vs

SLAs

All on the clusterRoles outside the

cluster are hard to

maintain

13

User Service

Thank you [email protected]