Measure your app internals with InfluxDB and Symfony2

Measure your app internals with InfluxDB(time-series database)Symfony Day 2015 - Reggio Emilia

Trust me

Why we measure things?

Measure - Learn - Build - Measure

Measure for a change

Measure for discovery

Measure for compare

What we need?* A data provider (our application)

* A database that will collects data

* A dashboard for representing information

What is a time-series database?A database optimized for handling time series data:

arrays of numbers indexed by time(a datetime or a datetime range)

sign elem is 4 -> @10:13 elem is 4

A time series is a sequence of data points, measured typically atsuccessive points in time spaced at uniform time intervals

Time series Databases

InfluxDBgithub.com/influxdb

InfluxDBEasy to useWritten in Golang (no external dependencies)ready for bilion of rows (scalable)UDP/IP and HTTP network adaptersSeveral client libraries (PHP, Golang, etc...)Integrated query web panelReady for external dashboards (grafana, chronograf, etc...)

Write information in your InfluxDBcurl ­XPOST 'http://localhost:8086/write?db=mydb' \ ­d 'cpu load=2 1434055562000000000'

Read data back from InfluxDBcurl ­G http://localhost:8086/query?pretty=true ­­data­urlencode "db=mydb" \ ­­data­urlencode "q=SELECT * FROM cpu WHERE host='server01' AND time < now() ­ 1d"

For example:

InfluxDB tagsYou can mark new points with a tag information

sign @10.32 key=value tag1=value tag2=value

sign @10.32 temperature=21.3 location=home env=test

sign @10.32 cpu=21.3,mem=123.3 location=eu­west­1 env=dev instance=i­a91afcd1

Use tags in your queriesExample

SELECT * FROM home_sensors WHERE location="home" AND temperature > 19.2 AND time >= now() ­ 10m

Example for websites:SELECT * FROM index_page WHERE env="prod" AND load_time > 19.2 AND time >= now() ­ 1d

Just use them like string column values

Play from the InfluxDB web consolehttp://your-domain.tld:8083/

GRAFANA DASHBOARD

An open source, feature rich, metrics dashboard and graph editorfor several time series databases (InfluxDB, Graphite, etc.)

Annotate your graphs

Thanks to annotations you can use a serieas an annotation list

Mark a new software release in your graph (Continuous Delivery)Mark any critic event in all your graphs (Information Point)

CHRONOGRAF DASHBOARD

A new data visualization tool for InfluxDB.

How to send my application data?* Send every single point directly in real-time

* Collect data during the application life-cycle and send thosepoints once per request

InfluxDB supports two network adapters

HTTP or UDP/IPWhich one we should have to use?

UDP/IP must be enabled by configuration

HTTP* TCP/IP based

* Data point delivery guaranteed

* Error handling

* ~ 374 Ops/second (for compare with UDP/IP)

UDP/IP* No delivery guarantees

* No error handling

* ~ 22917 Ops/second (for compare with TCP/IP)

HTTP* Sensitive data

* Good for once at a time strategy (or if we do not care about monitor impact on our application)

UDP/IP* Good for immediate delivery

InfluxDB-PHP-SDKhttps://github.com/corley/influxdb-php-sdk

Supports both: HTTP and UDP/IPStar 46 Fork 13

composer require corley/influxdb-sdk

How it works$client­>mark("serie_name", [ "customer_name" => "walter", "item_price" => 100.12, //...]);

Concise Format (good for immediate delivery)

How it works (2)$client­>mark([ "points" => [ [ "measurement" => "instance", "fields" => [ "cpu" => 18.12, "free" => 712423, ], ], [ "measurement" => "page_load", "fields" => [ "listeners_speed" => 1.12, "counted_elements" => 341143, ], ], ]]);

Extended Format (good for a bulk strategy)

InfluxDB ClientUDP/IP + HTTP

$httpOptions = new Options();$reader = new Http\Reader($httpOptions);$udpOptions = new Options();$writer = new Udp\Writer($udpOptions);$client = new Client($reader, $writer);

HTTP only

$http = new \GuzzleHttp\Client();

$options = new Options();$reader = new Http\Reader($http, $options);$writer = new Http\Writer($http, $options);$client = new Client($reader, $writer);

Supports Guzzle versions: ~4, ~5, ~6

Symfony: Dependency Injection Containerparameters: influxdb_host: "localhost" influxdb_port: 4444

services: influxdb: class: InfluxDB\Client arguments: ["@influxdb_reader", "@influxdb_writer"]

influxdb_writer: class: InfluxDB\Adapter\Writer\Udp arguments: ["@influxdb_udp_options"]

influxdb_reader: class: InfluxDB\Adapter\Http\Reader arguments: ["@influxdb_http_options"]

Symfony: Dependency Injection Container#Only options

influxdb_udp_options: class: InfluxDB\Options calls: ­ ["setHost", ["%influxdb_host%"]] # InfluxDB host ­ ["setPort", ["%influxdb_udp_port%"]] # UDP/IP Port ­ ["setTags", ["env": "%kernel.environment%"]] # tag every point with env

influxdb_http_options: class: InfluxDB\Options calls: ­ ["setHost", ["%influxdb_host%"]] # InfluxDB host ­ ["setPort", ["%influxdb_http_port%"]] # HTTP Port

parameters: influxdb_host: "influxdb.domain.tld" influxdb_udp_port: 4444 influxdb_http_port: 8086

Don't try this at homeclass DefaultController extends Controller public function indexAction(Request $request) $client = $this­>container­>get("influxdb");

$client­>mark(...);

Goals: single-responsibility, reduce coupling, simplifing testing

Controllers as a service?For a monitor? Make sense?

With eventsclass DefaultController extends Controller public function indexAction(Request $request) //... $dispatcher­>dispatch("my_event_name", $event);

Event listener

class MyMonitorAwareForMyEventListener public function __construct(Client $client) ...

public function onMyEvent(Event $event) $this­>client­>mark(...);

Deregister this listener in order to stop data collection

Compose your listenermonitor_listener: class: AppBundle\Listener\MonitorListener arguments: ["@influxdb", "@debug.stopwatch"] tags: ­ name: kernel.event_listener, event: kernel.request, method: startStopwatch ­ name: kernel.event_listener, event: kernel.response, method: stopStopwatch

The stopwatch will help us to collect page timing information

A listener example with stopwatchclass MonitorListener public function startStopwatch(GetResponseEvent $event) $routeName = $event­>getRequest()­>attributes­>get("_route"); $this­>stopwatch­>start($routeName); public function stopStopwatch(FilterResponseEvent $event) $routeName = $event­>getRequest()­>attributes­>get("_route"); $events = $this­>stopwatch­>stop($routeName); foreach ($events­>getPeriods() as $measure)

$this­>client­>mark($routeName, [ "memory" => $measure­>getMemory(), //... ]);

Check your data via InfluxDB

Prepare your dashboard with grafana

Checkout out the app examplehttps://github.com/wdalmut/symfonyday2015-example

Thanksgithub.com/wdalmut