WoMakersCode 2016 - Shit Happens

Empower is the first step for new revolutions

The first step in any revolution is going to be the moment of realization. If you've had enough, and are ready to commit to your dreams, that is when the real revolution begins. So i ask you...have YOU had enough?O primeiro passo para qualquer revolução vai ser o momento da realização. Se você já teve o suficiente, e está pronto para comprometer-se a seus sonhos, que é quando a verdadeira revolução começa. Então pergunto a vocês ... Vocês já tem o suficiente?

Shit happensCentralize logs and get your insight into the errors that

affect your customers.

Who?

Jackson F. de A. Mafrahttp://about.me/jacksonfdam https://bitbucket.org/jacksonfdam https://github.com/jacksonfdam http://linkedin.com/in/jacksonfdam @jacksonfdam

Software Engineer at Aggrega Group, mobile training instructor at Targettrust. Developer for 15 years with background in e-commerce projects and real estate, since 2009 with focused interests for the development of mobile and MEAP and applications interfaces.

Aspect oriented programming (AOP) allows us to keep implement different concerns in

isolation

What?

The term Aspect-Oriented Programming took shape in the mid-1990s, inside a small group at

Xerox Palo Alto Research Center (PARC).

AOP was considered controversial in its early days — as is the case with any new and

interesting technology — mostly due to its lack of clear definition. The group made the

conscious decision to release it in a half-baked form, in order to let the larger community

provide feedback. At the heart of the problem was the "Separation of Concerns" concept. AOP

was one possible solution to separate concerns.

Whereas DI helps you decouple your application objects from each other, AOP helps you decouple cross-cutting concerns from the

objects that they affect.

A PHP Developers Perspective...

Aspect Oriented Programming/Architecture is a practice in SOLID design principles.

It is an attempt to further abstract specific cross application concerns within your code - using a

techinique to intercept points within your call stack to perform specific functionality at given

times.

A PHP Developers Perspective...

Concerns, like security, cut across the natural units of modularity. For PHP the natural unit of

modularity is the class. But in PHP crosscutting concerns are not easily turned into classes

precisely because they cut across classes, and so these aren’t reusable, they can’t be refined or inherited, they are spread through out the

program in an undisciplined way, in short, they are difficult to work with.

Why?

Centralize concerns implementation More reusable code

Cleaner code Write less code

Easy to understand More maintainable

Less boilerplate code More interesting work

Why AOP?

Caching Profiling Security Pooling

Exception Handling Transactions

Logging

Concern

Program execution

Join Points

Advice

Pointcut

Terminology

Aspects are often described in terms of advice, pointcuts, and join points.

Terminology

Advice defines what needs to be applied and when. Jointpoint is where the advice is applied. Pointcut is the combination of different joinpoints where the advice needs to be applied. Aspect is applying the Advice at the pointcuts.

Definitions

Definitions

Method Method Method

Concern

Concern

Advice

Join Points

Logger

TransactionManager

Advice Types

Method

Method

Method

Method

Exception

Before advice

After advice

After returning advice

Around advice

Throws advice

AOP is a PECL extension that enables you to use Aspect Oriented Programming in PHP, without the need to compile or proceed to any other intermediate step before publishing your code.

The AOP extension is designed to be the easiest way you can think of for integrating AOP to PHP.

AOP aims to allow separation of cross-cutting concerns (cache, log, security, transactions, ...)

https://github.com/AOP-PHP/AOP

AOP

You can use pecl

sudo pecl install aop-beta

Installation

Basic tutorial

Now you want your code to be safe, you don't want non admin users to be able to call authorize methods.

Basic tutorial

Add some code to check the credentials "IN" you UsersServices class. The drawback is that it will pollute your code, and your core service will be less readable.

Let the clients have the responsibility to check the credentials when required. The drawbacks are that you will duplicate lots of code client side if you have to call the service from multiple places

Add some kind of credential proxy that will check the credentials before calling the actual service. The drawbacks are that you will have to write some extra code, adding another class on the top of your services.

What are your solutions ?

Moreover, those solutions tends to increase in complexity while you are adding more cross-cutting concerns like caching or logging.

What are your solutions ?

That's where AOP comes into action as you will be able to tell PHP to do some extra actions while calling your MyServices's admin methods.

What are your solutions ?

So let's first write the rule needed to check if we can or cannot access the admin services.

What are your solutions ?

Dead simple : we check the current PHP session to see if there is something telling us the current user is an admin (Of course we do realize that you may have more complex routines to do that, be we'll keep this for the example)

What are your solutions ?

Now, let's use AOP to tell PHP to execute this method "before" any execution of admin methods.

What are your solutions ?

Now, each time you'll invoke a method of an object of the class UsersServices, starting by authorize, AOP will launch the function basicAdminChecker before the called method.

What are your solutions ?

Logging is an important part of the app development/maintenance cycle.

Logging

To know the best method of logging data of different contexts for specific environments

such as test/dev and production

Take Away

Even with use of computers there was a real need to measure the overall performance of any reasearch

Early 1980's there was a Instrument called VELA (virtual laboratory) used for data harvesting

History of Logging

Late 1980's, A device was invented to collect information through sensors

Later then data logging/harvesting has been used widely in all applications/reasearches/products.

History of Logging

Track Users activity/Movement

Transaction Logging

Track user errors

System level failures/warnings

Research Data collection and Interpretation

Need of Logging

Error / Exception logs

Access logs

System logs

Application logs

Database logs

Transaction logs

Mailer logs etc...

Types of Logging

Apache NGINX

PostgreSQL MySQL

php php-fpm

System Logs

Debug Information - Errors (connections, uncaught exceptions, resource exhaustion)

Narrative Information - Methods Calls, Event Triggers

Business Events - Purchases, Logins, Registrations, Unsubscribes

Application Log

ssh [email protected] tail -f /var/log/nginx/my-site.access.log tail -f /var/log/my.application.log

ssh [email protected] tail -f /var/log/mysql/mysql.log

ssh [email protected]

tail -f /var/log/rabbitmq/nodename.log

Keeping Track Of All This

Apache/PHP <VirtualHost *:80>

<Directory /var/www/html/> Options Indexes FollowSymLinks MultiViews AllowOverride All Order allow,deny allow from all

</Directory> ErrorLog ${APACHE_LOG_DIR}/error.log LogLevel warn CustomLog ${APACHE_LOG_DIR}/access.log combined

</VirtualHost>

Current Conventions

Monolog is a PHP library that support different levels of logging for PHP Applications and depends on PSR.

Inspired by Python Logbook library

Provides stack of handlers

More Powerful than conventional way of logging in applications

Monolog Enters Here

Monolog sends your logs to files, sockets, inboxes, databases and various web services.

Channel based approach

Different stack of handlers for specific channels

Pile up handler stack based on severity.

Format Interpretation depending on severity and channel

Prevents Bubbling when severity is reached

What's different ?

Log Levels 2013 - PSR03 - PHP Logging Interface Standard

Phrase / Severity

emergency Emergency: system is unusable alert Alert: action must be taken immediately critical Critical: critical conditions error Error: error conditions warning Warning: warning conditions notice Notice: normal but significant condition info Informational: informational messages debug Debug: debug-level messages

http://www.php-fig.org/psr/psr-3/

Log Levels

What about Apache’s error_log?

error_log is too basic (message, file, line)

difficult to read / parse

depends on “error_reporting” setting

Why?

monolog

phpconsole

log4php

RavenPHP + Sentry

FirePHP (dev environment)

Roll your own Logging Options

Logging Options

Fire & forget

Minimum or zero latency

Highly available

Should be PSR-3 compatible

Log everything:

- Exceptions - Errors - Fatal Errors

Requirements (for everyone)

Typical PSR-3 Compatible Design

Capture Method

Logger (PSR-3)

Handler / Adapter

Data Storage

MonologMonologErrorHandler ->

handleException()

MonologLogger ->log()

MonologHandler ->handle()

MongoDB

Option to have different channel for different module

Custom detailing

Different handlers for different development

Thorough participation in different stages of lifecycle

Open for third party integration

Readable and Beautiful Layered message

Advantages

PSR-3 makes it easy

However you want…

Monolog has loads:

- syslog-compatible / error_log

- Email, HipChat

- AMQP, Sentry, Zend Monitor, Graylog2

- Redis, MongoDB, CouchDB

Sending Log Messages

CakePHP - https://github.com/jadb/cakephp-monolog Symfony2 - https://github.com/symfony/MonologBundle Slim – https://github.com/flynsarmy/Slim-Monolog Zend2 - https://packagist.org/packages/enlitepro/enlite-monolog CodeIgniter - https://github.com/pfote/Codeigniter-Monolog Laravel – Inbuilt Support. Drupal - https://drupal.org/project/monolog Wordpress - https://packagist.org/packages/fancyguy/wordpress-monolog

more: https://github.com/Seldaek/monolog#frameworks-integration

Do you use Frameworks / CMS ?

Monolog is available on Packagist, which means that you can install it via Composer.

composer require 'monolog/monolog:1.13.*'

Installation

Basic Usage

Loggers And Handlers

Loggers And Handlers

Loggers And Handlers

Event Logging

http://www.sitepoint.com/logging-with-monolog-from-devtools-to-slack/

More usages

Stop logging exceptions the old fashioned way.

The Elk Stack

Indexing and search engine

Near real-time

Distributed, auto-discover clustering

– AWS Plugin

Elasticsearch

Collects logs

Parses, extracts and formats data

Passes data to Elasticsearch

Logstash

example filter { if [file] == "/var/log/secure" and (

[syslog_message] =~ /Invalid user/ or

[syslog_message] =~ /User root from/ ) {

grok {

add_tag => [ "LOGIN" ]

match => {"syslog_message" => “user %{ WORD:username}

from %{IP:srcip}" }

}

}

}

Logstash

Web interface to query Elasticsearch

node.js

Kibana

Kibana

Kibana

WHAT IS REALTIME?

THERE IS ALWAYS A DELAY

HOW MUCH DELAY CAN YOU ACCEPT?

ARCHITECTURE OF DELAY

DATA LIFECYCLE

DATA LIFECYCLE

DATA LIFECYCLE

DATA LIFECYCLE

DATA LIFECYCLE:ELK

DATA LIFECYCLE:ELK

DATA LIFECYCLE:ELK

DATA LIFECYCLE:ELK

DATA LIFECYCLE:ELK

Logstash Architecture

AWS Architecture

I recommend

Questions?

Thank you.