Тимофей Титовец — Elastic+Logstash+Kibana: Архитектура и опыт использования

ElasticSearch Kibana Logstash

What is it?

l ElastiSearch — Store and Search enginel Logstash — Converter between text data

formatsl Kibana — Web Gui for visualize ES data

ElasticSearch

l Writen on Java working on Apache Lucene.l

Apache Lucene - high-performance, full-featured text search engine library

ElasticSearch: Index

Shard 1 Shard 2 Shard N

Shard 1Replica

Shard 2Replica

Shard NReplica

Index

ElasticSearch: Cluster

ESNode 1

ESNode 2

ESNode N

Logstash

l Writen on Java & Rubyl Can filter/edit/collect data, based on cool,

simple and powerfull language for writing rules.

Kibana

l NodeJS + JS client for ESl Can visualize data from ES

Common architecture

DATASOURCE

logstash

ESNode

Kibana 4

Log collection

Linux

rsyslog1

logstash

ESNode

Kibana 4

rsyslog2

rsyslogN

ESNode

ESNode

Windows

Windows 1NXLog

logstash

ESNode

Kibana 4

Windows 2NXLog

Windows nNXLog

ESNode

ESNode

IDS System: Suricata

OpenSource IDS & IPS System like snortCan sniff, analize and trasparent edit trafficAlso detect network attack, and defend network from it.Like very powerfull firewall

IDS

IDS 1

logstash

ESNode

Kibana 4IDS 2

ESNode

ESNode