Elasitcsearch + Logstash + Kibana 日誌監控

ELK Ansible + Vagrant 10 ELK

[email protected]

2015/10/17

•

• API

•

• ...

• DAU

•

•

•

• NagiosZabbix

• elastic.co elasticsearch elasticsearch

• elasticsearch shard inverted index : http://www.slideshare.net/rueian3/elasticsearch-45855699

ELK• ELK = elasticsearch + logstash + kibana

• elastic.co elasticsearch logstash

kibana

• elasticsaerch logstash kibana ELK logstash-forwarder beats watcher

logstash

• logstash ( RSS) elasticsearch

• logstash Java Fluentd

https://www.elastic.co/products/logstash

logstash-forwarder

• logstash-forwarder elastic.co logstash go

logstash

• logstash-forwarder elastic.co filebeat

https://github.com/elastic/logstash-forwarder

kibana

• kibana Hapi Node.js Web UI elasticsearch

• kibana Grafana Grafana Graphite v2.5

elasticsearch

https://www.elastic.co/products/kibana

kibana

beats• logstash elastic.co

beats

• packetbeat HTTPThrift-RPC Mysql PostgreSQL MongoDB RedisMemcache protocol

• topbeat process CPU

• filebeat logstash-forwarder

https://www.elastic.co/products/beats

Marvel

• Marvel Web UI elasticsearch elasticsearch cluster

CPU Index

https://www.elastic.co/products/marvel

Marvel

watcher•

• watcher elastic.co elasticsearch

email slack webhook

• watcher

https://www.elastic.co/products/watcher

logstash

https://www.elastic.co/guide/en/logstash/2.0/deploying-and-scaling.html

logstash

elasticsearch elasticsaerch

https://www.elastic.co/guide/en/logstash/2.0/deploying-and-scaling.html

logstash

Redis RabbitMQ

logstash

logstash

packetbeat & topbeat

packetbeat elasticsaerch

https://www.elastic.co/guide/en/beats/packetbeat/current/packetbeat-logstash.html

packetbeat & topbeat

Redis logstash elasticsearch logstash

https://www.elastic.co/guide/en/beats/packetbeat/current/packetbeat-logstash.html

Marvel Watcher

• elastic.co Marvel Watcher elasticsearch

• elasticsearch

https://www.elastic.co/guide/en/watcher/watcher-1.0/installing-watcher.htmlhttps://www.elastic.co/guide/en/marvel/current/_installation.html

nginx

nodejs

nodejs

nodejs

postgrespostgres

elastic search

elastic search

elastic search

elastic search monitor

logstashredis borker

logstash

logstash

logstash-forwarder

logstashredis borker

packetbeat & topbeat

elastic search monitor

elasticsearch • elasticsearch

• Index shard 5 5 elasticsearch shard

• ES_HEAP_SIZE elasticsearch

• 64000 1024 elasticsearch

• JVM swapping elasticsearch elastic.co swapping

1. sudo swapoff -a swapping

2. sysctl vm.swappiness 0

3. elasticsearch bootstrap.mlockall: true

https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html

elasticsearch

• elasticsearch

• Index Curator

• Index Curator elasticsearch

repository ( S3)

https://www.elastic.co/guide/en/elasticsearch/client/curator/current/_features.html

elasticseach

• elasticsearch

elasticsearch

• logstash statsD Graphite Graphite

1s 1m

ELK • Ansible + Vagrant ELK

https://github.com/rueian/ansible-elk-example

• vagrant up kibana

README

• Vagrant

ELK

nginx + nodejs

postgres

elastic search

elastic search

redis + logstash

logstash

logstashelastic search kibana

redis + logstash

logstash-forwarder packetbeat

ELK

redis + logstash

topbeat

• [ ] Fluentd: An data collector for unified logging layer

• [ ] StatsD: Simple daemon for easy stats aggregation

• [ ] Grafana: Gorgeous metric viz, dashboards & editors for Graphite, InfluxDB & OpenTSDB

• [ ] Graphite: Scalable Realtime Graphing

• [ ] Nagios: The Industry Standard In IT Infrastructure Monitoring

• [ ] Zabbix: The Enterprise-class Monitoring Solution for Everyone

• [ ] QBox: Optimizing Elasticsearch: How Many Shards per Index?

• [ ] QBox: What is Elasticsearch, and How Can I Use It?

• [ ] Elasticsearch

• [ ] Ansible: Application Deployment + Configuration Management + Continuous Delivery

• [ ] Vagrant: Create and configure lightweight, reproducible, and portable development environments

Thanks

Any Questions?

[email protected]