Logstash, ElasticSearch, Kibana

ELK Cecil

EasticSearch: Logstash: Kibana: ElasticSearch Viszualazation Tool

ServerLogs

ELK? ( )

Logstash (input -> filter -> output)

input: filter: (, )output:

Logstash ./logstash -f configfile : input, filter, output input { // Input pluginfile {path => ../apiTrans.log}}filter { // filter plugingrok {match => {message => %{COMBINEDAPACHELOG}}}geoip {source => [clientip]"}}output { // output pluginelasticsearch {hosts => ["http://localhost:9200" ]}}

ElasticSearch (full-text) / ( , , )

, , , , ,JSON , Restful API Features Elasticsearch

ELK curl -XPUT 'localhost:9200/customer/external/1?pretty' -d '{ "name": "John Doe"}'curl -XGET 'localhost:9200/customer/external/1?pretty'{ "_index" : "customer", "_type" : "external", "_id" : "1", "_version" : 1, "found" : true, "_source" : { "name": "John Doe" }}

RDBDatabaseTableRowColumnSchemaElastic SearchIndexTypeDocumentFieldMapping

RDB : : Mapping

Kibana (with ElasticSearch)(Elastic Search )

Discover: Elasticsearch Visualize: Dashboard: 2

VisualizeElasticsearch aggregation Bucket: metric:

MetricBucket{ "query": {} , "aggs" : { "grades_stats" : { "stats" : { "field" : "grade" } } }}{ "query": {} , "aggs" : { "genders" : { "terms" : { "field" : "gender" } } }}{ ... "aggregations": { "grades_stats": { "count": 6, "min": 60, "max": 98, "avg": 78.5, "sum": 471 } }}{ ... "aggregations" : { "genders" : { ... "buckets" : [{ "key" : male", "doc_count" : 10 },{ "key" : female", "doc_count" : 10 }, ] } }}

Production Broker Elastic Search / .ex) syslog

Redis Production Environment

Referenceselastic, https://www.elastic.co