Top Banner
Centralised Logging with Logstash and Kibana (and rsyslog, and elasticsearch, and ...) Matthew Richardson (Engineering) 18th January 2013
8

Centralised Logging with Logstash and Kibana - LCFG

Feb 12, 2022

Download

Documents

dariahiddleston
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Centralised Logging with Logstash and Kibana - LCFG

Centralised Loggingwith

Logstash and Kibana

(and rsyslog, and elasticsearch,

and ...)

Matthew Richardson(Engineering)

18th January 2013

Page 2: Centralised Logging with Logstash and Kibana - LCFG

Centralised Logging?

Page 3: Centralised Logging with Logstash and Kibana - LCFG

Rsyslog

Default syslogd in LCFG

Easy to log centrally Disk Buffering for

safety

$ActionQueueType LinkedList

$ActionQueueFileName LogstashBuffer

$ActionResumeRetryCount -1

$ActionQueueSaveOnShutdown on

$ActionQueueMaxDiskSpace 1G

*.* @@log.see.ed.ac.uk:5544RSYSLOG ?

Page 4: Centralised Logging with Logstash and Kibana - LCFG

Logstash

Inputs (25+)

Filters (20+)

Outputs (35+)

file grep filetcp/udp grok tcp/udpXMPP (Jabber)

mutate elasticsearch

log4j anonymize

XMPP (Jabber)

stdin dns emailWindows eventlog

XML ganglia

metrics graphite... ... ...

Page 5: Centralised Logging with Logstash and Kibana - LCFG

Logstash Configuration

TCP/UDP input Types and Tagging Powderful Filtering Structured Output

input { tcp { port => 5544 type => syslog }}filter { grok { pattern => [ "<%{POSINT:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{PROG:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" ] } syslog_pri { type => "syslog" } date { match => [ “syslog_timestamp”, "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ] } mutate { exclude_tags => "_grokparsefailure" replace => [ "@message", "%{syslog_message}" ] } mutate { remove => [ "syslog_message", "syslog_timestamp" ] }}output { elasticsearch { embedded => false }}

RSYSLOGLOGSTASH

tcp input

LOGSTASHFilters ?

Page 6: Centralised Logging with Logstash and Kibana - LCFG

Log Analysis

Elasticsearch backend Kibana web frontend

ES/Kibana

LOGSTASHtcp input

LOGSTASHFilter → JSON

RSYSLOGLOGSTASH

tcp input

LOGSTASHFilters

Page 7: Centralised Logging with Logstash and Kibana - LCFG

Future Plans

Encryption/Authentication for logging Handling other (non-syslog) logs Other Inputs Metrics

ES/KibanaLOGSTASHFilter → JSON

RSYSLOGTLS

LOGSTASHtcp(tls) input

LOGSTASHFilters

LOGSTASHXMPP input

LOGSTASHUDP input

JabberServer

HardwareDevice

ApacheLCFG

etc

Graphite/Ganglia

XMPP/Email

Notification

Page 8: Centralised Logging with Logstash and Kibana - LCFG

Questions?

[email protected](dst): mrichar1