YOU ARE DOWNLOADING DOCUMENT

Please tick the box to continue:

Transcript
Page 1: Log Analysis using Open Source Scalable Systems - …samling:2013:log... · Log Analysis using Open Source Scalable Systems Gurvinder Singh Verktøysamling Oct'2013. ... gemfire filters

Log Analysis using Open Source Scalable SystemsGurvinder SinghVerktøysamling Oct'2013

Page 2: Log Analysis using Open Source Scalable Systems - …samling:2013:log... · Log Analysis using Open Source Scalable Systems Gurvinder Singh Verktøysamling Oct'2013. ... gemfire filters

Motivation

● Distributed Systems

● In the moment of heat

● Centeralized interface to logs

● Easier access

10/2/13 SLIDE 2

http://www.themeparkreview.com/tatsumediaday/tatsumediaday57.jpg

Page 3: Log Analysis using Open Source Scalable Systems - …samling:2013:log... · Log Analysis using Open Source Scalable Systems Gurvinder Singh Verktøysamling Oct'2013. ... gemfire filters

Challenges

● Almost every component generates logs

● Different Formats and logging methods

● Different requirements for processing

● Dashboards

● Alerts

10/2/13 SLIDE 3

http://img72.imageshack.us/img72/3885/nephew2logs.jpg

Page 4: Log Analysis using Open Source Scalable Systems - …samling:2013:log... · Log Analysis using Open Source Scalable Systems Gurvinder Singh Verktøysamling Oct'2013. ... gemfire filters

Components

● Logstash

● Redis

● Elasticsearch

● Kibana

● Beaver

● lumberjack

● .....

10/2/13 SLIDE 4

Page 5: Log Analysis using Open Source Scalable Systems - …samling:2013:log... · Log Analysis using Open Source Scalable Systems Gurvinder Singh Verktøysamling Oct'2013. ... gemfire filters

Architecture

Page 6: Log Analysis using Open Source Scalable Systems - …samling:2013:log... · Log Analysis using Open Source Scalable Systems Gurvinder Singh Verktøysamling Oct'2013. ... gemfire filters

Architecture

Page 7: Log Analysis using Open Source Scalable Systems - …samling:2013:log... · Log Analysis using Open Source Scalable Systems Gurvinder Singh Verktøysamling Oct'2013. ... gemfire filters

Architecture

Page 8: Log Analysis using Open Source Scalable Systems - …samling:2013:log... · Log Analysis using Open Source Scalable Systems Gurvinder Singh Verktøysamling Oct'2013. ... gemfire filters

Architecture

Page 9: Log Analysis using Open Source Scalable Systems - …samling:2013:log... · Log Analysis using Open Source Scalable Systems Gurvinder Singh Verktøysamling Oct'2013. ... gemfire filters

Architecture

Page 10: Log Analysis using Open Source Scalable Systems - …samling:2013:log... · Log Analysis using Open Source Scalable Systems Gurvinder Singh Verktøysamling Oct'2013. ... gemfire filters

Demo

Page 11: Log Analysis using Open Source Scalable Systems - …samling:2013:log... · Log Analysis using Open Source Scalable Systems Gurvinder Singh Verktøysamling Oct'2013. ... gemfire filters

Possible Input/Processing/Output Options

10/2/13SLIDE

11

Page 12: Log Analysis using Open Source Scalable Systems - …samling:2013:log... · Log Analysis using Open Source Scalable Systems Gurvinder Singh Verktøysamling Oct'2013. ... gemfire filters

Thanks:-)

Questions?

10/2/13SLIDE

12


Related Documents