Top Banner
cd neighborly-nagios/vm vagrant up server vagrant up client
91

Neighborly nagios

Apr 13, 2017

Download

Technology

David Josephsen
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Neighborly nagios

cd neighborly-nagios/vm vagrant up server vagrant up client

Page 2: Neighborly nagios

[email protected]@davejosephsen

github.com/djosephsen

Neighborly NagiosDave Josephsen !

Katops Illustrations by Emma Rongey

Page 3: Neighborly nagios

To Nagios or um.. to not Nagios

Page 4: Neighborly nagios
Page 5: Neighborly nagios
Page 6: Neighborly nagios

Nagios just works:• Availability Data • Minute + resolution • Threshold detection • Alert Handling • Scales “OK” • Ubiquitous • Will Teach you about Monitoring • Will make you a better engineer

Page 7: Neighborly nagios

Availability Data

Page 8: Neighborly nagios

Minute+ Scale

Page 9: Neighborly nagios

Performance Data

Page 10: Neighborly nagios

Second-Scale Resolution

Page 11: Neighborly nagios

400,000 Service Checks

Page 12: Neighborly nagios

WAT is Nagios?

Page 13: Neighborly nagios

KatopsHey.

Page 14: Neighborly nagios

KatopsBusyCorp

Page 15: Neighborly nagios

KatopsBusyCorp

Page 16: Neighborly nagios

Like it was my idea to run node.js in production

Page 17: Neighborly nagios

I can fix this in one line of shell

Page 18: Neighborly nagios

ping -c2 appserver1 || (echo ‘server down’ | /bin/mail [email protected]!

Page 19: Neighborly nagios

ping -c2 appserver1 || (echo ‘server down’ | /bin/mail [email protected]!

Yeah, basically I’m a genius.

Page 20: Neighborly nagios

Oh. Crud.

Page 21: Neighborly nagios
Page 22: Neighborly nagios

X

X X X

Page 23: Neighborly nagios

Sigh

Page 24: Neighborly nagios

does it do Oracle?

We heard you can monitor

thingsAdd me!!

Add me!!

Bob said you can set us up with

Monitoring

How about Apache web servers?

Page 25: Neighborly nagios

Problems:▪Too Many Alerts ▪No way to “squelch them” ▪Redundant Alerts ▪No dependency Tracking ▪No Alerts on system recoveries ▪Ping isn’t enough ▪No way to check ports, etc.. ▪Most people don’t want EVERY alert ▪No way to to track groups of users, and what

alerts they want.

Page 26: Neighborly nagios

Back to the Drawing Board▪Execute arbitrary scripts

Page 27: Neighborly nagios

Back to the Drawing Board▪Execute arbitrary scripts ▪Circular scheduling

Page 28: Neighborly nagios

Back to the Drawing Board▪Execute arbitrary scripts ▪Circular scheduling ▪First-class user/alert entities

Page 29: Neighborly nagios

Back to the Drawing Board▪Execute arbitrary scripts ▪Circular scheduling ▪First-class user/alert entities ▪Track inter-system dependencies

Page 30: Neighborly nagios

Back to the Drawing Board▪Execute arbitrary scripts ▪Circular scheduling ▪First-class user/alert entities ▪Track inter-system dependencies ▪Fine-grained alerting control

Page 31: Neighborly nagios

Nagios is a circular scheduler that executes user-provided scripts and interprets their output.

Page 32: Neighborly nagios

Notifications are also user-provided scripts

Page 33: Neighborly nagios

Hosts, Services, Users, Groups, and Alerts are all individually-defined first-class entities

Page 34: Neighborly nagios
Page 35: Neighborly nagios
Page 36: Neighborly nagios
Page 37: Neighborly nagios
Page 38: Neighborly nagios
Page 39: Neighborly nagios

0: OK 1: Warning 2: ERROR 3: Unknown

Page 40: Neighborly nagios

OK. Let’s do this

Page 41: Neighborly nagios
Page 42: Neighborly nagios
Page 43: Neighborly nagios

Alright lets get this thing installed already.

Page 44: Neighborly nagios
Page 45: Neighborly nagios
Page 46: Neighborly nagios
Page 47: Neighborly nagios
Page 48: Neighborly nagios
Page 49: Neighborly nagios
Page 50: Neighborly nagios
Page 51: Neighborly nagios

0: OK 1: Warning 2: ERROR 3: Unknown

Page 52: Neighborly nagios

resolve

Page 53: Neighborly nagios

command definitions and macros are your front-line data I/O hook

Page 54: Neighborly nagios
Page 55: Neighborly nagios
Page 56: Neighborly nagios
Page 57: Neighborly nagios
Page 58: Neighborly nagios

Commands are hooks

Page 59: Neighborly nagios

Commands provide context

Page 60: Neighborly nagios

$HOSTADDRESS$ -> ${NAGIOS_HOSTADDRESS}

Page 61: Neighborly nagios

https://assets.nagios.com/downloads/nagioscore/docs/nagioscore/4/en/macrolist.html

woah that’s awesome

Page 62: Neighborly nagios
Page 63: Neighborly nagios
Page 64: Neighborly nagios

Remote Execution

Page 65: Neighborly nagios
Page 66: Neighborly nagios
Page 67: Neighborly nagios
Page 68: Neighborly nagios
Page 69: Neighborly nagios

like every remote execution problem evar

SSH

Page 70: Neighborly nagios

Remote Execution Is Expensive

Page 71: Neighborly nagios

• RE Proxies • Agents • 3rd party queries

Page 72: Neighborly nagios

Remote Execution Proxy

Page 73: Neighborly nagios

Remote Execution Proxy

Page 74: Neighborly nagios

NRPE

Please run plugin X *runs plugin X*

Page 75: Neighborly nagios

NSClient++

check X asks WMI about X

Page 76: Neighborly nagios

Monitoring Agent

Page 77: Neighborly nagios

Monitoring Agent

Page 78: Neighborly nagios

Monitoring Agent

Page 79: Neighborly nagios

Monitoring Agent

NRDP

Page 80: Neighborly nagios

Check_MK

Page 81: Neighborly nagios

NCPA

Page 82: Neighborly nagios

3rd Party Queries

Page 83: Neighborly nagios

3rd Party Queries

collectd

Graphite/Graphana

Page 84: Neighborly nagios

3rd Party Queries

collectd

Graphite/Graphana

Page 85: Neighborly nagios

3rd Party Queries

collectd

Graphite/Graphana collectd-nagios

Page 86: Neighborly nagios

3rd Party Queries

Page 87: Neighborly nagios

you planning on talking all day or what?

Page 88: Neighborly nagios

NCPA

Page 89: Neighborly nagios

Data Out

Page 90: Neighborly nagios
Page 91: Neighborly nagios

Data Out