Neighborly nagios

cd neighborly-nagios/vm vagrant up server vagrant up client

dave@librato.com@davejosephsen

github.com/djosephsen

Neighborly NagiosDave Josephsen !

Katops Illustrations by Emma Rongey

To Nagios or um.. to not Nagios

Nagios just works:• Availability Data • Minute + resolution • Threshold detection • Alert Handling • Scales “OK” • Ubiquitous • Will Teach you about Monitoring • Will make you a better engineer

Availability Data

Minute+ Scale

Performance Data

Second-Scale Resolution

400,000 Service Checks

WAT is Nagios?

KatopsHey.

KatopsBusyCorp

KatopsBusyCorp

Like it was my idea to run node.js in production

I can fix this in one line of shell

ping -c2 appserver1 || (echo ‘server down’ | /bin/mail katops@busycorp.com!

ping -c2 appserver1 || (echo ‘server down’ | /bin/mail katops@busycorp.com!

Yeah, basically I’m a genius.

Oh. Crud.

X

X X X

Sigh

does it do Oracle?

We heard you can monitor

thingsAdd me!!

Add me!!

Bob said you can set us up with

Monitoring

How about Apache web servers?

Problems:▪Too Many Alerts ▪No way to “squelch them” ▪Redundant Alerts ▪No dependency Tracking ▪No Alerts on system recoveries ▪Ping isn’t enough ▪No way to check ports, etc.. ▪Most people don’t want EVERY alert ▪No way to to track groups of users, and what

alerts they want.

Back to the Drawing Board▪Execute arbitrary scripts

Back to the Drawing Board▪Execute arbitrary scripts ▪Circular scheduling

Back to the Drawing Board▪Execute arbitrary scripts ▪Circular scheduling ▪First-class user/alert entities

Back to the Drawing Board▪Execute arbitrary scripts ▪Circular scheduling ▪First-class user/alert entities ▪Track inter-system dependencies

Back to the Drawing Board▪Execute arbitrary scripts ▪Circular scheduling ▪First-class user/alert entities ▪Track inter-system dependencies ▪Fine-grained alerting control

Nagios is a circular scheduler that executes user-provided scripts and interprets their output.

Notifications are also user-provided scripts

Hosts, Services, Users, Groups, and Alerts are all individually-defined first-class entities

0: OK 1: Warning 2: ERROR 3: Unknown

OK. Let’s do this

Alright lets get this thing installed already.

0: OK 1: Warning 2: ERROR 3: Unknown

resolve

command definitions and macros are your front-line data I/O hook

Commands are hooks

Commands provide context

$HOSTADDRESS$ -> ${NAGIOS_HOSTADDRESS}

https://assets.nagios.com/downloads/nagioscore/docs/nagioscore/4/en/macrolist.html

woah that’s awesome

Remote Execution

like every remote execution problem evar

SSH

Remote Execution Is Expensive

• RE Proxies • Agents • 3rd party queries

Remote Execution Proxy

Remote Execution Proxy

NRPE

Please run plugin X *runs plugin X*

NSClient++

check X asks WMI about X

Monitoring Agent

Monitoring Agent

Monitoring Agent

Monitoring Agent

NRDP

Check_MK

NCPA

3rd Party Queries

3rd Party Queries

collectd

Graphite/Graphana

3rd Party Queries

collectd

Graphite/Graphana

3rd Party Queries

collectd

Graphite/Graphana collectd-nagios

3rd Party Queries

you planning on talking all day or what?

NCPA

Data Out

Data Out