Nagios Certified Professional

Preparation for the Nagios Certified Professional Certification Exam.

Working Lab Manual

This book is designed to be a working manual, a book you can write notes in, underline and use as a reference for a
long time. The manual is loaded with Labs to learn and practice skills that you are developing.

Lab – a short training option to illustrate one aspect of the manual

Note: The labs not only contain practical application for information that has already been presented but they also
contain new information. This means that the labs are an essential part of the learning process.

Date of Manual Version: March 22, 2012

Copyright and Trademark Information

Nagios is a registered trademark of Nagios Enterprises. Linux is a registered trademark of Linus Torvalds. Ubuntu
registered trademarks with Canonical. Windows is a registered trademark of Microsoft Inc. All other brand names
and trademarks are properties of their respective owners.

The information contained in this manual represents our best efforts at accuracy, but we do not assume liability or
responsibility for any errors that may appear in this manual.
Table of Contents
About This Manual....................................................................................................................................6
Intended Audience.....................................................................................................................................6
Preparation for Exercises............................................................................................................................6
Chapter 1: Introduction..............................................................................................................................1
Nagios Monitoring Solutions................................................................................................................1
Technical Support.............................................................................................................................2
Official Training...............................................................................................................................2
Nagios Terminology..............................................................................................................................3
Plugins .............................................................................................................................................3
Host ..................................................................................................................................................6
Service .............................................................................................................................................6
Users.................................................................................................................................................6
Contacts............................................................................................................................................6
Contactgroups...................................................................................................................................6
Acknowledgment..............................................................................................................................7
Downtime..........................................................................................................................................8
Disabled............................................................................................................................................8
Latency.............................................................................................................................................9
State..................................................................................................................................................9
Host and Service States...................................................................................................................10
Agents.............................................................................................................................................10
Unhandled.......................................................................................................................................11
Installation ..........................................................................................................................................11
Chapter 2: Configuration .........................................................................................................................13
Initial Set Up........................................................................................................................................13
Contact Information........................................................................................................................13
Pre­Flight Check.............................................................................................................................13
Creating a Password........................................................................................................................15
Eliminating the HTTP Error...........................................................................................................15
Nagios Check Triangle........................................................................................................................15
Nagios Checks.....................................................................................................................................17
Active..............................................................................................................................................17
Passive ............................................................................................................................................18
Security Risks......................................................................................................................................19
Chapter 3: Updates...................................................................................................................................21
Checking for Updates..........................................................................................................................21
Chapter 4: User Management...................................................................................................................23
Authentication and Privileges.........................................................................................................23
Authentication.................................................................................................................................23
Notification ....................................................................................................................................28
Multi_Level Notifications...............................................................................................................31
Escalation........................................................................................................................................34
Notification: Host and Service Dependencies................................................................................39
Chapter 5: Management...........................................................................................................................41
Web Interface ......................................................................................................................................41
 Home...............................................................................................................................................41
Documentation................................................................................................................................42
Tactical Overview...........................................................................................................................43
Map.................................................................................................................................................44
Hosts...............................................................................................................................................46
Services ..........................................................................................................................................49
Host Groups....................................................................................................................................50
Service Groups................................................................................................................................51
Problems.........................................................................................................................................52
Quick Search...................................................................................................................................53
Availability......................................................................................................................................54
Trends.............................................................................................................................................56
Alerts..............................................................................................................................................58
Notifications...................................................................................................................................62
Event Log........................................................................................................................................62
Comments.......................................................................................................................................63
Downtime.......................................................................................................................................64
Process Info.....................................................................................................................................67
Performance Info ...........................................................................................................................68
Scheduling Queue ..........................................................................................................................69
Configuration..................................................................................................................................70
Event Handlers.....................................................................................................................................71
Host Groups.........................................................................................................................................74
Service Groups....................................................................................................................................76
Managing Nagios Time.......................................................................................................................77
Nagios Core BackUp...........................................................................................................................78
Reachability ........................................................................................................................................81
Network Outages.................................................................................................................................86
Volatile Service ..................................................................................................................................86
State Stalking.......................................................................................................................................86
Flapping...............................................................................................................................................86
Resolving Problems.............................................................................................................................89
Disabling Notifications........................................................................................................................90
Sending Mail From Nagios..................................................................................................................91
Commit Error from the Web Interface ...............................................................................................94
Chapter 6: Monitoring..............................................................................................................................97
Plugin Use...........................................................................................................................................97
Monitoring Public Ports......................................................................................................................97
check_ping......................................................................................................................................98
check_tcp........................................................................................................................................98
check_smtp.....................................................................................................................................99
check_imap...................................................................................................................................100
check_simap..................................................................................................................................101
check_ftp.......................................................................................................................................101
check_http.....................................................................................................................................102
check_mysql..................................................................................................................................104
 Monitoring Linux..............................................................................................................................108
NRPE Concepts............................................................................................................................108
SSH Concepts................................................................................................................................111
Monitoring Windows.........................................................................................................................113
NSClient++ Concepts...................................................................................................................113
MSSQL .........................................................................................................................................116
Log Monitoring..................................................................................................................................117
Monitor Nagios Logs.....................................................................................................................118
Network Printers................................................................................................................................119
Checking Printers with SNMP......................................................................................................121
Chapter 7: Practical Exercises................................................................................................................125
Exercise #1: Login and Research.......................................................................................................125
Exercise #2: Responding to Problems...............................................................................................130
Exercise #3: Reports..........................................................................................................................135
Exercise #4: Passive vs. Active Checks.............................................................................................142
About This Manual
The purpose of this manual is to provide a study resource for the Nagios Certified Professional exam. This manual has
been written to aid those taking the exam, but it is also a resource for those who are professionals that will use Nagios
on a daily basis, for example those working on a Helpdesk. The questions that are presented in the exam are framed
in context in this manual. In order to facilitate learning at a deeper level, exercises are included to help students work
through the practical solutions that the exam represents.

Intended Audience
The information contained in this manual is intended for those who will be pursuing the Nagios Certified Professional
Certification from Nagios and for professionals working with Nagios on a daily basis. Those taking the exam will find
the solutions to the questions on the test within the manual placed in context to help aid the learning process. Often
the solutions will be illustrated with screenshots to make it more practical. Those who work at a Helpdesk or those
who are in management and need to view the activities on the network and create reports about the network will find
this manual helpful as well.

Preparation for Exercises

There are several step­by­step exercises included in the manual which will illustrate these aspects that a professional
using Nagios needs to understand:

* How to handle outages as they occur on the network.

* How to investigate incidents that occur on the network.
* How to acknowledge alerts in order to prevent additional notifications and communicate that information to others.
* How to schedule downtime when hosts or applications need to be intentionally shut down for maintenance.
* How to generate reports about the network that can be shared.
* How to predict problems before they occur by analyzing information provided by Nagios.

Generally the exercises can be performed on any network and illustrate skills that all networks using Nagios will
employ.
 Copyright by Nagios Enterprises, LLC
Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 1: Introduction
Nagios is the industry standard for Open Source network monitoring that provides the ability for an organization to
identify and resolve infrastructure problems. Nagios encompasses many features that allow it to accomplish this task.
Here is a summary of features:

Flexibility
Flexibility in an ever changing environment is a requirement to modern network monitoring. Nagios has been
designed to be able to meet these flexibility requirements by providing the tools to monitor just about anything that is
connected to a network. In addition, Nagios allows the administrator to monitor both the internal metrics like CPU,
users, disk space, etc. and the application processes on those devices. The flexibility of Nagios Core allows you to use
it to perform and schedule checks, perform event handling and alert administrators as needed.

Extensibility
Nagios is designed to be able to use both plugins and addons designed by Nagios as well as be able to implement
plugins and addons created by third­party organizations. Nagios is able to integrate with almost any script languages
that an organization may be using including; shell scripts, Perl, ruby, etc.

Scalability
As companies grow more equipment will need to be monitored and greater diversity of equipment will be
implemented. Nagios is designed to be able to scale with companies as they grow and have changing needs.

Open Source code

Nagios Core is an Open Source Software licensed under the GNU GPL V2.

Customizable
Customization not only includes what devices to monitor, how those devices and applications within the devices will
be monitored, but also includes the protocol, plugin, addon, etc, that is incorporated into Nagios to allow that
monitoring to occur.

Nagios Monitoring Solutions

Nagios Core is the foundational application that provides the monitoring and alerting options that Nagios is known
for. Administration of the Nagios interface is mainly achieved through the CLI or Command Line Interface. The
Nagios web interface which uses CGI as the backend by default can be modified to use a MySQL database. The
frontend or web interface, can be modified with custom options to provide the look and feel that an organization
needs. Several examples of frontends would be themes that are available (i.e. Exfoliation, Vautour and Arana), Web
Interfaces like VShell, Nagiosdigger, MNTOS, Check_MK and Mobile Interfaces like Nagios Mobile, NagMobile and
iNag. Vshell is the official PHP interface for Nagios Core. Nagios Core by design features and supports many
different addons that can be used with it.

Nagios XI takes the Nagios Core and builds upon it to create an enterprise­class monitoring and alerting solution that
is easier to set up and configure using a PHP frontend. Nagios XI using easy to use network wizards provides
infrastructure monitoring of all of an organizations critical hardware, applications, network devices and network
metrics. The dashboard feature allows you to view the entire infrastructure visually as you monitor all of these

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 1: Introduction 2

services and devices. You also have the alerting options which communicate to administrators when services and
hosts have problems. The trending and hardware capacity limits help you create proactive decisions about the network
and devices on the network. The graphical interface is easy to customize to fit the organization needs and by
monitoring the graphs will help you predict network, hardware and application problems.

Nagios Fusion provides a GUI for central management of a network infrastructure spread over a large geographical
area. With central management Nagios Fusion allows the organization to review the organization's entire structure in
one location through one interface and yet allow each location to manage their infrastructure independently. Tactical
overview screens provide a snapshot of the monitored devices globally.

Nagios Fusion is distributed monitoring the easy way. It provides scalability and comprehensive server support
worldwide and in a central location. Fusion also provides the opportunity to create a failover situation with multiple
Fusion servers.

Technical Support
The official support site for Nagios can be found at http://support.nagios.com/forum. This site provides both free
support open to anyone and also customer support for those who have purchase a support contract. The user can ask
questions of the technical staff at Nagios and receive answers usually within the same business day.

Official Training
Nagios provides Official Nagios Training for both Nagios Core and Nagios XI. The training options can be found at
http://nagios.com/services/training Training services include Live Training performed over the Internet or on­site as
well as self­paced training for those wanting to work on their own as they have available time. The Official Nagios
training provides users with comprehensive manuals with step­by­step instructions and videos which students can
view in order to understand how to implement Nagios in a variety of ways.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 1: Introduction 3

Nagios Terminology
Nagios terminology can be a challenge, especially for those who will monitor the Nagios interface but who will not
typically install and configure Nagios. This section will try to add clarity to some of the more important terms.
If you have problems understanding terms or would like additional information, there is a”Documentation” link
under “General” in the menu which may provide answers to questions.

Plugins
Nagios uses plugins which are external programs that can consist of either a script (Perl, shell script, ruby,etc.) or a
compiled executable. These plugins are used to check services and hosts on the network. Plugins provide
communication between the Nagios core logic process and the hosts and services required to monitor. Each plugin
must be configured specifically for the host or service which will be evaluated. Plugins are created separated from the
Nagios process so they will need to be downloaded and installed separately. The Official Nagios Plugins are a group
of plugins designed, tested and compiled specifically for Nagios. You can download from these locations.

Nagios Plugins
Official Nagios Plugins http://nagiosplugins.org/
Nagios Plugin Downloads http://www.nagios.org/download/
NagiosExchange http://exchange.nagios.org/

Here is an illustration of the Nagios server communicating with the remote client through a plugin.

Plugins can be used to connect to the remote server using various ports and protocols in the communication process.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 1: Introduction 4

Here are several examples of how Nagios can connect to a client using plugins.

Public Service Checks

There are a number of protocols that exist which allow the Nagios server to test them externally. For example the
common port 80 is available on any web server.

FTP ­ port 21
SSH ­ port 22
SMTP ­ port 25
WEB ­ port 80
POP3 ­ port 110
IMAP ­ port 143
Secure Web ­ port 443

These public services allow Nagios to not only check to see if the port is open but to verify the correct application is
running on the specific port. This can be done because each of these public services run specific protocols which
provide the information needed to monitor them correctly and to differentiate them from other services on the same
server.

Checks Using SSH

Nagios can connect to a client server using SSH and then execute a local plugin to check internal functions of the
server like CPU load, memory, processes, etc. The advantage of using SSH is that checks are secure in the connection
and the transfer of information. The disadvantage of SSH is the complexity of setting up keys and the configuration
required on the host including editing visudo for some checks.

Nagios Remote Plugin Executor

NRPE, Nagios Remote Plugin Executor, executes plugins internally on the client and then returns that information to
the Nagios server. The Nagios server connects on port 5666 in order to execute the internal check. NRPE is protected
by the xinetd daemon on the client so that an administrator can restrict the connections to the NRPE plugins. The
advantage is that it is the easiest agent to set up.

Monitoring with SNMP

SNMP, Simple Network Management Protocol, is used extensively in network devices, server hardware and software.
SNMP is able to monitor just about anything that connects to a network, that is the advantage. The disadvantage is
that it is not easy to work with. The complexity of SNMP is made even worse by the fact that vendors write
propitiatory tools to monitor SNMP that are not easily accessed using Nagios. SNMP can be monitored directly using
Nagios plugins or the device itself can monitor SNMP and send information to SNMP traps which can be located on
the Nagios server. The difficulties are further aggravated when using traps as the SNMP trap information must be
translated into data that Nagios can understand.

Nagios Service Check Acceptor

NSCA, Nagios Service Check Acceptor, employs a daemon on the Nagios server which waits for information
generated by passive checks which execute independently on the client being monitored by Nagios. The advantage of
NCSA is that services are monitored locally independent of the Nagios server and then sent to the Nagios server so
this is a good option when a firewall between the Nagios server and the client prevent other types of communication.
The disadvantage is that passive checks use plugins but often require scripts to execute on the client.

Communication can be encrypted between the client and the Nagios server and a password will be required to

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 1: Introduction 5

complete communication.

Another use for NSCA is distributed monitoring. Distributed monitoring allows a wide geographical base of network
devices to be monitored by multiple Nagios servers which use NSCA to send service checks and host checks to a
central Nagios server.

Nagios Remote Data Processor

NRDP is another way of monitoring using passive checks. The advantage of using NRDP is that it uses less resources
and it connects on the common port 80 or 443 on the Nagios server.

NSClient ++
This agent is installed on Windows servers and desktops in order to monitor with either check_nt (port 12489), NRPE
(port 5666) or using passive checks. This is the most reliable Windows agent available and has the advantage of
multiple options for monitoring.

Currently the plugins provided in the nagios­plugins package provides about 80 plugins and another 80 in the contrib
directory. This certainly provides you with adequate plugins to get started. By searching Google, SourceForge and
GitHub you will be able to find additional plugins.

If you need to find out more information about a specific plugin you can use this command after moving into the
plugins directory:

./<plugin_name> ­­help

The “help” feature provides the versin, structure and options that the plugins uses. Often examples of how to use the
plugin are included as well.

Here is an example of the common ping plugin.

./check_ping ­­help
check_ping v1.4.15 (nagios­plugins 1.4.15)
Copyright (c) 1999 Ethan Galstad <nagios@nagios.org>
Copyright (c) 2000­2007 Nagios Plugin Development Team
<nagiosplug­devel@lists.sourceforge.net>

Use ping to check connection statistics for a remote host.

Usage:
check_ping ­H <host_address> ­w <wrta>,<wpl>% ­c <crta>,<cpl>%
[­p packets] [­t timeout] [­4|­6]

Options:
­h, ­­help
Print detailed help screen
­V, ­­version
Print version information
­4, ­­use­ipv4
Use IPv4 connection
­6, ­­use­ipv6
Use IPv6 connection

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 1: Introduction 6

­H, ­­hostname=HOST
host to ping
­w, ­­warning=THRESHOLD
warning threshold pair
­c, ­­critical=THRESHOLD
critical threshold pair
­p, ­­packets=INTEGER
number of ICMP ECHO packets to send (Default: 5)
­L, ­­link
show HTML in the plugin output (obsoleted by urlize)
­t, ­­timeout=INTEGER
Seconds before connection times out (default: 10)

Host
A host is a server, switch, router, printer or any other network device that you want to monitor. Nagios requires an IP
Address for the host or a FQDN (Fully Qualified Domain Name) to determine the exact location of the device. Each
host must also have a unique name that will tie the host name reference to the IP Address of FQDN. The host
information is required to be able to configure a check for the host or service.

Service
A service is any metric that may be required to evaluate on the host, including internal metrics like CPU, memory,
users, disk space, etc. A service can also monitor daemons and applications running on the server like a database,
MySQL or Postfix for example.

Users
Users is a reference to an individual that has been given access to the Nagios web interface in order to view hosts and
services and in order to manages those hosts and services. Note: users and contacts are different as users access the
web interface and contacts receive notifications. However, a user can be set up to also be a contact.

Contacts
Contacts are the individual administrators that are notified by Nagios because of a host or service problem. These
contacts are typically a part of a contactgroup. The contact information provides a way to communicate to the
administrator.

Contactgroups
These groups are the connection between detected problems and communication with individuals in the group.
Contactgroups usually are related to the type of device. For example, organizations may group windows
administrators into a separate group from Linux administrators as often the skills required to solve problems is quite
different. Contactgroups provide an excellent way to manage notification in a rapidly changing environment.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 1: Introduction 7

Acknowledgment
Acknowledgment will temporarily suppress alert notifications until the host or service returns to an OK state. This
can be achieved inside the web interface by selecting the service and then choosing “Acknowledge this service
problem” (see image).

Once that option is selected the administrator can enter the reason for the problem and that it is currently the issue.
Enter the host name, the service, your name and the comment you want to communicate to other administrators (these
are all required). You also can select if the comment will be sticky ,persistent or if you want to to send notification.
The “Sticky Acknowledgement”, when it is checked, will prevent further notifications if the problem continues.
“Persistent Comment” in Nagios 3 will retain the comment even after a reboot and must be manually unchecked when
it is fixed. If you leave it unchecked Nagios will remove the comment when a solution is found.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 1: Introduction 8

Other administrators may now see that the problem is being worked on by viewing System and Comments in the
menu.

If the host or service was disabled permanently then a better solution is to disable checks permanently.

Downtime
If you are going to work on a server or device and need to schedule downtime so Nagios does not notify administrators
that can be performed at the web interface. When you select the host or service that will be down you have an option
to schedule downtime. When downtime is scheduled Nagios will place a comment in the web interface in order to
communicate the fact to all administrators who access the web interface.

Disabled
“Disabled” is a term which refers to turning off a feature that Nagios provides. For example, active checks, passive
checks, obsessing, notifications, event handlers of flap detection. The example shows the option on the right menu
was selected to “Disable flap detection for this service”.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 1: Introduction 9

Latency
Latency is the difference between when a check is scheduled to run and when it does actually run. Latency is often
used as a metric for Nagios performance. The greater the time difference between when a check was scheduled to run
and when it actually runs means a greater degradation in performance. Latency can be observed by choosing the
menu on the web interface and selecting “Performance Info” under System. The latency for service and host checks is
listed. See Performance Info under the Web Interface for how to view this information.

State
Nagios has a built in protection mechanism against false positives called state. State is measured by two values SOFT
and HARD. When a failure is detected for a host or service which was originally working, the initial state is SOFT,
which does not create a notification. Nagios checks several times before a state is determined to be HARD, which
does create a notification. The max_check_attempts setting is used to determine how many times Nagios rechecks a
SOFT state before it is moved to a HARD state. So if the max_check_attempts setting is 5, Nagios will check 5 times
and remain in a SOFT state, until another check is made to create a HARD state. Over this time, the SOFT state must
remain in a non­OK state for it to be moved to a HARD state. It is the HARD state which triggers notification.

In the example, the HARD state is illustrated in this service check, which shows the max_check_attempts setting is 3
and the first of 3 checks has occurred.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 1: Introduction 10

Host and Service States

Nagios uses four different return values from plugins in order to determine the state of a host or service. These four
values determine the color of the output in the web interface.

0 OK green
1 WARNING yellow
2 CRITICAL red
3 UNKNOWN orange

The color changes are what is reflected in the web interface.

Agents
Agents refer to the programs, usually daemons, that must be placed on the client to listen for connections coming
from the Nagios server.

Typical Agents
SSH ­ port 22
NRPE ­ port 5666
NSClient++ ­ ports 5666 or 12489

Agents provide greater access to the clients internal metrics as plugins can be run on the client by Nagios using the
agent that is installed on the client.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 1: Introduction 11

Unhandled
Unhandled host or services are those in non­OK states which have not been acknowledged, are not in a scheduled
downtime and if they are services they are associated with a host that is not in a problem state.

Installation
Understanding installation options is an important part of troubleshooting as the installation method determines the
location of binaries, configuration files and plugins. These locations may even differ based on the version of the
Linux distribution. The following chart provides common locations using the examples of compiling, using a CentOS
RPM or using a Debian/Ubuntu Deb file. The point is, know how Nagios was installed before starting the
troubleshooting process.

NAGIOS Program Location Configuration File Plugins

Compile /usr/local/nagios/bin/nagios /usr/local/nagios/etc/nagios.cfg /usr/local/nagios/libexec
CentOS /usr/bin/nagios /etc/nagios/nagios.cfg /usr/lib/nagios/plugins
Debian/Ubuntu /usr/bin/nagios3 /etc/nagios3/nagios.cfg /usr/lib/nagios/plugins
Web Server Program Location Web Server Configuration Nagios Web Config
CentOS /usr/sbin/httpd /etc/httpd/conf/httpd.conf /etc/httpd/conf.d/nagios.cfg
Debian/Ubuntu /usr/sbin/apache2 /etc/apache2/apache2.conf /etc/nagios3/apache2.conf
Users htpasswd Database
Compile /usr/local/nagios/etc
CentOS /etc/nagios
Debian/Ubuntu /etc/nagios3/

The implications for documentation are that you must translate any documentation to the installation method that was
chosen.
Installation from source is a process where the source code that was developed by the programmer is converted into a
binary format that the server can run. Compiling Nagios may require a few extra steps in setting up Nagios but there
are several advantages over using a RPM repository or a DEB repository. The biggest advantage of installing from
source is that the installation process can be repeated on almost any Linux distribution and therefore each distribution
will have the same location for binaries, configuration files and plugins.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 1: Introduction 12

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 2: Configuration 13

Chapter 2: Configuration

Nagios configuration can be a complex process over a long period of time. Gaining a basic outline of that process can
be useful in troubleshooting and determining the intricacies of how Nagios functions.

Initial Set Up
Whether you are installing using a repository or from source, there are some initial steps to take to get started. The
first step is to add a contact email for the nagiosadmin. The user nagiosadmin by default is the only user able to access
the whole web interface.

Contact Information
In order to receive notification of a problem, the nagiosadmin user must have a valid email address configured. Edit
/usr/local/nagios/etc/objects/contacts.cfg (RPM repository /etc/nagios/objects/contacts.cfg).
Place nagiosadmin user email in the email location.

define contact{
contact_name nagiosadmin ; Short name of user
use generic­contact ; Inherit default values
alias Nagios Admin ; Full name of user
email your_email ; <<***** CHANGE THIS TO YOUR EMAIL
}

Restart Nagios to make the changes take effect.

Pre-Flight Check
The pre­flight check is a procedure that checks the configuration of Nagios and returns any errors, before Nagios is
started. The becomes a necessary process before restarting Nagios in order to maintain the integrity of the system.
Nagios will restart when it encounters Warnings but will not restart if it encounters Errors. In order to use the pre­
flight check execute the Nagios binary and point it to the location of the nagios.cfg file, using the verbose option “­v”.

nagios ­v /usr/local/nagios/etc/nagios.cfg
(RPM repository /etc/nagios/nagios.cfg)

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 2: Configuration 14

Nagios 3.3.1
Copyright (c) 1999­2008 Ethan Galstad (http://www.nagios.org)
Last Modified: 12­01­2008
License: GPL

Reading configuration data...

Running pre­flight check on configuration data...

Checking services...
Checked 8 services.
Checking hosts...
Checked 1 hosts.
Checking host groups...
Checked 1 host groups.
Checking service groups...
Checked 0 service groups.
Checking contacts...
Checked 1 contacts.
Checking contact groups...
Checked 1 contact groups.
Checking service escalations...
Checked 0 service escalations.
Checking service dependencies...
Checked 0 service dependencies.
Checking host escalations...
Checked 0 host escalations.
Checking host dependencies...
Checked 0 host dependencies.
Checking commands...
Checked 24 commands.
Checking time periods...
Checked 5 time periods.
Checking for circular paths between hosts...
Checking for circular host and service dependencies...
Checking global event handlers...
Checking obsessive compulsive processor commands...
Checking misc settings...

Total Warnings: 0
Total Errors: 0

Things look okay - No serious problems were detected during the pre-flight check

The output of the pre­flight check, as in this example, clearly indicates everything is OK and the changes that were
made can be applied to Nagios safely.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 2: Configuration 15

Creating a Password
The nagiosadmin user is created by default and will allow access to the web interface using that pre­created username.
However, the password for that user should be created. Execute the following command in order to create a new
user/password combination.

htpasswd ­c /usr/local/nagios/etc/htpasswd.users nagiosadmin

New password:
Re­type new password:
Adding password for user nagiosadmin

Now login to the web interface with http://ip_address/nagios.

Note: The “­c” option in the command above creates a new file, erasing an old file if it exists. Therefore, if the
htpassword.users file has been created, use the command without the “­c” to create additional users/passwords.

Eliminating the HTTP Error

When you set up the Nagios server and either review your log files in /var/log/nagios/nagios.log or review the web
interface you may initially see an error related to the web server. The error is related to the fact that you do not have a
an index.html file that exists. Note: If you do not see the error it is because you have the necessary files so you
can skip this step. Here is what it will look like in the log.

WARNING: HTTP/1.1 403 Forbidden ­ 5240 bytes in 0.001 second response time
Sep 26 10:00:18 nagios nagios: SERVICE ALERT: localhost;HTTP;WARNING;HARD;4;HTTP

You can easily eliminate the error by creating an index.html file. Create a simple HTML.

vi /var/www/html/index.html

<HTML>
<BODY>
Nagios Server
</BODY>
</HTML>

chmod 755 /var/www/html/index.html

chown apache:apache /var/www/html/index.html

The example is for a CentOS system. If you are using Debian or Ubuntu for example the name of the web server and
the location of the web directory are different.

Nagios Check Triangle

One of the major concepts of creating checks is to remember that all plugins with Nagios will require three elements
to be configured. There must be a host definition, a service definition and a command definition. Think of it as a
triangle each time you want to use a plugin.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 2: Configuration 16

These three definitions are all located in three separate files, hosts.cfg, services.cfg and commands.cfg. The name of
the file can be different. You may need to create hosts.cfg and services.cfg as they are not created by default. These
files are typically located in this directory if you compiled:

/usr/local/nagios/etc/objects

Host Defintion
Nagios needs to know an IP Address of the host you want to check. This is configured in the hosts.cfg file. The
hosts.cfg file does not exist initially so you will need to create it. In this example the host_name is “win2008” and it is
tied to the address “192.168.3.114”. This is the information Nagios must have to know where to point a request and
how to record information for a specific host.

define host{
use windows­server
host_name win2008
alias Windows Server
address 192.168.3.114
}

Service Definition
The second part of the triangle is the service definition. Nagios needs to know what service you want to check, so that
service or plugin must be defined. In this example the host “win2008”, which Nagios knows now is tied to the IP
Address 192.168.3.114, is being checked with the ping plugin. So you can see the host_name determines which host
the plugin acts upon and then the service_description is really the text that shows up in the web interface. The
check_command, defines the parameters of the plugin. Here you can see that “check_ping” is the plugin and it is
followed by two different sections of options divided by “!”. The first section, “60.0,5%”, provides a warning level if

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 2: Configuration 17

packets are take longer than 60 milliseconds or if there is greater than a 5% loss of packets when the ping command is
performed. The second section is the critical level where a CRITICAL state will b e created if packets take longer
than 100 milliseconds or if there is more than 10% packet loss.

define service{
use generic­service
host_name win2008
service_description Ping
check_command check_ping!60.0,5%!100.0,10%
}

Command Definition
The command definitions are located in the commands.cfg file which is created by default in the objects directory.
Many commands are already defined so you do not have to do anything with those. The check_ping command is one
example that has been defined. The command_name, “check_ping”, is what is part of the service definition. The
command_line specifically defines where the plugin is located with the “$USER1$ macro. This is equal to saying that
the plugin check_ping is located in /usr/local/nagios/libexec (if you compiled). The other 4 options include the host,
using the $HOSTADDRESS$ macro, a warning level (­w) using the $ARG1$ macro, the critical level (­c) using the
$ARG2$ macro and the number of pings to use by default (­p 5).

# 'check_ping' command definition

define command{
command_name check_ping
command_line $USER1$/check_ping ­H $HOSTADDRESS$ ­w $ARG1$ ­c $ARG2$ ­p 5
}

In each of the elements of the Nagios triangle you can see the importance of the term “definition” as each element
must be clearly defined and each element is dependent upon the other definitions.

Nagios Checks
Nagios can perform checks two different ways; active or passive. Understanding which method is being used is key to
troubleshooting. When comparing active and passive checks one of the biggest differences is that in active checks
Nagios explicitly controls each step but in passive checks Nagios is at the mercy of the external host sending data to be
processed. In passive checks the client performs the check itself and provides the information to the Nagios server at
the interval determined by the client, not Nagios.

Active
With active checks Nagios initiates and manages each step of the process. This means each step of the process is
closely monitored and manipulated by Nagios. The schedule for when checks occur, the organization of resources and
the initialization of those resources are controlled by Nagios. The scheduling queue is an example. During time of
heavy load Nagios may push this schedule to control activity.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 2: Configuration 18

Passive
Typically passive checks are used when a firewall prevents the Nagios server to make a request to the client or when
the client is running an application that asynchronous, in other words the time schedule for a service is erratic and
cannot be fully determined. Security events is one example of a situation where you do not know when the event may
occur. Passive checks may also be used for distributed monitoring where you have multiple Nagios servers providing
information to a master Nagios server.

When Passive Checks are used the client uses a program called NSCA (Nagios Service Check Acceptor) and the
evaluation occurs locally on the client and then is sent to the Nagios server using NSCA. NSCA runs on the Nagios
server as a daemon protected by xinetd. The daemon will listen for requests on port 5667 sent by the client. When the
server receives the request the remote server is authenticated using a password that is shared between the Nagios
server and the client. The password is encrypted on one of 22 levels to protect it as it moves over the network.
A similar passive check can be used with NRDP (Nagios Remote Data Processor) which communicates to the Nagios
server using a secret token and connects on port 80 or 443.

The Nagios server only processes passive checks that are sent to it. In other words, the client must automate the
checks using a cron job or the passive checks must be a response to an event.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 2: Configuration 19

Security Risks
Nagios can pose security risks to organizations which do not configure Nagios properly. Because the Nagios server is
able to execute commands on the hosts that it monitors, special care should go into protecting the Nagios server. Here
are a few of the items that need to be considered:

* use a firewall on the Nagios server to limit access to administrators and client machines that will be sending passive
check data
* encrypt communication to protect data that will be over a public network
* restrict user privileges to only what is required to administer the Nagios server
* use a box dedicated to Nagios
* monitor changes on the Nagios server
* tighten security on the clients that Nagios will monitor

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 2: Configuration 20

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 3: Updates 21

Chapter 3: Updates

Keeping your Nagios installation up to date is an important part of administration. However, this should only be
performed when you have an established backup and restore process just as in any situation.

Checking for Updates

The web interface for Core allows you to easily check for updates by going to the home page.

This page also contains links for training, certification, tutorials, labs, plugins and provides the latest Nagios news. By
selecting the “Check for updates” the link will assess the current version to see if it is up to date.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 3: Updates 22

If your version of Nagios Core is out of date it is important that the first thing you do is backup the current version
before proceeding to the update process.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 23

Chapter 4: User Management

Users and contacts can be separate functions within Nagios. Users are individual accounts that have access to the web
interface. Contacts are users who will be sent notification if there are problems with hosts or services.

Authentication and Privileges

The authentication parameters in the cgi.cfg is a way to configure access so that the contacts that log in must match
the hosts and services which they are responsible for. This eliminates them being able to access other hosts.

In order to provide access for a user to be able to see all computers and services on the Web Interface you will need to
activate these two parameters on the /usr/local/nagios/etc/cgi.cfg (RPM repository /etc/nagios/cgi.cfg).

authorized_for_all_services=fred
authorized_for_all_hosts=fred

If you want to allow a user (like fred) to run any commands on the web interface even if they are not listed with
permissions that match a service or host you will need to modify these two parameters.

authorized_for_all_service_commands=nagiosadmin,fred
authorized_for_all_host_commands=nagiosadmin,fred

If you wanted to set up configuration so that all users who authenticate to the web interface can do everything they
choose, not recommended, then you would place a “*” at the end of each line for all users.

authorized_for_all_services=*
authorized_for_all_hosts=*
authorized_for_all_service_commands=*
authorized_for_all_host_commands=*

Authentication
Authentication is the process that allows users to access the web interface. Authentication is controlled by the use of a
database using the htpasswd command. The database, called htpasswd.users, is located in the /usr/local/nagios/etc
directory (/etc/nagios if using the RPM repository). The name and location of the database is determined by the
configuration options found in /etc/httpd/conf.d/nagios.conf. In this example, from a CentOS install, you can see that
several directories require authentication from this database.

ScriptAlias /nagios/cgi­bin "/usr/local/nagios/sbin"

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 24

<Directory "/usr/local/nagios/sbin">
# SSLRequireSSL
Options ExecCGI
AllowOverride None
Order allow,deny
Allow from all
# Order deny,allow
# Deny from all
# Allow from 127.0.0.1
AuthName "Nagios Access"
AuthType Basic
AuthUserFile /usr/local/nagios/etc/htpasswd.users
Require valid­user
</Directory>

Alias /nagios "/usr/local/nagios/share"

<Directory "/usr/local/nagios/share">
# SSLRequireSSL
Options None
AllowOverride None
Order allow,deny
Allow from all
# Order deny,allow
# Deny from all
# Allow from 127.0.0.1
AuthName "Nagios Access"
AuthType Basic
AuthUserFile /usr/local/nagios/etc/htpasswd.users
Require valid­user
</Directory>

Access is maintained through the database but the permissions a user has once they authenticate are determined by
contacts, contact groups and cgi permissions determined from the cgi.cfg file. An important point to remember when
setting up permissions is that the contact is only able to see the host or service that they are responsible for by default.
Make sure contact names match the user created for access to the web interface.

These settings represent the default settings in the cgi.cfg file for permissions to the web interface. The user
“nagiosadmin” is the default nagios user with access and unlimited permissions to the web interface. The defaults
demonstrate why it is so important to correctly set up the nagiosadmin user as part of the initial configuration.

use_authentication=1
use_ssl_authentication=0
#default_user_name=guest
authorized_for_system_information=nagiosadmin
authorized_for_configuration_information=nagiosadmin
authorized_for_system_commands=nagiosadmin
authorized_for_all_services=nagiosadmin
authorized_for_all_hosts=nagiosadmin
authorized_for_all_service_commands=nagiosadmin

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 25

authorized_for_all_host_commands=nagiosadmin
#authorized_for_read_only=user1,user2

Scenario: Turn Off All Authentication

Turning off all authentication is not recommended under any circumstances. It is only demonstrated here in order to
aid in the understanding of how Nagios authentication works. These changes allow anyone to make changes to the
Nagios interface, hosts and services.

Security Tip
Warning, this is a serious security issue and should not be implemented.

There are two steps required to turn off all security. Edit the cgi.cfg file located in /usr/local/nagios/etc (/etc/nagios if
using the RPM repository) and change the “use_authentication” to a “0”.

use_authentication=0

The second step required is to access the /etc/httpd/conf.d/nagios.conf file and comment out the lines that require
authentication for the Nagios directories.

ScriptAlias /nagios/cgi­bin "/usr/local/nagios/sbin"

<Directory "/usr/local/nagios/sbin">
# SSLRequireSSL
Options ExecCGI
AllowOverride None
Order allow,deny
Allow from all
# Order deny,allow
# Deny from all
# Allow from 127.0.0.1
# AuthName "Nagios Access"
# AuthType Basic
# AuthUserFile /usr/local/nagios/etc/htpasswd.users
# Require valid­user
</Directory>

Alias /nagios "/usr/local/nagios/share"

<Directory "/usr/local/nagios/share">
# SSLRequireSSL
Options None
AllowOverride None
Order allow,deny
Allow from all
# Order deny,allow
# Deny from all
# Allow from 127.0.0.1
# AuthName "Nagios Access"

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 26

# AuthType Basic
# AuthUserFile /usr/local/nagios/etc/htpasswd.users
# Require valid­user
</Directory>

Restart Nagios and the web server.

Scenario: Create a View Only Account

This scenario will create a user that can view all hosts and services but not be allowed to make any changes to those
hosts or services. This is typically the settings you may choose for management to review the status of hosts and
services.

Create the user in the htpasswd.users database.

htpasswd htpasswd.users management

New password:
Re­type new password:

Make modifications to the cgi.cfg file by adding the user separated by a comma, without spaces. The user has global
access, which means they are not required to be listed as contacts for hosts and services. The user is also added to the
read only list.

authorized_for_all_services=nagiosadmin,management
authorized_for_all_hosts=nagiosadmin,management
authorized_for_read_only=management

Restart Nagios and the web server.

Scenario: Create System Administrator with No Contact Information

In this scenario the settings will allow a user to have full access to all settings on all hosts and services just like the
nagiosadmin user. However, this user is not associated with any contact information so will not be notified at any
time. This account is strictly administration only.

htpasswd htpasswd.users john

New password:
Re­type new password:

Edit the cgi.cfg file and add john to each of the lists indicated below.

authorized_for_system_information=nagiosadmin,john
authorized_for_configuration_information=nagiosadm,john
authorized_for_system_commands=nagiosadmin,john
authorized_for_all_services=nagiosadmin,john
authorized_for_all_hosts=nagiosadmin,john
authorized_for_all_service_commands=nagiosadmin,john
authorized_for_all_host_commands=nagiosadmin,john

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 27

Restart Nagios and the web server.

Scenario: Create an Administrator with Limited Access

This user will only be allowed to access the hosts and services that they are associated with via contact information.
This may be the type of settings used when an organization has divided responsibilities for routers, Windows servers
and Linux servers for example.

htpasswd htpasswd.users sue

New password:
Re­type new password:

Create a new contact entry in contacts.cfg and specify the contact_name, alias and email contact information for the
user.

define contact{
contact_name sue
use generic­contact
alias Router Admin
email sue@example.com
}

Add the user to a group or create a new group in the contacts.cfg file. This example shows a user added to a new
contact group called router­admins. By creating a new group it enables an administrator to assign that group to a
series of devices, like routers.

define contactgroup{
contactgroup_name router­admins
alias Router Administrators
members sue
}

At this point you will need to edit the hosts and services and add the “contact_groups router­admins” which will
override the default settings in the template. This will enable only those users in this contact group access to these
hosts and services unless they have global access from the cgi.cfg file.

define host{
use generic­switch
host_name cisco
alias cisco router
address 192.168.5.220
contact_groups router­admins
}
define service{
use generic­service
host_name cisco
service_description PING
check_command check_ping!200.0,20%!600.0,60%
normal_check_interval 5
retry_check_interval 1

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 28

contact_groups router­admins
}

Restart Nagios and the web server.

Notification
Nagios provides the ability to notify administrators when problems develop. The key to working with Nagios on
notifications is to avoid the false alarms that can be very frustrating to those who have to come in and fix the reported
problems.

Notification is triggered when the max_check_attempts parameter has been reached and a HARD state has occurred.
In other words, it is confirmed that the machine has moved from a functioning state to a broken state. So here is also
a key to preventing false alarms, move the max_check_attempts for a service or host to a higher number so that it must
check a number of times before it notifies administrators. Here is an example that requires 10 attempts.

max_check_attempts 10

Until the max_check_attempts has been reached, Nagios still considers this a SOFT state. The other important point
here is that these 10 attempts must all return a CRITICAL status consecutively before a HARD state is reached. In
other words, if you change to 10 as the max_check_attempts, the hard state is reached when it returns 10 CRITICAL
states in a row.

The notification process flows through a number of filtering options that you can provide for a fine tuned set up.

System Wide Filtering

Notifications can be turned on or off by editing the nagios.cfg file, “1” indicating that it is on and “0” indicating it
should be off system wide. The default is to have it on.

enable_notifications=1

This setting will automatically take into account downtime which is scheduled in the web interface.

Service / Host Filtering

Notifications are sent for host objects for d(down), u(unreachable), r(recovered), f(flapping­up then down state) and
now with Nagios 3 s(start of planned maintenance). Notifications are sent for service objects for c(critical),
w(warning), u(unknown), r(recovered), f(flapping) and s(start planned maintenance). These options can be entered
into the options line to select those options that you want notifications for.

notification_options=c,r

If you set notifications to n(none) or "0" it will not send notifications. Each host and service references a template,
“use generic­switch”. The specific settings you enter into a host or service definition will override the template
settings. In this example, this host will not send notifications regardless of the settings which are in the template and
the global settings as well.

define host{

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 29

use generic­switch
host_name zyxel
alias zyzel router
notifications_enabled 0
address 192.168.5.79
}

Service Groups
You will need to create a file called servicegroups.cfg and put an entry in nagios.cfg to indicate where it is. Note the
entries are in pairs (first host, then service) “web,HTTP”.

# SERVICE GROUPS
define servicegroup{
servicegroup_name webservice
alias Web­Sites
members web,HTTP ,web2,HTTP
}

Host Groups
Create an entry in nagios.cfg to the location of hostgroups.cfg.

define hostgroup{

hostgroup_name linux­web
alias Linux­Sites
hostgroup_members web
}

The notification_period parameter allows you to set when these notifications should occur. Here are several examples
of timeperiods. Note that time parameters must be defined in the timeperiods.cfg so that these can be used.

notification_period 24x7h

notification_period workhours

notification_period shift2

notification_period shift3

The notification_interval is an parameter that you can adjust so that you will be able to determine how often you will
be notified of a situation. The default setting is in minutes. This example will notify the contacts every 60 minutes.

notification_interval 60

If you only wanted one notification sent you can set this parameter to “0”.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 30

define host{
name linux­box
use generic­host
check_period 24x7
check_interval 5
retry_interval 1
max_check_attempts 10
check_command check­host­alive
notification_period 24x7
notification_interval 0
contact_groups admins
register 0
}

Contact Filtering
Nagios will allow you to determine who gets contacted for each situation. It also allows you to create groups so that
different administrators can be contacted for different reasons and at different times.

Contact Admins
define contact{
contact_name nagiosadmin
use generic­contact
alias Nagios Admin
email jane@some_email.com
}
define contact{
contact_name joe
alias Win Admin
use generic­contact
service_notification_period workhours
host_notification_period workhours
service_notification_options w,u,c,r
host_notification_options d,r
service_notification_commands notify­service­by­email
host_notification_commands notify­host­by­email
email joe@some_email.com
}

Contact Groups
Notice that each group has a contact name and the members are different. An important aspect of contacting different
admins at different times is that you want to consistently run the service and host to continue sending messages so that
you do not miss contact with an administrator. So the result is that the service and host send messages 24x7 and on a
regular interval but the admins are divided by who accepts responsibility during different time periods and what
groups they are in.

define contactgroup{
contactgroup_name admins
alias Nagios Administrators

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 31

members nagiosadmin
}
define contactgroup{
contactgroup_name win­admins
alias Windows Administrators
members joe
}

Here is an example of the many notification options that you have. Notice that the choices are spread quite wide over
the whole spectrum of Nagios.

Multi_Level Notifications
Multi_level notifications allow an administrator to perform different levels of notification to different administrators
on the same host or service. For example, if there was a service that the administrator wanted to send WARNING
level messages to one admin but send CRITICAL level messages to a different administrator they could use this
multi_level notification process. The first step in understanding the configuration options is to understand the generic­

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 32

contact template. If a service notification was the focus, the options for notification are by default:

service_notification_options w,u,c,r,f,s

define contact{
name generic­contact
service_notification_period 24x7
host_notification_period 24x7
service_notification_options w,u,c,r,f,s
host_notification_options d,u,r,f,s
service_notification_commands notify­service­by­email
host_notification_commands notify­host­by­email
register 0
}

In order to use multi_level notification you will need to create two separate contacts with different levels of service
notification; w (warning) and c (critical). Here joe is created as a contact with the “w” only in the
service_notifications_options. Note that even though the generic­contact is used for joe by placing a specific
reference to service_notifications_options in this contact information it overrides the default template settings.

define contact {
use generic­contact
contact_name joe
service_notification_options w
service_notification_commands notify­service­by­email
email joe@localhost.localdomain
}

Bob is created as a contact with the “c” option.

define contact {
use generic­contact
contact_name bob
service_notification_options c
service_notification_commands notify­service­by­email
email bob@localhost.localdomain
}

Now in order to use these administrators they must be associated with a contactgroup, here a new contactgroup has
been created, “linux_admins”.

define contactgroup{
contactgroup_name linux_admins
alias admin group
members joe,bob
}

Once the contactgroup has been created, the group must be associated with the service that you want to have multi­
level notifications associated with. In this example, the check_local_users is used with a WARNING state at 2 and a
CRITICAL state at 3.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 33

define service{
use local­service
host_name localhost
service_description Current Users
check_command check_local_users!2!3
contact_groups linux_admins
}

Once that has been completed, restart Nagios and then create a situation where the WARNING level and the
CRITICAL levels can be attained. In the example you see the process of the soft states created and then the hard state
and at that point joe is sent an email for the WARNING state.

Jul 9 04:56:04 localhost nagios: SERVICE ALERT: localhost;Current

Users;WARNING;SOFT;1;USERS WARNING ­ 3 users currently logged in
Jul 9 04:56:14 localhost nagios: SERVICE ALERT: localhost;Current
Users;WARNING;SOFT;2;USERS WARNING ­ 3 users currently logged in
Jul 9 04:57:14 localhost nagios: SERVICE ALERT: localhost;Current
Users;WARNING;SOFT;3;USERS WARNING ­ 3 users currently logged in
Jul 9 04:57:14 localhost nagios: SERVICE ALERT: localhost;Current
Users;WARNING;HARD;4;USERS WARNING ­ 3 users currently logged in
Jul 9 04:57:14 localhost nagios: SERVICE NOTIFICATION: joe;localhost;Current
Users;WARNING;notify­service­by­email;USERS WARNING ­ 3 users currently logged in

Here is the email that only joe receives.

From nagios@localhost.localdomain Sat Jul 9 05:07:11 2011

Date: Sat, 9 Jul 2011 05:07:11 ­0600
From: nagios@localhost.localdomain
To: joe@localhost.localdomain
Subject: ** PROBLEM Service Alert: localhost/Current Users is WARNING **
***** Nagios *****
Notification Type: PROBLEM
Service: Current Users
Host: localhost
Address: 127.0.0.1
State: WARNING
Date/Time: Sat Jul 9 05:07:11 MDT 2011
Additional Info:
USERS WARNING ­ 3 users currently logged in

Now after creating the CRITICAL state bob is the only one to receive an email.

Jul 9 05:02:47 localhost nagios: SERVICE ALERT: localhost;Current

Users;CRITICAL;HARD;4;USERS CRITICAL ­ 4 users currently logged in
Jul 9 05:02:47 localhost nagios: SERVICE NOTIFICATION: bob;localhost;Current
Users;CRITICAL;notify­service­by­email;USERS CRITICAL ­ 4 users currently logged
in

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 34

From nagios@localhost.localdomain Sat Jul 9 05:15:34 2011

Date: Sat, 9 Jul 2011 05:15:34 ­0600
From: nagios@localhost.localdomain
To: bob@localhost.localdomain
Subject: ** PROBLEM Service Alert: localhost/Current Users is CRITICAL **
***** Nagios *****
Notification Type: PROBLEM
Service: Current Users
Host: localhost
Address: 127.0.0.1
State: CRITICAL
Date/Time: Sat Jul 9 05:15:34 MDT 2011
Additional Info:
USERS CRITICAL ­ 4 users currently logged in

Escalation
Escalation is a process in which if a solution is not produced for a host or service in a specified response time, the
problem is referred to the next level. This of course implies that an organization will have a number of levels for
administrators. It provides a way to focus resources on those who are capable of solving situations within a reasonable
amount of time. Reasonable, is determined by the significance of the host or service that is down.

In this example you can see the initial contact will be the Nagios administrators who will normally handle Nagios
problems. Once the 5 message is sent out the problem is escalated to the Level 2 Engineers who will become the
default notification. Once the 9th message is sent out it will be escalated to Level 3 Engineers who will become the
default.

It is important to recognize in the example that Nagios does not measure response in time but rather in the number of
messages that are sent out, which are measured on a time interval. The generic­service template for example will
notify administrators every 60 minutes.

notification_interval 60

Messages are sent to contact groups so those groups and the administrators that are a part of those groups must be
created.

Setting up the Contacts and Contact Groups

Create all of the users in the htpasswd.users file so they have access to the web interface. This will be important if one

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 35

of the administrators leaves a message about the information they found on the problem.

cat htpasswd.users
nagiosadmin:fTx/AJMMvBp22
fred:X8agiOot2dRzk
management:0e/oiKAJS0Erc
john:PwI3eSx5QDCp.
sue:YDqeTdQIqn9tE
jim:cW790LhQsU3v6
mark:MCCR2eMPWBx4Y
tom:WK8dJ8ksJeNtU
mary:y4ogaqxCr43OM
ralph:zp5jafw5H2.wA

Edit the cgi.cfg file to provide the correct rights so your admins can all fix the same things.

authorized_for_system_information=nagiosadmin,john,sue,mark,tom,mary,ralph
authorized_for_configuration_information=nagiosadmin,john,sue,mark,tom,mary,ralph
authorized_for_system_commands=nagiosadmin,john,sue,mark,tom,mary,ralph
authorized_for_all_services=nagiosadmin,management,john,sue,mark,tom,mary,ralph
authorized_for_all_hosts=nagiosadmin,management,john,sue,mark,tom,mary,ralph
authorized_for_all_service_commands=nagiosadmin,john,sue,mark,tom,mary,ralph
authorized_for_all_host_commands=nagiosadmin,john,sue,mark,tom,mary,ralph

Create your contacts and provide them with email addresses.

define contact{
contact_name nagiosadmin
use generic­contact
alias Nagios Admin
email nagios@localhost
}
define contact{
contact_name sue
use generic­contact
alias Linux Admin
email sue@localhost
}

Define the needed contact groups with the appropriate admins.

define contactgroup{
contactgroup_name admins
alias Nagios Administrators
members nagiosadmin,sue,john
}
define contactgroup{
contactgroup_name engineer2

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 36

alias Linux Administrators

members mark,tom
}
define contactgroup{
contactgroup_name engineer3
alias Linux Administrators
members mary,ralph
}

When you set up the notification process you can allow for overlap so that two contact groups could be working on the
problem.

Create a new configuration file, or place this information in an existing file in the /usr/local/nagios/etc/objects. Here
you can see serviceescalation is defined. You will need the host, service_description and then when notification will
start and end for this group. You also need to enter the notification_interval.

define serviceescalation {
host_name bash
service_description Procs
first_notification 5
last_notification 8
notification_interval 60
contact_groups engineer2
}
define serviceescalation {
host_name bash
service_description Procs
first_notification 9
last_notification 12
notification_interval 60
contact_groups engineer3
}

If you create notification intervals which which overlap, Nagios will use the interval that is the smallest.

Once that is saved and Nagios is restarted you should be able to go to the web interface and select
configuration/service escalations and see your changes.

When you use escalation_period it is important to realize that the notification_period is not removed, rather the
escalation_period must intersect the notification_period. If you have a time period defined as 24x7 as the

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 37

notification_period then anything you place in escalation_period will work. However, if the notification_period is a
the 12x4, illustrated below, and the escalation_period is workhours, illustrated below, escalations only occur for where
the two time periods intersect. So you would get escalations only from 09:00­17:00 on Mon­Thurs.

define timeperiod{
timeperiod_name 12x4
alias Tech Staff Hours
monday 06:00­18:00
tuesday 06:00­18:00
wednesday 06:00­18:00
thursday 06:00­18:00
}
define timeperiod{
timeperiod_name workhours
alias Normal Work Hours
monday 09:00­17:00
tuesday 09:00­17:00
wednesday 09:00­17:00
thursday 09:00­17:00
friday 09:00­17:00
}

You may control the escalation time frame but you will always want to take into account the notification_period first.
Here you can see the escalation_period is for workhours.

define serviceescalation {
host_name localhost
service_description Current Users
first_notification 5
last_notification 8
notification_interval 60
escalation_period workhours
contact_groups engineer2
}
define serviceescalation {
host_name localhost
service_description Current Users
first_notification 9
last_notification 12
notification_interval 60
escalation_period workhours
contact_groups engineer3
}

There may be times when you want the escalation to continue until a resolution of the problem. If that is the case then
the last_notification must be “0”. see the example. The final “last_notification” is changed to “0” instead of “12”.

define serviceescalation {
host_name localhost

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 38

service_description Current Users

first_notification 5
last_notification 8
notification_interval 60
escalation_period workhours
contact_groups engineer2
}
define serviceescalation {
host_name localhost
service_description Current Users
first_notification 9
last_notification 0
notification_interval 60
escalation_period workhours
contact_groups engineer3
}

Another way to fine tune escalations is when you look at escalation_options. The options that are available are:

Service Options
w­ warning
u­ unknown
c­ critical
r­ recovered
f­ flapping
s­ scheduled maintenance

Host Options
d­ down
u­ unreachable
r­ recovered
f­ flapping
s­ scheduled maintenance

The example below shows that the warning, flapping and scheduled maintenance have been removed for this service.

define serviceescalation {
host_name localhost
service_description Current Users
first_notification 5
last_notification 8
notification_interval 60
escalation_period workhours
escalation_options u,c,r
contact_groups engineer2
}
define serviceescalation {
host_name localhost

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 39

service_description Current Users

first_notification 9
last_notification 0
notification_interval 60
escalation_period workhours
escalation_options u,c,r
contact_groups engineer3
}

Notification: Host and Service Dependencies

A major factor of frustration for administrators is when checks depend upon a service or host and that when the host of
service goes down administrators receive notifications for all of the dependent hosts and services. This does not help
administrators solve the real problem and in fact directs the administrator in the wrong direction. For example, if you
were monitoring your Linux network using NRPE and the NRPE service failed, in addition to getting notified about
the NRPE service failure, you would receive notification from all of the services network wide you were
monitoring....ugly.

This problem can be eliminated by implementing host and service dependencies. In the example, all of the Linux
servers being monitored by NRPE are dependent upon NRPE. So the idea is to monitor the NRPE process to verify
that it is working and to add all of the NRPE checks as dependents.

The first thing you need to do is create a command definition for checking the NRPE process. Here a simple
command will determine if the process is functioning correctly.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 4: User Management 40

define command {
command_name nrpe_verify
command_line $USER1$/check_nrpe ­H $HOSTADDRESS$
}

Next you need to create a service definition.

define service{
host_name bash
service_description NRPE
check_command nrpe_verify
}

You will need to set up the service or host dependencies which are related. Here you can see that the one server has
listed 5 services which are dependent upon NRPE.

##### Dependency Checks #####

define servicedependency{
host_name bash
service_description NRPE
dependent_host_name bash
dependent_service_description DNS,Load,Mailq,NTP,Procs
notification_failure_criteria c,u
execution_failure_criteria n
}

By setting up these dependencies it will eliminate the unnecessary notifications.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 41

Chapter 5: Management

Management of the Nagios server can often be performed from the web interface. Users are able to view,
acknowledge problems and communicate situations using the web interface.

Web Interface
The web interface of Nagios Core provides a menu on the left with specific details in the main body of the page which
contains more clickable links to information.

Home
The “Home” link provides access to the main page which provides the current version with a link to check on updates
as well as links to training and certification options, news items, tutorials and links to Nagios plugins at Nagios
Exchange. This page will keep you up to date on the changes that happen with Nagios.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 42

Documentation
This link will take you to the Nagios site where you will find manuals, tutorials and links to other information about
Nagios.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 43

Tactical Overview
The “Tactical Overview” is designed to provide an overall look at the health of not only the Nagios server in the
“Monitoring Performance” but also the entire network that is being monitored including all of the hosts and services.
This information is updated every 90 seconds to provide a fresh look. Each window represents a different category on
your network; Monitoring Performance (Nagios server resources), Network Outages, Network Health (summary),
Hosts, Services and Monitoring Features (features currently available). Most of these are links to more detailed
information.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 44

Map
The “Map” represents ping connections to each of the hosts being monitored on the network. The map provides a
Layout Method which basically provides different views of the hosts. The “Drawing Layers” allows the selection of
different host groups. The “Scaling factor” allows for size representation on the image.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 45

The map has options to change how it looks and to select or deselect (Include or Exclude) categories of hosts. In this
example Ubuntu Servers is selected and the “Include” is used to only show Ubuntu servers.

Here is an example of the Circular (Ballon).

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 46

The Balanced Tree shows the same hosts and levels. The levels of hosts represent the parent relationships between
hosts. For example, if you had a virtual server hosting containers as you can see in this example. Or it may represent
the relationships of hosts, routers and switches.

Hosts
The “Hosts” link provides access to each of the hosts which are being monitored on the network. At the top of the
page is a summary of the network conditions. There are links to more specific details for host groups, and how to
view them in different ways. There are also two summary windows for hosts status and service status on the hosts.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 47

The link “Service Status Detail” for the hosts provide the complete list of all hosts and all services on those hosts.

“Status Overview” provides a graphical summary of host groups and services.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 48

“Status Summary” is a more compact way of viewing similar information. More information can be gained by
clicking on the links in the information.

Finally, there is the option to view the status in a grid.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 49

Services
The services can be viewed altogether by selecting the “Services” link in the menu. The “Host Status” and “Service
Status” summaries are at the top with additional links for a history of ths hosts, notifications sent and host detail
information.

For each service listed you see the host that the service check was on and the name of the “Service” which is a link for
more detailed information. This includes links to service details and service commands that can be executed.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 50

Host Groups
The “Host Groups” provides 3 views, status, summary and grid.

“Status Summary” is a more compact way of viewing similar information. More information can be gained by
clicking on the links in the information.

Finally, there is the option to view the status in a grid.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 51

Service Groups
The “Service Groups” provides 3 views, status, summary and grid.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 52

Problems
Detecting problems and making notification to administrators is the heart of what Nagios can do for you. The basic
view provides only problems related to hosts, whether they are WARNING states or CRITIAL states. If you click on
the specific problem you will be taken to greater detail about the problem.

If you select the history link you will see what has happened historically with problems. Note that you can make
modifications concerning what you see and how you see it with the options on the right.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 53

The notifications link takes you to a page which shows each notification and the reason the notification was sent. Note
that it provides information about how the notification was made as well. This will be especially helpful in tracking
escalation using various methods of communication.

The status detail link on the original “Problems” page simply provides access information about each host.

The “Problems” section in the menu also provides links to all hosts or services or network outages.

Quick Search
Once you start getting the web interface loaded with many hosts and services the “Quick Search” link can be very
helpful in finding hosts that are monitored on the network. Here the search for a host named “mail” shows results.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 54

Availability
Availability provides a way to see the average uptime of a system over a period of time. This is a common request for
evaluating performance and creating a report. “Availability” which is part of the “Reports” section has several steps
to take in making a selection of the information required. The first option allows you to select from; Hostgroup, Host,
Servicegroup or Service.

The second step allows you to select all of the hosts or services or to limit the output to a specific host or service.

Step three allows you to select the detail that you need to review for the report. These specifics include choosing time
parameters for the information, state information and assumptions and the ability to scan previous reports in order to
get a better picture of a problem developing over time.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 55

The output of the report breaks down the information into host and service details. The host details provides “State”,
the reason the state was reported, the time of the state as well as the percentage of time the host was in that state. The
service detail focuses on the time and percentage a service was in a particular state.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 56

Trends
Reviewing “Trends” is a way to review information that may be helpful in determining problems which are developing
over time or limitations which are developing for the network or hosts. The first step provides a choice of service or
host.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 57

The second step allows you to choose which host or service.

In the final step you can choose date limits, state assumptions and scan back archives for a better understanding of the
trend.

The trends report then provides the data in a chart format to help understand the trends.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 58

Alerts
The “Alerts” section provides a history of alerts that have been sent out. You can choose based on the type of
alert(soft or hard state), or the specific alerts you want to see like host, service, warning, critical, etc. Those options
work for the history.

The “Summary” for alerts allows you to configure the output of the alert information and limit it to a timeperiod, or

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 59

specific hosts, services that you need to review. This shows most recent alerts, top alert producers and alert totals.

Here is the output of the summary.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 60

The “Histogram” creates a historic graph of the alerts you select. This is a three step process. In the first step select
host or service.

In the second step select the specific host or service you would like to see.

The third step allows for a lot of fine tuning including time period, hosts and states.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 61

The graph has options as well so that you can refresh (update) and see new information. This is the Alert Histogram
which could be used for a report about alerts showing the frequency and number of alerts over time.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 62

Notifications
Notifications is simply a list of notifications that have been sent, how they were sent and to whom they were sent.
Once again there are options to limit the output.

Event Log
The event log represents the information gathered in the Nagios log. The event log provides a way for an
administrator to see a log of all that has happened over a period of time. You do have the option to reverse the order
seeing older first.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 63

Comments
“Comments” allows administrators to communicate about the status of hosts and services. This has the advantage of
demonstrating that specific problems are being handled so there is no duplication of effort.

As an administrator the major interest that you will have with the web interface is the ability to recognize and respond
to problems. The quickest access to all of the recognized problems is the “Problems” page. This page provides a
summary of all problems related to services that Nagios detects.

If an administrator wants to respond to an outage, the host can be selected and then at the bottom of the page a
response option is available.

Here the administrator can “Add a new comment” so that the next administrator recognizes that this problem is in the
process of being resolved.

The administrator can now add a comment to indicate the information that is know about the server.

Once this is entered other administrators will be able to see the situation and not repeat the steps that have already

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 64

been taken.

This way administrators can communicate about the situation.

When an administrator is going to take responsibility to solve the problem they can select the “Acknowledge this
problem” option in Service Commands.

When the Commands Options opens you have several options. The “Sticky Acknowledgment” when it is checked,
will prevent further notifications if the problem continues. The “Send Notifications” when checked, will notify the
other administrators so that they do not take action on something that is already being fixed.

“Persistent Comment” in Nagios 3 will retain the comment even after a reboot and must be manually unchecked when
it is fixed. If you leave it unchecked Nagios will remove the comment when a solution is found.

Downtime
If you are going to work on a server or device and need to schedule downtime so Nagios does not notify
administrators. This task can be performed at the web interface. When you select the host or service that will be
down you have an option to schedule downtime. When downtime is scheduled Nagios will place a comment in the
web interface in order to
communicate the fact to all administrators who access the web interface.

There are two types of downtime. Fixed downtime allows for an exact start and end time when the host or service

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 65

will be unavailable. Flexible downtime allows for a start time but an open ended startup time as the exact time cannot
be determined based on the nature of the situation.

Triggered downtime is when the downtime of a parent will trigger downtime for all of it's children. In other words,
the downtime for a switch, will impact all of the devices connected to it.

Scheduling Downtime for a Host

In order to schedule downtime for a host, select host details from the web interface. On the right hand side you will
notice the “yellow clocks” permit scheduling for host or services. Select the host option.

Once you have selected the host, “Command Options” appears and provides a place to explain why the downtime to
other administrators in the comment area, which is a good idea in most situations. If you select a “Fixed” time you
will enter the start and end of the downtime. If this machine provided network connection with other devices you
may want to notify downstream devices with a “triggered by” option that is created by this device going down. Or you
may choose to do nothing.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 66

On the Nagios interface on the left menu, if you select “Downtime” you will see a list of all scheduled downtimes for
hosts and services. Remember it may take a few minutes to allow the devices to show up.

Here is how the host looks with downtime (this is the exfoliation frontend), note the “yellow clock” which is an
indicator of scheduled downtime.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 67

If you select the clock you will see the details on the host list it as being in a scheduled downtime.

At this point it will be listed in the “Downtime” menu. Note you can cancel by deleting the downtime.

Notifications for downtime should stop in the downtime period. If the notifications do not stop verify that you do not
have the “d” option set for your contacts. The “d” option will send notifications on downtime.

Process Info
The Nagios process can be managed from this window. All of the options for stopping/starting/restarting the process
are here just as the ability to turn on/off notifications, service checks, passive checks, event handlers, flapping, and
performance data.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 68

Performance Info
Performance information can be viewed in a summary format by going to “System” “Performance Info” in the menu.
This summary gives you a idea about what is occurring on the Nagios server. This chart will list the services and
hosts and break up the checks based on whether they are active or passive checks and also list the latency information
based on those checks. The “Check Statistics” provides a summary of checks over three different time periods,
1,5,and 15 minute intervals.

All of this information is based on the nagiostats command which can provides the same information at the command
line:

nagiostats

or use the full path to the command

/usr/local/nagios/bin/nagiostats

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 69

Scheduling Queue
The scheduling queue gives you a window into how Nagios is performing by being able to see what is coming up in
the queue. This will help you understand queue health and detect latency issues.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 70

By selecting the clock you can alter the schedule for a service check. Once you enter “Commit” the queue will
change.

Configuration
This link give you the option to modify the configuration of your hosts and services from the web interface, not the
command line. As you can see in the image this scrolls off one page into numerous pages of options.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 71

If you select a link you will be able to modify that aspect of the service or host check. Here is an example of a “Check
Command” modification. Just make the changes and select “Update”.

Event Handlers
Event handlers are options to use when the host or service changes between an OK state to an error state. An
administrator may implement a “self­healing” script which will repair a situation before anyone is notified. Now
“self­healing” of course is a stretch because situations that arise repeatedly need to be examined by an administrator
and fixed properly. And yet even handlers do have a place in maintaining an organizations hosts and services.

There are several handler types that may be implemented; global service and host event handlers and host and service
specific event handlers. If global event handlers are run of course they will run for all hosts and services. Typically
organizations will select specific hosts or services to run event handlers on.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 72

Event Handler for Nagios Server

This example of setting up an event handler is performed on the localhost, or Nagios server. The goal is to restart the
web interface if it fails. There are four elements to setting up an event handler; the service definition, the command
definition; the script for the event handler and the permissions required.

The service is the typical check_http service definition but an additional line has been added for the event_handler.
This file is the localhost.cfg file and the service shows that this is a service­specific event handler. The event handler
name must match that of the command definition.

define service{
use local­service
host_name localhost
service_description HTTP
check_command check_http
event_handler httpd­restart
}

The commands.cfg contains definitions of commands and is where the definition for the event handler must be
entered. This is an event handler for a service so these macros must be added after the script name:
$SERVICESTATE$ $SERVICESTATETYPE$ $SERVICEATTEMPT$ . If it was a host these macros would be
required: $HOSTSTATE$ $HOSTSTATETYPE$ $HOSTATTEMPT$ . Note the location and name of the event
handler script. Your script name will vary.

define command{
command_name httpd­restart
command_line $USER1$/eventhandlers/httpd­restart.sh $SERVICESTATE$
$SERVICESTATETYPE$ $SERVICEATTEMPT$
}

The event handler script in this example will attempt to restart the web server after 3 SOFT problem states and once
again after the HARD state is reached if it has not been restarted. Paths for the commands indicated may change
based on the Linux distro used so check paths with:

which sudo
which service

#!/bin/sh
# Event Handler for Web Server on Nagios

case "$1" in
OK)
;;
WARNING)
;;
UNKNOWN)
;;
CRITICAL)
case "$2" in

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 73

SOFT)
case "$3" in

3)
echo ­n "Restarting Web service"
/usr/bin/sudo /sbin/service httpd restart
;;
esac
;;

HARD)
echo ­n "Restarting Web service"
/usr/bin/sudo /sbin/service httpd restart
;;
esac
;;
esac
exit 0

Once the file has been saved change the permissions and ownership so that it is
executable and is owned by nagios.

chmod 755 httpd­restart.sh

chown nagios http­restart.sh

The permissions to execute a service will need to be modified.

Security Tip
Whenever sudo is used it is important to consider the security implications. In this example the nagios user
is able to elevate privileges to root in order to execute the restart of a service. Note that the nagios user is
not required to have a password to obtain these rights. Only root can restart services, this is why it is
required. Open visudo as root and add these lines.

User_Alias NAGIOS = nagios,nagiocmd

Cmnd_Alias NAGIOSCOM = /sbin/service, /etc/rc.d/init.d/httpd
Defaults:NAGIOS !requiretty
NAGIOS ALL=(ALL) NOPASSWD: NAGIOSCOM

Once you have saved changes now test the set up by turning the web server off and viewing logs.

service httpd stop

Here is an example of the log file output indicating that the httpd server is up (HTTP;OK;SOFT) and then showing 3
SOFT problem states after which the script executes and the web server is running again.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 74

tail /var/log/nagios/nagios.log
Nov 21 06:59:18 nag2 nagios: SERVICE EVENT HANDLER: localhost;HTTP;OK;SOFT;4;httpd­restart
Nov 21 07:04:13 nag2 nagios: SERVICE EVENT HANDLER: localhost;HTTP;CRITICAL;SOFT;1;httpd­
restart
Nov 21 07:05:16 nag2 nagios: SERVICE EVENT HANDLER: localhost;HTTP;CRITICAL;SOFT;2;httpd­
restart
Nov 21 07:06:22 nag2 nagios: SERVICE EVENT HANDLER: localhost;HTTP;CRITICAL;SOFT;3;httpd­
restart
service httpd status
httpd (pid 25826) is running...

Host Groups
Often you will want to create a group of devices that have similar monitoring needs. The hostgroup allows you to then
create service checks that monitor all of the devices in the hostgroup. Specifically what this means is that the
services defined for the group will be available for all hosts in the group without making individual configurations.
Nagios will also list the hosts together in the web interface if they are in the same hostgroup.

Define Each Host

In order to set up a hostgroup, each server must be defined as a host. In this example, 3 Ubuntu servers are defined.

define host{
use linux­server
host_name ub
alias Ubuntu Server
address 192.168.5.180
}
define host{
use linux­server
host_name ub1
alias Ubuntu Server
address 192.168.5.181
}
define host{
use linux­server
host_name ub3
alias Ubuntu Server
address 192.168.5.183
}

Define Host Groups

Create hostgroups.cfg in the objects directory and create an entry in nagios.cfg to the location of hostgroups.cfg.

cfg_file=/usr/local/nagios/etc/objects/hostgroup.cfg

Define the hostgroup, in this example the hostgroup ubuntu_servers is defined with the three members that were
defined in hosts.cfg file.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 75

define hostgroup {
hostgroup_name ubuntu_servers
alias Ubuntu Servers
members ub,ub1,ub3
}

Define Services for the Group

The advantage of the hostgroup is that you can create one service definition and add that to the whole group of
servers. This is exactly the same as a regular service definition except you use hostgroup_name instead of host.

define service{
use generic­service
hostgroup_name ubuntu_servers
service_description Ping
check_command check_ping!60.0,5%!100.0,10%
}
define service{
use generic­service
hostgroup_name ubuntu_servers
service_description SSH Server
check_command check_tcp!22
}
define service{
use generic­service
hostgroup_name ubuntu_servers
service_description Web Server
check_command check_tcp!80
}

Now if you go to the web interface and select “Hostgroups” you will have a group of servers that are all related with
the same service checks.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 76

If you want to add individual service checks for one of the servers in the hostgroup that would be done as a regular
service definition using the host.

Service Groups
Nagios combines devices that are checking the same services into groups in order to make the set up faster and more
efficient. This allows an administrator to group machines based on services. Each of these services must be
configured as service checks for each host. Once that is complete the services may be grouped in the
servicegroups.cfg. The other major advantage is that the administrator may manage all those in the service group with
servicegroup commands in the web interface.

You will need to create a file called servicegroups.cfg and put an entry in nagios.cfg to indicate where it is. Note the
entries are in pairs (first host, then service) “host,service, host2,service2”.

define servicegroup{
servicegroup_name web
alias Web Servers
members ub, HTTP ,ub1, HTTP ,ub3, HTTP
}

Define each host with a normal service check.

define service{
use generic­service

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 77

host_name ub
service_description HTTP
check_command check_http
}
define service{
use generic­service
host_name ub1
service_description HTTP
check_command check_http
}
define service{
use generic­service
host_name ub3
service_description HTTP
check_command check_http
}

This now allows the administrator to group these services and view them as a group when “ServiceGroups” is selected
in the web interface.

Managing Nagios Time

The correct time on a server is critical, especially for Nagios. Synchronized time with other servers is important to
coordinate tasks and discover problems. One easy way to perform this task is to install ntp (Network Time Protocol)
and then manually have the system check time once a day in order to manage time.

yum install ­y ntp

Manually update your time. Here you can see the system is off by 12 seconds so it will be corrected gradually with
each check.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 78

ntpdate pool.ntp.org
11 Mar 12:09:55 ntpdate[24725]: step time server 71.245.107.83 offset 10900.806712 sec
12 sec

Create a cron entry so that each day your system is brought up to date.
crontab ­e

45 23 * * * /usr/sbin/ntpdate pool.ntp.org

Nagios Core BackUp

Most of the important information that is used for Nagios is located in the /usr/local/nagios directory. However, if you
use additional addons this will make it more difficult to find all of the files you need. Here are some common
directories that you may want to add:

/usr/local/pnp4nagios
/usr/local/nagvis

You can also search for all files on the system that are owned by Nagios with:

find / ­user nagios

Nagios Core Backups with Flat Files

Weekly Timestamped Backups

The weekly backups should be placed on a separate disk from the disk that Nagios is on. That will at least give you
way to rebuild even if you had to move the disk to a new location. Timestamps are important in that they allow you to
return to a known date. Make sure you provide enough disk space so that you can save 6 months worth. The script
creates a time stamp so that you know not only when it was created but so that no file will ever be able to overwrite it.
The time stamp is year, month, day, hour, minute and second so there will be no two the same. The “echo” command
makes sure that on each execution of the script all files in the script have the same time stamp.

bk.sh

#!/bin/bash
# Weekly Backup
TIMESTAMP=`date +%Y%m%d_%H%M%S`;
echo $TIMESTAMP
tar ­czvf /bk/nagios_$TIMESTAMP.tar.gz /usr/local/nagios
tar ­czvf /bk/httpd_$TIMESTAMP.tar.gz /etc/httpd

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 79

Daily Backups
Daily backups are provided so that you have quick access to restore a days work. Note, these backups are overwritten
each day because the backup name is the same.

daily.sh

#!/bin/bash
# Daily Backup
tar ­czvf /bk/nagios.tar.gz /usr/local/nagios
tar ­czvf /bk/httpd.tar.gz /etc/httpd

Backup Directory
Make sure this is a separate drive. You can then copy backups to offsite or another location as well. It is good to
maintain these files on the Nagios server so they are handy to get to. The backup directory should have two sets of
files, timestamped and non­timestamped.

/bk
httpd_20110403_171355.tar.gz
httpd.tar.gz
nagios_20110403_171355.tar.gz
nagios.tar.gz

Nagios Core with MySQL Database

You must also perform a proper backup of your MySQL database if you are using MySQL. The user that performs
mysqldump must be the root user and you will need to either do this manually or add the password to the script.

Weekly Timestamped Backups with MySQL

bk.sh

#!/bin/bash
# Weekly Backup
TIMESTAMP=`date +%Y%m%d_%H%M%S`;
echo $TIMESTAMP

tar ­czvf /bk/nagios_$TIMESTAMP.tar.gz /usr/local/nagios

tar ­czvf /bk/httpd_$TIMESTAMP.tar.gz /etc/httpd
mysqldump ­u root ­­password=your_password nagios > /bk/nagios_sql_$TIMESTAMP

Daily Backups with MySQL

daily.sh
#!/bin/bash
# Daily Backup

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 80

tar ­czvf /bk/nagios.tar.gz /usr/local/nagios

tar ­czvf /bk/httpd.tar.gz /etc/httpd
mysqldump ­u root ­­password=your_password nagios > /bk/nagios.sql

Restore Backups
If you are going to use backups the actual backup is only half the story. You must practice restoring information that
is backed up. When you restore tar files you must tell tar that since the files were created in reference to the “/”
directory you need to restore them that was so you will need to add “­C /”.

tar ­xzvf /bk/nagios.tar.gz ­C /

tar ­xzvf /bk/httpd.tar.gz ­C /

Automatically BackUp
It only makes sense to create cron jobs that make the backup process automatic. The tool to use for cronjobs is
crontab. The first thing you need to verify is the location of the scripts you will use. This is especially important with
the crontab is that you wan to use the full path for all scripts and commands.

Open a cronjob as root with:

crontab ­e

The “­e” is for edit and it will open an empty file is using CentOS. You will need to edit the file with vi so in order to
add text you must use the “i” to enter edit mode.

cron Format
There are six fields that must be used.
1 ­ minute 0­59, a ­ between numbers means a range 1­30, a comma between numbers means individual 1,5,8
2 ­ hour 0­23
3 ­ day of month 0­31
4 ­ month 0­12
5 ­ day of week 0­7 (both 0 and 7 are Sunday)
6 ­ command
Example: Run a program at 3:14 every day.
14 3 * * * /root/scripts/./bk.sh

Edit the crontab and then save in the normal vi fashion.

ESC ­to get out of edit mode.

:wq ­ to save

List the crontabs that you currently have with:

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 81

crontab ­l

It is important that you verify the backups are working and that you are capable of restoring them.

Reachability
Nagios has the ability to determine if a host is in a down state or if it is in an unreachable state. The practical
implications of both of these states is the same, stuff does not work. However, the troubleshooting aspect is quite
different. If a host is down, then of course the administrator needs to investigate the host specifically. However, if a
network device is down or so heavily loaded it restricts communication then the network administrator needs to focus
on the network devices and related issues. So reachability is concerned with the overall network health and how it
impacts your monitored hosts.

Nagios is able to discern the network structure and how it alters these down states and unreachable states by
understanding the path for data packets on the network. In other words, Nagios needs to know how equipment is
connected because that will help determine the situation. This is done by making a reference to the parent/child
relationships of connected network devices. This process allows Nagios to take into account the physical topology of
the network.

The next step in configuration is to look at the IP Address and hostname of the next network device. If Nagios is
connected to a switch then that should be also configured with a host definition. The difference is that you want to tell
Nagios that the parent of that switch device is the hostname nagios or localhost.

define host{
host_name ciscoswitch
parents localhost
}

You can only add devices that have the ability to be assigned an IP Address and /or a hostname.

The key in the design is recognizing the network configuration and telling Nagios which is the parent, or network
device, directly above the host you are working with. Once your data packet hits your external interface on your router
you cannot specify routers on the Internet as the path will vary depending upon best route. So if you were tracing the
data packet path from Nagios to a remote device you would need to indicate the IP Address of the external router
connecting the device to the Internet.

Virtual Hosting Example

A good example of a parent/child relationship is when a virtual host is used to manage several containers. The concern
is if the host goes down of course all of the containers will go down as well.

vzlist ­a
CTID NPROC STATUS IP_ADDR HOSTNAME
161 25 running 192.168.5.161 mk
162 22 running 192.168.5.162 nagvis

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 82

170 ­ stopped 192.168.5.170 host

172 ­ stopped 192.168.5.172 mail
173 16 running 192.168.5.173 test
174 14 running 192.168.5.174 test2
180 18 running 192.168.5.180 ub
181 9 running 192.168.5.181 ub2
183 9 running 192.168.5.183 ub3
190 17 running 192.168.5.190 bash

Here is an example of the host definitions for several containers as well as the parent. Note that vz is listed as
“parents” in the host definition.

vz (parent)
nagvis (container)
corp­mail (container)
mk (container)

define host{
use linux­server
host_name vz
alias VZ Server
address 192.168.5.160
}
define host{
use linux­server
host_name nagvis
alias Mapping Server
address 192.168.5.162
parents vz
}
define host{
use linux­server
host_name corp­mail
alias Mail Server
address 192.168.5.172
parents vz
}
define host{
use linux­server
host_name mk
alias Passive Server
address 192.168.5.161
parents vz
}

This image demonstrates the relationship that the containers have to the parent.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 83

An additional container does not list vz as “parents” and therefore is treated differently.

define host{
use linux­server
host_name bash
alias Bash Scripts
address 192.168.5.190
}

If an administrator needed to schedule downtime for vz it would certainly impact all of the containers. In this example
the “Child Hosts” are triggered with this downtime.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 84

Here the scheduled downtime clearly demonstrates that the containers that have “vz” listed as “parents” are all
included. However, the container “bash” is not included in the process as it does not list the child/parent relationship
with “vz”.

There is another issue here to consider. As an administrator you understand the topology of the network, it is obvious
if “vz” is down the containers will also be down. By default the administrator will not only get a notification that “vz”
is down but also get notifications that all containers are down as well, possibly unwanted notifications. However, as an
administrator you WILL want notification on a container that is down but the parent is up.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 85

The default notification options in the linux­server template include notifications for down, unknown and recovery.

notification_options d,u,r

The unknown state relates to the example below where Nagios knows the host is down but does not know the state of
the containers because they are “unreachable” or “unknown”. Therefore if you did not want notifications for the
containers being down, remove the “u” option from notification_options as in the example.

define host{
use linux­server
host_name nagvis
alias Mapping Server
address 192.168.5.162
parents vz
notification_options d,r
}

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 86

If the administrator does not enter the parent/child relationship for “bash” which is a container, then there is not going
to be an opportunity to disable notifications for the “unreachable” state as Nagios will believe it is a direct connection
and as a result will determine the state to be “down” instead of “unreachable”. Obviously, creating the relationships
provides better control of accurate notifications and limiting notifications.

Network Outages
Network outages represent hosts that are in a DOWN state and as a result are blocking downstream hosts and services.
Network outages therefore, relate to the parent/child relationship that has been created.

Volatile Service
A volatile service is a service that will automatically return itself to an "OK" status when it is checked. Or, it is a
service that needs to be checked by an administrator on each occurrence, like a security event. Volatile services are
different than normal services in that:

* the non­OK state is logged

* contacts will be notified on each event
* event handlers are run on each event

Since the state is returned to an OK state after each event, one of the changes that should be made with a volatile
service is that the “max_check_attempts” are set to one so that each event will trigger a hard state. If this was set to a
higher number than one it would never reach the HARD state as it is reset.

is_volatile = 1
max_check_attempts =1

State Stalking
State stalking provides for detailed logging information. The goal in using state stalking is that as much detail as
possible is placed in the logs for further review after the event. Instead of just logging state changes, from OK to
WARNING for example, stalking logs any changes from the previous check. So not only state changes are logged but
information that occurs during the same state is also logged.

stalking_options = [o,d,u]

The stalking directive provides three options: “o” for stalking on UP states, “d” for stalking on down states and “u” for
stalking on UNREACHABLE states.

Flapping
A flapping state is when a service or host changes from an OK state to CRITICAL state rapidly. These changing
states will send multitudes of notifications to administrators which can be non­productive. When flapping is detected

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 87

Nagios will recognize the changing states and move into a state of flapping which provides additional options for an
administrator which could allow unwanted notifications.

In order to detect this flapping state Nagios saves in memory 21 checks for each host and service. Nagios reviews the
last 20 changes to determine if the host or service is changing states based on a percentage. In this review of states the
more recent checks are provided a greater weight than the older checks as this is probably more important to an
administrator. Nagios also provides two thresholds for a service and a host so that an administrator can set an upper
and lower threshold which means that when the service or host goes above the upper threshold Nagios recognizes this
as state flapping which means notifications will be stopped, an entry in the log is created and a comment is placed in
the web interface so it can be reviewed by administrators. Once the percentage goes below the lower limit the
comment is removed and the service is returned to a normal state with notifications enabled. This process takes a
period of time to occur. Here is an example of a service that is flapping. If you look closely you can see the
percentage of state change.

Notifications for this service are being suppressed because it was detected as having been flapping between different
states (12.7% change ). When the service state stabilizes and the flapping stops, notifications will be re­enabled.

To make changes to the settings for flap detection, first access the nagios.cfg file which provides global settings. The
first setting that can be altered is that an administrator can turn flapping off by changing the value to “0”. The
thresholds may be modified to meet specific requirements for the organization. Remember these thresholds are
percentages so the low end is 5%, or one state change and the upper end is 20% which equals five state changes.

enable_flap_detection=1

low_service_flap_threshold=5.0
high_service_flap_threshold=20.0
low_host_flap_threshold=5.0
high_host_flap_threshold=20.0

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 88

Specific changes could be made with the specific service as well. The “flap_detection_enabled” must be included to
allow the override of the global settings. The two thresholds then may be modified to meet the needs of the service.

define service{
use generic­service
host_name centos
service_description SMTP
check_command check_smtp
flap_detection_enabled 1
low_flap_threshold 10.0
high_flap_threshold 30.0
}
There is another option that is available with flapping. This option allows an administrator to control which states
indicated flapping. The states available are o(OK), w(WARNING), c(CRITICAL) and u(UNKNOWN). States that
are not listed are not taken into account to determine flapping.

flap_detection_options o,w,c,u

As you can see in this illustration you can also “Disable flap detection for this host” under the “Host Commands”.
This provides the option to just perform the task as it happens. Here is the verification before you commit the change.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 89

Resolving Problems
If you believe the problem for a host or service has been resolved and you do not want to wait until the check is
performed again, you can proceed to the host or service and then click the “Re­schedule the next check” option to get
an immediate check performed.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 90

Disabling Notifications
A permanent solution for false notifications is the option to disable notifications for a host or service. Select the host
or service and then click “Disable notifications for this service/host”.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 91

Sending Mail From Nagios

Every Linux server that is installed, uses a mail server to send mail locally. The mail server, whether it is Sendmail or
Postfix, is enabled to send mail by default but not to receive mail. In addition, the mail server is configured to run at
start up of the server. You can confirm this with the netstat command:

netstat ­aunt
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN

Here the fact the mail server is running on port 25 and is listening, but only on the localhost, 127.0.0.1 What this
means is that the mail server will be sending reports to the root user on the localhost and could send mail to another
mail server on port 25, but it could not receive any mail outside of the mail server. That is good news, you do not want
to turn your Nagios server into a mail relay. Nagios is able to use the mail system to send mail notifications to mail
recipients and all of it is set up automatically. However, this is only part of the equation. Whenever you have an
application sending email the other half of the equation is the receiving mail server. Often email notifications do not
work so here are several solutions.

Solutions for the Nagios Server

1. Create a Fully Qualified Domain Name (FQDN) for the Nagios server
The FQDN looks like this in two parts: mail.example.com. The hostname is mail and the domain is represented in
example.com, which when you combine the two becomes the FQDN. The mail server that receives the email from
Nagios probably requires a FQDM. So your hostname on the Nagios server must satisfy this requirement.

On a CentOS system you can create a FQDN by editing two files. The first file to edit is /etc/sysconfig/network. Note

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 92

the “HOSTNAME” is specifically listed.

NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=nagios.example.com
GATEWAY=192.168.3.1

The second file to edit is /etc/hosts. Note that the 127.0.0.1 represents the localhost. There are also two examples of
the hostname “nagios.example.com” and “nagios”. These both must be listed. Of course change the name of the
server and the domain to fit your situation.

127.0.0.1 nagios.example.com nagios localhost.localdomain localhost

::1 localhost6.localdomain6 localhost6

2. Check tcp_wrappers
One of the major reasons that mail may not be sent when you have a Nagios box set up is that tcp_wrappers could be
stopping the ability of the localhost to send email. Edit the /etc/hosts.allow so that it has this line:

ALL: 127.0.0.1

That will allow the localhost to send mail to the Nagios administrators.

3. Allow Nagios to Resolve Using DNS

One of the tests that most mail servers perform is to see if the DNS resolves correctly for the sending mail server, in
this case Nagios. If you create a DNS entry for Nagios so that your Nagios server resolves on the Internet, the mail
sent from Nagios is more likely to be received. However, as is often the case, organizations may be more concerned
about the security of Nagios so allowing DNS resolution may not be the best choice. In that case, relay the mail sent
from Nagios to the corporate mail server.

4. Allow Nagios to Relay Mail Through a Mail Server

Often mail will not be delivered because the mail server that Nagios is sending to will not relay the mail sent from
Nagios. By default all mail servers are designed to stop relays. Therefore the organization will need to configure the
corporate mail server to be able to relay mail from the Nagios server.

Without making Nagios a full blown mail server, those four options should get the mail working.

Testing Your Mail

If you wanted to check to see if you can send mail use this procedure. First, send email from the Nagios server to a
gmail account. Gmail is more likely to accept mail from a Nagios as it is not as restricted as the corporate mail server,
probably. As a user run the mail command:

mail user@some_gmail_account

Here is what it will look like when yo u send mail from the command line(note you end the mail body with a “.” on a
line by itself:

mail user@gmail.com
Subject: Testing Nagios Mail

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 93

This is a test of the Nagios mail notifications.

.
Cc:

That will send mail using Sendmail(CentOS) to the email address you specified. You can check your logs and you
should see something like this indicating the mail was sent.

tail /var/log/maillog
Mar 27 06:52:45 nagios sendmail[24474]: n2RCqjn3024474: to=user@some_email.com,
ctladdr=root (0/0), delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=30045,
relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (n2RCqjDf024475 Message
accepted for delivery)

You can test mail locally on the Nagios server with telnet. Be sure to indicate you are connecting on port 25 and on
the localhost as it will only allow connections on the localhost. The commands you need for telnet are highlighted.
Be sure to use the domain of your Nagios server. The body of the email is the “DATA” and you end that with a “.” on
a line by itself.

telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
220 nag2.local.net ESMTP Sendmail 8.13.8/8.13.8; Sat, 13 Nov 2010 07:30:29 ­0700
MAIL FROM:<test@example.com>
250 2.1.0 <test@example.com>... Sender ok
RCPT TO:<user@local.net>
250 2.1.5 <user@local.net>... Recipient ok
DATA
354 Enter mail, end with "." on a line by itself
Testing the Mail System.
.
250 2.0.0 oADEUTI6002388 Message accepted for delivery
QUIT
221 2.0.0 nag2.local.net closing connection
Connection closed by foreign host.

If you cannot telnet to the localhost on port 25 the mail server may not be working.

Redundant Mail Servers

When Nagios sends mail out it is important that Nagios relay through a mail server, or multiple mail servers, that will
provide dependable delivery. The standard way to provide this is to have a primary and secondary mail server to
insure the mail is relayed to the administrators. The illustration demonstrates this principle.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 94

The mail server that is primary and the one that is secondary is determined by the DNS MX records for the domain.

IN NS ns.example.com.
example.com. IN A 192.168.3.1
mail.example.com. IN A 192.168.3.2
mail2.example.com. IN A 192.168.3.3
example.com. IN MX 10 mail.example.com.
example.com. IN MX 20 mail2.example.com.

In this example mail.example.com is the primary because it has the lowest number. If the primary is not available the
secondary will accept mail delivery.

Commit Error from the Web Interface

If you run into this error it is related to a permissions problem.

Error: Could not open command file '/usr/local/nagios/var/rw/nagios.cmd' for

update!
The permissions on the external command file and/or directory may be incorrect.
Read the FAQs on how to setup proper permissions.

An error occurred while attempting to commit your command for processing.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 95

Here are the permissions on named pipe, which are correct.

ls ­l /usr/local/nagios/var/rw
total 0
prw­rw­­­­ 1 nagios nagcmd 0 Mar 6 15:25 nagios.cmd

However, the problem relates to the apache process being able to make the commit change which would require the
user apache to have write access to the pipe. To make that happen add apache to the nagcmd group in /etc/group.

nagcmd:x:501:nagios,apache

Now restart apache and you will see that it works fine.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 5: Management 96

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 97

Chapter 6: Monitoring
Nagios provides numerous methods to monitor a device whether that be with plugins or scripts which access clients
using public ports, NRPE, SSH NSClient++, SNMP or even accepting passive checks from clients. Monitoring
choices are based on network protocol requirements or administrative skills in configuring the options for monitoring.

This chapter provides a snapshot of some of those options for monitoring so that the student has basic framework for
how these are set up. This information is not intended for a comprehensive approach to using each option.

Plugin Use
Plugins are used to gather information and return that information to the Nagios server. Each plugin will evaluate the
situation and return a status value to Nagios. There are four status values that Nagios interprets.

0 OK the stats is as expected

1 WARNING a warning limit has been reached
2 CRITICAL a critical limit has been reached
3 UNKNOWN the status is unknown, misconfiguration

In order for Nagios to provide these four levels of status settings, warning and critical limits must be established. An
important aspect of setting these limits is that each network will have different equipment and varying needs so these
settings should reflect the individual network. Here are typical options for most plugins.

Typical Options
­h, ­­help Print detailed help screen
­V, ­­version Print version information
­H ­­hostname=ADDRESS Host name, IP Address, or unix socket (must be an absolute path)
­w ­­warning=DOUBLE Response time to result in warning status (seconds)
­c ­­critical=DOUBLE Response time to result in critical status (seconds)
­t ­­timeout=INTEGER Seconds before connection times out (default: 10)
­v ­­verbose Show details for command­line debugging (Nagios may truncate output)
­4 ­­use­ipv4 Use IPv4 connection
­6 ­­use­ipv6 Use Ipv6 connection

Monitoring Public Ports

Public ports are simply ports that are available to anyone, so they provide ans easy way to monitor a host or service.
However, the extent that a host can be monitored with public ports is limited to what can be discerned from the
outside. Monitoring public ports, whether they may be TCP or UDP, is a basic option for Nagios. Keep in mind that
firewalls can provide limited access to these public ports.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 98

check_ping
Ping is a standard method of checking to see if a network device is up and this use of ICMP is the typical method
used to monitor a host with Nagios. If the host does not respond to the ICMP check it is assumed the host is down.

Uniq Options
­p ­­packets=INTEGER number of ICMP ECHO packets to send (Default: 5)

Here is a service definition with a warning level of 60 milliseconds or 5% packet loss and a critical level of 100
milliseconds or 10% loss. This demonstrates that the settings need to be specific to the device or the network as
networks vary. The default is 5 packets in the ping.

define service{
use generic­service
host_name centos
service_description Ping
check_command check_ping!60.0,5%!100.0,10%
}

The command definition can include the settings for warning and critical level if you want to make them standard for
all uses of ping on a network.

define command{
command_name check­host­alive
command_line $USER1$/check_ping ­H $HOSTADDRESS$ ­w 3000.0,80% ­c
5000.0,100% ­p 5
}

check_tcp
This plugin will provide the flexibility you need if you need to monitor a port just to verify that the port is available.
Here is an example of portmap service checks.

define service{
use generic­service
host_name centos
service_description Portmap
check_command check_tcp! 111
}

define command{
command_name check_tcp
command_line $USER1$/check_tcp ­H $HOSTADDRESS$ ­p $ARG1$ $ARG2$
}

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 99

Note that a common problem with check_tcp is that often the “­p” is added to the service definition. This will create
the error “Port must be a positive integer” if the command definition already has the “­p”.

If you have any problems run the command from the command line to experiment.

./check_tcp ­H 192.168.5.1 ­p 111

TCP OK ­ 0.000 second response time on port 111|
time=0.000386s;;;0.000000;10.000000

check_smtp
Mail server communication on port 25 which is SMTP, Simple Mail Transfer Protocol. In order to check to see if the
mail server is able to communicate with other mail server check port 25.

Uniq Options
­p ­­port=INTEGER Port number (default: 25)
­e ­­expect=STRING String to expect in first line of server response (default: '220')
­C ­­command=STRING SMTP command (may be used repeatedly)
­R ­­command=STRING Expected response to command (may be used repeatedly)
­f ­­from=STRING FROM­address to include in MAIL command, required by Exchange 2000
­F ­­fqdn=STRING FQDN used for HELO
­D ­­certificate=INTEGER Minimum number of days a certificate has to be valid.
­S ­­starttls Use STARTTLS for the connection.
­A ­­authtype=STRING SMTP AUTH type to check (default none, only LOGIN supported)
­U ­­authuser=STRING SMTP AUTH username
­P ­­authpass=STRING SMTP AUTH password

Here are two examples of service checks, one default on port 25 and the other on port 587.

define service{
use generic­service
host_name centos
service_description SMTP
check_command check_smtp
}
define service{
use generic­service
host_name centos
service_description SMTP Secure
check_command check_smtp!­p 587
}

This is the default command definition which allows the argument to use the port number.

define command{
command_name check_smtp
command_line $USER1$/check_smtp ­H $HOSTADDRESS$ $ARG1$
}

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 100

Check from the command line to verify your check works as expected.

./check_smtp ­H 192.168.5.1
SMTP OK ­ 0.002 sec. response time|time=0.002099s;;;0.000000

Or if you are using port 587 you can test with this check which just adds a different port.

./check_smtp ­H 192.168.5.1 ­p 587

SMTP OK ­ 0.012 sec. response time|time=0.011947s;;;0.000000

check_pop, check_spop, check_imap, check_simap, check_mysql

­p ­­port=INTEGER Port number (default: none)

­E ­­escape Can use \n, \r, \t or \ in send or quit string. Must come before send or quit
option Default: nothing added to send, \r\n added to end of quit
­s ­­send=STRING String to send to the server
­e ­­expect=STRING String to expect in server response (may be repeated)
­A ­­all All expect strings need to occur in server response. Default is any
­q ­­quit=STRING String to send server to initiate a clean close of the connection
­r ­ ­­refuse=ok|warn|crit Accept TCP refusals with states ok, warn, crit (default: crit)
­M ­­mismatch=ok|warn|crit Accept expected string mismatches with states ok, warn, crit (default: warn)
­j ­­jail Hide output from TCP socket
­m ­­maxbytes=INTEGER Close connection once more than this number of bytes are received
­d ­­delay=INTEGER Seconds to wait between sending string and polling for response
­D ­­certificate=INTEGER Minimum number of days a certificate has to be valid.
­S ­­ssl Use SSL for the connection.

check_imap
IMAP, Internet Message Access Protocol, is the interface that accesses mail on the mail server and provides access to
user. There are several plugins that are options.

Of course you need to set up the hosts for the mail servers that you want to monitor. The first service is a check on
IMAP port 143 with a timeout of five seconds and it searches for a text string “OK” which is part of the response of
the IMAP server to a connection request. The “­e” is the expect string.

define service{
use generic­service
host_name lts
service_description IMAP4 Response Check
check_command check_imap!­t 5 ­e "OK"
}

define command{

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 101

command_name check_imap
command_line $USER1$/check_imap ­H $HOSTADDRESS$ $ARG1$
}

Here you can see a response from a mail server in the logs, that you can scan for a text string you want to test for.

nagios: SERVICE ALERT: lts;IMAP4 Dovecot Check;OK;SOFT;2;IMAP OK ­ 0.002 second

response time on port 143 [* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL­IR LOGIN­
REFERRALS ID ENABLE STARTTLS LOGINDISABLED] Dovecot ready.]

check_simap
The service IMAPS shows the port listing as port 993 while it searches for a more detailed text string in “Dovecot
ready”. That search string specifically indicates that Dovecot, the MDA (Mail Delivery Agent), is ready to
communicate. Searches are case sensitive and you will need to place specific text in these searches that relate to your
MDA and the ports you are using. The expect string "Dovecot ready" is an example for a Linux Dovecot daemon so it
is important to add the string expected with the Mail Delivery Agent you are using.

define service{
use generic­service
host_name ubpost
service_description IMAPS Check
check_command check_simap!­p 993 ­t 5 ­e "Dovecot ready"
}

define command{
command_name check_simap
command_line $USER1$/check_simap ­H $HOSTADDRESS$ $ARG1$
}

check_ftp
The basic command will connect on port 21 expecting a 220 reply from the FTP server. In this example, the return
information not only shows the “220” but also the version of the FTP daemon (vsFTPd 2.0.5).

./check_ftp 192.168.5.1
FTP OK ­ 0.003 second response time on port 21 [220 (vsFTPd 2.0.5)]|time=0.002800s;;;0.000000;10.000000

define command{
command_name check_ftp
command_line $USER1$/check_ftp ­H $HOSTADDRESS$ $ARG1$
}

define service{

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 102

use generic­service
host_name centos
service_description FTP
check_command check_ftp
}

Modifications
If you wanted to lower the timeout from the default 10 seconds to 4 seconds by using the “­t” timeout option and
adding the timeout. By changed the expect string “­e” to vsFTPd 2.0.5" Nagios will send notifications if the string
does not show up meaning either the FTP server is down or the version has changed, both are valuable information.

check_command check_ftp! ­t 4 ­e "vsFTPd 2.0.5"

check_http
A common public port that often is check is port 80, http. There are a significant number of options with this plugin
to get out of it as much as possible.

­I ­­IP­address=ADDRESS IP address or name (use numeric address if possible to bypass DNS

lookup).
­p ­­port=INTEGER Port number (default: 80)
­S ­­ssl Connect via SSL. Port defaults to 443
­­sni Enable SSL/TLS hostname extension support (SNI)
­C ­­certificate=INTEGER Minimum number of days a certificate has to be valid. Port defaults to 443
­e, ­­expect=STRING Comma­delimited list of strings, at least one of them is expected in
the first (status) line of the server response (default: HTTP/1.) If specified skips all other status line logic (ex: 3xx,
4xx, 5xx processing)
­s ­­string=STRING String to expect in the content
­u ­­url=PATH URL to GET or POST (default: /)
­P ­­post=STRING URL encoded http POST data
­j ­­method=STRING (HEAD, OPTIONS, TRACE, PUT, DELETE) Set HTTP method.
­N ­­no­body Don't wait for document body: stop reading after headers.
­M ­­max­age=SECONDS Warn if document is more than SECONDS old. the number can also be of
the form "10m" for minutes, "10h" for hours, or "10d" for days.
­T ­­content­type=STRING specify Content­Type header media type when POSTing
­l ­­linespan Allow regex to span newlines (must precede ­r or ­R)
­r ­­regex, ­­ereg=STRING Search page for regex STRING
­R ­­eregi=STRING Search page for case­insensitive regex STRING
­­invert­regex Return CRITICAL if found, OK if not
­a ­­authorization=AUTH_PAIR Username:password on sites with basic authentication
­b ­­proxy­authorization=AUTH_PAIR Username:password on proxy­servers with basic authentication
­A ­­useragent=STRING String to be sent in http header as "User Agent"
­k ­­header=STRING Any other tags to be sent in http header. Use multiple times for additional
headers
­L ­­link Wrap output in HTML link (obsoleted by urlize)
­f ­­onredirect=<ok|warning|critical|follow|sticky|stickyport>
­m, ­­pagesize=INTEGER<:INTEGER> Minimum page size required (bytes) : Maximum page size required (bytes)

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 103

This is the standard way to use the check_http. It checks to verify communication is available on port 80 of a web
server. This is in fact, a better check on the server than the check_ping which can only determine if the server is up.
This simple check provides some peace of mind and a place to start.

define service{
use generic­service
host_name centos
service_description HTTP
check_command check_http
}

These two checks are related to the SSL options with the web server. Note that the checks change to port 443 if you
use the ­­ssl option, they are testing to see if the web server can serve secure pages and if the web server certificate is
valid for the next 21 days. The first check will test for a response within a limited time frame, 5 seconds for a warning
or more than 10 seconds for a critical state.

define service{
use generic­service
host_name centos
service_description Secure HTTP
check_command check_http! ­w 5 ­c 10 ­­ssl
}

This check is focused on the certificate. In this example, if the certificate is good for more than 21 days an “OK” is
returned. A WARNING state is triggered if the certificate has less than 21 days before it expires. A CRITICAL state
is triggered when the certificate has expired.

define service{
use generic­service
host_name centos
service_description Certificate
check_command check_http! ­C 21
}

Both of the service checks above will return the following output in the Nagios web interface.
OK ­ Certificate will expire on 05/25/2012 23:59.

This usage of check_http allows you to check to see if a directory requiring authorization with username and password
is working correctly. Note that the check_http has been redefined to check_http_auth so that additional arguments can
be used. The service definition includes the IP Address of the server, the directory that requires authentication (­
u/sales) and the username and password required to access the directory. Each is separated by a “!”. Note the
command definition included.

define service{
use generic­service
host_name centos
service_description Sales Authorization

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 104

check_command check_http_auth!192.168.5.1 ­u/sales!tom!

user_password
}

define command{
command_name check_http_auth
command_line $USER1$/check_http ­H $ARG1$ ­a $ARG2$:$ARG3$
}

If the user login is not correct warning will be issued with the “401 Authorization Required”. This enables you to
verify password changes and integrity. However, leaving a plain text password in the Nagios config files is not the
best idea.

The check_http plugin can also be used to monitor content on a web page.

check_mysql
If you had a Wordpress blog with a MySQL database you could monitor that database from a public port or you can
monitor internally and return the information to Nagios using NRPE.

Security Tip
The first thing you want to make sure of is that you have placed a password on the root account for
MySQL.

mysqladmin ­h localhost ­u root password "your_password"

Note: If you compile the Nagios plugins, you must install packages for mysql if you want to use the check_mysql
plugin.

yum install ­y mysql mysql­devel

Host to be Monitored
If your Wordpress database was called "word" you could allow read rights to the nagios user from the IP Address of
your Nagios server.

GRANT select ON word.* TO nagios@192.168.5.51;

This setting is important to only allow “select” on the

Edit the commands.cfg file to create a command option that represents your database.

define command{
command_name check_mysql
command_line $USER1$/check_mysql ­H $HOSTADDRESS$ ­u nagios ­d word
}

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 105

Edit the services.cfg file to create a service definition.

define service{
use generic­service
host_name centos
service_description Wordpress_Database
check_command check_mysql
}

Another way to design this is to allow the administrator to enter different users and databases if the need was to
monitor several different databases.

define command{
command_name check_mysql
command_line $USER1$/check_mysql ­H $HOSTADDRESS$ ­u $ARG1$ ­d $ARG2$
}

define service{
use generic­service
host_name centos
service_description Wordpress_Database
check_command check_mysql!nagios!word
}

Checking multiple databases on the same server will require the same basic steps as before with MySQL and you will
probably want a different user. Here the database name is db7 and the user is webadmin.

GRANT select ON db7.* TO webadmin@192.168.5.51;

Here you can use the same command definition but you will need to change the service_description and the
check_command to fit the database you want to check.

define service{
use generic­service
host_name centos
service_description DB7_Database
check_command check_mysql!webadmin!db7
}

The command “show staus;” once you have selected a database provides similar output as the information provided
when Nagios collects a summary.

show status;
+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­+
| Variable_name | Value |
+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­+
­­­cut­­­
| Bytes_received | 428 |

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 106

| Bytes_sent | 14514 |
­­­cut­­­
| Com_show_databases | 1 |
| Com_show_errors | 0 |
| Com_show_fields | 11 |
­­­cut­­­
| Com_show_processlist | 1 |
­­­cut­­­
| Compression | OFF |
| Connections | 62 |
­­­cut­­­
| Handler_read_rnd_next | 267 |
| Handler_rollback | 0 |
| Handler_savepoint | 0 |
| Handler_savepoint_rollback | 0 |
| Handler_update | 0 |
| Handler_write | 396 |
| Innodb_buffer_pool_pages_data | 19 |
| Innodb_buffer_pool_pages_dirty | 0 |
| Innodb_buffer_pool_pages_flushed | 0 |
| Innodb_buffer_pool_pages_free | 493 |
| Innodb_buffer_pool_pages_misc | 0 |
| Innodb_buffer_pool_pages_total | 512 |
| Innodb_buffer_pool_read_ahead_rnd | 1 |
| Innodb_buffer_pool_read_ahead_seq | 0 |
| Innodb_buffer_pool_read_requests | 77 |
| Innodb_buffer_pool_reads | 12 |
­­­cut­­­
| Innodb_data_read | 2494464 |
| Innodb_data_reads | 25 |
| Innodb_data_writes | 3 |
| Innodb_data_written | 1536 |
| Innodb_dblwr_pages_written | 0 |
| Innodb_dblwr_writes | 0 |
| Innodb_log_waits | 0 |
| Innodb_log_write_requests | 0 |
| Innodb_log_writes | 1 |
| Innodb_os_log_fsyncs | 3 |
| Innodb_os_log_pending_fsyncs | 0 |
| Innodb_os_log_pending_writes | 0 |
| Innodb_os_log_written | 512 |
| Innodb_page_size | 16384 |
| Innodb_pages_created | 0 |
| Innodb_pages_read | 19 |
­­­cut­­­
| Key_blocks_unused | 7189 |
| Key_blocks_used | 56 |
| Key_read_requests | 3128 |
| Key_reads | 56 |
| Key_write_requests | 494 |
| Key_writes | 307 |
| Last_query_cost | 10.499000 |

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 107

| Max_used_connections | 5 |
| Not_flushed_delayed_rows | 0 |
| Open_files | 34 |
| Open_streams | 0 |
| Open_tables | 17 |
­­­cut­­­
| Queries | 1439 |
| Questions | 19 |
| Rpl_status | NULL |
| Select_full_join | 0 |
| Select_full_range_join | 0 |
| Select_range | 0 |
| Select_range_check | 0 |
| Select_scan | 4 |
| Slave_open_temp_tables | 0 |
| Slave_retried_transactions | 0 |
| Slave_running | OFF |
| Slow_launch_threads | 0 |
­­­cut­­­
| Threads_created | 61 |
| Threads_running | 1 |
| Uptime | 19367 |
| Uptime_since_flush_status | 1439 |
+­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­+­­­­­­­­­­­+
249 rows in set (0.00 sec)

Testing for Page Content

Nagios can be used to actually verify that content can be read on the web page. So not only will you know that the web
server is up and the port open but you can verify that a specific page is readable.

/usr/local/nagios/libexec/./check_http ­H 192.168.5.1 ­u /2011/01/09/nagios­

test/ ­s "Nagios Test"
HTTP OK: HTTP/1.1 200 OK ­ 9381 bytes in 0.344 second response time |
time=0.344154s;;;0.000000 size=9381B;;;0

The check_http plugin is used followed by the host, in the command definition. The “­u /2011/01/09/nagios­test/” is
the page. Since this is a Wordpress site it contains the folder that you see before the page itself. Note that the page is
followed by a “/”.

define command{
command_name check_http_page
command_line $USER1$/check_http ­H $HOSTADDRESS$ ­u /2011/01/09/nagios­
test/ ­s "Nagios Test"
}

define service{
use generic­service
host_name centos
service_description Wordpress_Page
check_command check_http_page
}

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 108

Monitoring Linux
The Nagios Remote Plugin Executor or NRPE allows you to execute programs for monitoring purposes on the remote
server. One advantage of NRPE is that it does not require a login to perform the tests on the remote server.NRPE
allows you to monitor internal aspects of a Linux server from the Nagios server. When you monitor public ports like
HTTP you can determine if the web server is running by using these service checks, but you are not able to monitor
other aspects of the server which you may need information on, which is why you will want to use NRPE.

NRPE Concepts
NRPE is able to perform two types of checks, Direct and Indirect. In direct NRPE checks the Nagios server executes
check_nrpe which then connects to the NRPE daemon which is running on the client. The NRPE daemon then will
execute the command that was requested from the Nagios server. This command could execute a plugin locally as is
illustrated in the example. The command could also execute any script on the client whether it be a bash shell script, a
perl script or any other type of script.

The example also illustrates that NRPE can be used to execute Indirect Checks. This may be a situation where the
Nagios server has access to the client but not the web server that needs to be monitored. However, the client may have
access to the web server. So in this example the Nagios server executes a plugin or script on the client which in turn
monitors another box on the network, thus Nagios is indirectly monitoring the web server.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 109

Install the Daemon xinetd

A daemon, or service, is a background process that performs a specific function. Originally all daemons started at
boot time, running in the background and providing services when called upon. However, as the number of daemons
grew, the resources that these daemons took up increased and began slowing down the server so developers began
looking for a way to limit the number of daemons that run on a system. The daemon inetd was developed to help
manage other daemons by listening for requests in their behalf. When a request was received for a daemon managed
by inetd it was then notified and started up to respond. This has saved considerable resources on servers and it has
increased the interest in superdaemons and methods of reducing resources.

The xinetd superdaemon has replaced inetd on most Linux distributions today. xinetd has become more popular
because of security restrictions that can be placed on those who access the daemons managed by xinetd. xinetd also
provides better protection from denial of service attacks, better log management, and more flexibility. Both inetd and
xinetd only work with daemons that provide connections over a network.

How to Protect the NRPE Daemon

Server daemons must be protected to be effective. There is no perfect or complete option, but there are definite ways
to minimize the risk.

1. Limit Connections to Daemons

Connections to daemons can be limited by using several powerful tools. Iptables firewall is probably the most
flexible and powerful tool than an administrator has access to. However, it is at the same time the most
complex. Tcp_wrappers is a tool that is easy to use and works with most daemons to limit access to daemons
to specific subnets or IP Addresses.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 110

2. Limit the Number of Connections

Your network and hardware can only handle a limited number of connections safely. When connections push
your resources to the limit you will often see vulnerabilities appear that would not normally exist. When
resources begin to fail some options and security programs cannot function to their full extent.

You will need to install xinetd and make sure you have a file in /etc/xinetd.d called nrpe on the client and it looks like
this:

# default: off
# description: NRPE (Nagios Remote Plugin Executor)
service nrpe
{
flags = REUSE
type = UNLISTED
port = 5666
socket_type = stream
wait = no
user = nagios
group = nagios
server = /usr/sbin/nrpe
server_args = ­c /usr/local/nagios/etc/nrpe.cfg ­­inetd
log_on_failure += USERID
disable = no
only_from = 127.0.0.1 192.168.5.50
}

These are the two most important lines. By default all daemons monitored by xinetd are disabled so the default line
says “disable = yes”. The “only_from” line allows you to determine which machines can monitor this server using
NRPE, this is where you will enter the IP Address for the Nagios server as well as the localhost.

disable = no
only_from = 127.0.0.1 192.168.5.50

Edit /etc/services and add this line:

nrpe 5666/tcp # Nagios Remote Monitoring

Restart xinetd and view the log at /var/log/daemon.log

service xinetd restart

tail /var/log/daemon.log

Look for errors to correct.

Edit the /usr/local/nagios/etc/nrpe.cfg (RPM respository /etc/nagios/nrpe.cfg).

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 111

Change your allowed_hosts address to reflect the nagios monitoring server. You should also allow the localhost so
that you can do testing if necessary. If you are using xinetd then this does not have any impact.

allowed_hosts=127.0.0.1 192.168.5.180

The basic plugins that are running for you initially are these listed below. Note the path will be different if you
installed using the RPM repository.

command[check_users]=/usr/local/nagios/libexec/check_users ­w 5 ­c 10
command[check_load]=/usr/local/nagios/libexec/check_load ­w 15,10,5 ­c 30,25,20
command[check_hda1]=/usr/local/nagios/libexec/check_disk ­w 20 ­c 10 ­p /dev/hda1
command[check_zombie_procs]=/usr/local/nagios/libexec/check_procs ­w 5 ­c 10 ­s Z
command[check_total_procs]=/usr/local/nagios/libexec/check_procs ­w 150 ­c 200

Change ownership on the /usr/local/nagios/etc/nrpe.cfg (RPM repository /etc/nagios/nrpe.cfg) file so that nagios is
able to read the file.

chown nagios /usr/local/nagios/etc/nrpe.cfg

SSH Concepts
Generally there are three good reasons for using SSH to monitor remote Linux servers. First, of course the data
transmission and password authentication is always encrypted. This makes it an excellent choice when you are
located in a hostile environment. The second reason for using SSH is that you can employ the check_multi
command which allows you to check almost an unlimited number of services with one check. This is a powerful way
to save on bandwidth as you are only making one check for multiple services and the process is encrypted.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 112

The final reason you may consider SSH to the checking on the remote server is that you may not be able to install
applications or compile applications on the remote server.

In order to use SSH you need to use the check_by_ssh plugin which will enable you to perform local checks on the
remote machine.

Configure the Nagios Server

There are a number of steps that you must take to prepare the Nagios server to be able to use SSH without using a
password each time. This involves setting up a SSH keypair for the nagios user and providing the public key for the
nagios user on the remote boxes that will be monitored using SSH.

Using SSH to Check Services

Now you can set up individual checks using SSH. Here are several services listed in the services.cfg. Note several
differences with other services that you already have. The check_command is different showing that it is a SSH
command that will run the normal check. What this means is that you must edit the commands.cfg and create each one
of these SSH commands that you will use.

define service{
use generic­service
host_name class
service_description SSH Check Disk
check_command check_ssh_disk!10%!5%! /dev/mapper/VolGroup00­
LogVol00
}

Here is the listing in the commands.cfg. It illustrates how you must configure the command to use the SSH key so
that you do not need to use a password and it functions just like the check_disk but you must configure it to use SSH.
Note that this check is testing for the disk usage on this partition which is a logical volume. Make sure you understand
the description of your partition.

dev/mapper/VolGroup00­LogVol00

define command {
command_name check_ssh_disk
command_line $USER1$/check_by_ssh ­H $HOSTADDRESS$ ­i
/usr/local/nagios/etc/.ssh/id_dsa ­C "$USER1$/check_disk ­w $ARG1$ ­c $ARG2$ ­p
$ARG3$"
}

This service listing in services.cfg checks for system load.

define service{
use generic­service
host_name class
service_description SSH Check Load
check_command check_ssh_load!5.0,4.0,3.0!10.0,6.0,4.0
}

Here is the corresponding command that must be placed in commands.cfg.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 113

define command {
command_name check_ssh_load
command_line $USER1$/check_by_ssh ­H $HOSTADDRESS$ ­i
/usr/local/nagios/etc/.ssh/id_dsa ­C "$USER1$/check_load ­w $ARG1$ ­c $ARG2$"
}

Monitoring Windows
The Windows client NSClient++ can be used monitor a Windows machine with NSClient++ using the check_nt
command or using NRPE. Because the configuration for both aspects involves the NSClient++ they are viewed
together. The first step in setting up NRPE for Windows is to download a client for the Windows machine.

Download the NSCLient++ from http://sourceforge.net/projects/nscplus

This will provide a .zip file which you can unzip and it will provide the NSClient++­Win32­x.x.x folder.

NSClient++ Concepts
Login as the Administrator to the server.

Create a directory under the C:\ drive and download NSClient++ into that drive. Unzip the file and enter the directory
that was created.

Once you install you will have to make a note of the location of the install directory path.
Here is the contents of the directory.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 114

On the Windows machine place the path for the .exe file in the run command and install the program.

C:\NSClient++­Win32­0.3.8\NSClient++.exe /install

You will see a security warning but continue the install.

Now start the program, note your path may be different.

C:\NSClient++­Win32­0.3.8\NSClient++.exe /start
In order to stop the program use this command.

C:\NSClient++­Win32­0.3.8\NSClient++.exe /stop

You can test with:

C:\NSClient++­Win32­0.3.8\NSClient++.exe /test

If you make any changes to the configuration, stop the service and restart it.

Edit the NSC.ini file that is in the NSClient directory. Note that the file is divided by keywords placed in brackets.
First go to the [modules] section and edit the checks that you want to use. Uncomment the lines that you see below.
The FileLogger.dll will log the activities of the NSClient++. CheckDisk.dll will check for file size and hard disk use.
The CheckSystem.dll will check for memory, uptime, service stats and processes. You will also need to uncomment
the NSClientListener.dll and the NRPEListener.dll in order to communicate with Nagios.

In order to use some of the options available with NSClient++ you need to allow to additional features. There are
characters that need to be used with commands |`&<>'”\[]{} that you will want allow, “nasty_meta_chars”. The
“allow_arguments” will allow NRPE parameters to be passed along. Now there some security issues with enabling
this option so you need to consider that factor.

Go to the global section, [Settings], and be sure to limit the access to the Windows server that you are going to
monitor. Under the Allowed Hosts section enter the local host and any other connections that you want to enable.
These addresses will be separated by a comma.

allowed_hosts=127.0.0.1/32,192.168.5.50

In the Windows firewall open two ports, 5666 for NRPE and 12489 for NSClient++. Both are TCP ports. You can see
in the example how it should look when you review the firewall.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 115

Limit access to these ports to the Nagios server only.

Option Description
port=5666 port the NRPEListener.dll will listen to
command_timeout=60 maximum number of seconds that the NRPE daemon will allow plug­ins
to complete check
allow_arguments=1 allow clients to specify arguments to commands that are executed
allow_nasty_meta_chars=1 allow clients to specify nasty ( |`&><'"\[]{}) characters
use_ssl=0 boolean option to use a SSL connection
bind_to_address= bind to specific address, blank means all addresses
allowed_hosts=192.168.1.1 Security option to limit access to Windows server and NRPE
script_dir=scripts\ Files in this directory become check commands, has security implications
socket_timeout=30 Socket time to close incoming connections

The NRPE Handlers represent the actual commands that will be used.

Unfortunately this section is deprecated and it is not recommended to use these settings.

[NRPE Client Handlers]

command[check_users]=/usr/local/nagios/libexec/check_users ­w 5 ­c 10
check_disk1=/usr/local/nagios/libexec/check_disk ­w 5 ­c 10
check_disk_c=inject CheckFileSize ShowAll MaxWarn=1024M MaxCrit=4096M

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 116

Once you have the remote host set up you will need to set up the Nagios monitoring server. First install the nrpe
plugin.

MSSQL
Monitoring a Windows SQL server is an aspect that many Nagios administrators will need to do. One of the better
plugins for this is check_mssql_health. This plugin provides health related checks that are useful in understanding the
status of a SQL server.

Service Definitions
In this example the host definition is listed with service definitions which may be the case if you are only checking one
server. These all use the port 1433 so one check is designed to monitor the port itself with a tcp check.

define host{
use windows­server
host_name win2008
alias Windows Server
address 184.106.77.67
}
define service{
use generic­service
host_name win2008
service_description SQL Connectivity 1433
check_command check_tcp!1433
}
define service{
use generic­service
host_name win2008
service_description SQL io­busy
check_command check_mssql _health!io­busy
}
define service{
use generic­service
host_name win2008
service_description SQL Backup of model
check_command check_mssql _health!database­backup­age
­­name model
}
define service{
use generic­service
host_name win2008
service_description SQL Page Life
check_command check_mssql _health!page­life­expectancy
}
define service{
use generic­service

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 117

host_name win2008
service_description SQL Locks
check_command check_mssql _health!list­locks
}
define service{
use generic­service
host_name win2008
service_description SQL Datafiles
check_command check_mssql _health!list­datafiles
}
define service{
use generic­service
host_name win2008
service_description SQL Databases
check_command check_mssql _health!list­databases
}
define service{
use generic­service
host_name win2008
service_description SQL Connected Users
check_command check_mssql _health!connected­users
}

It is best practice to test from the command line first. Be sure the Windows server firewall is open on port 1433 for the
Nagios server only as the username and password are plain text.

Log Monitoring
The basic plugin for analyzing logs is check_log. The check_log will check a current log for a text string and after the
check save the information in a temporary log so that it can compare changes. This plugin has two settings; 0 for
“OK” and 2 for “Critical”. Here is an example check.

check_log ­F /var/log/messages ­O /tmp/check_log.fail ­q "Warning"

The (­F) will point to the log you want to monitor. The (­O) refers to the “old log” allowing the system to compare.
The (­q “text_string) is what you will be searching for in the log.

Here is a return from that command which shows that it found 2 instances:

/usr/local/nagios/libexec/./check_log ­F /var/log/messages ­O /tmp/check_log.fail

­q "Warning"
(2) < Apr 28 14:28:03 fw3 nagios: Warning: Return code of ­1 for check of service
'Traffic Load OUT' on host 'cisco2500' was out of bounds.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 118

Monitor Nagios Logs

In this example the commands that are defined will be used for a local check of the Nagios logs. Each one is defined
separately as you will actually need different commands for each text string you are looking for.

Edit /etc/nagios/commands.cfg

define command{
command_name check_log_warning
command_line $USER1$/check_log ­F /var/log/messages ­O /tmp/check_log.fail
­q "Warning"
}
define command{
command_name check_log_fail
command_line $USER1$/check_log ­F /var/log/messages ­O /tmp/check_log.fail
­q "Fail"
}

Service Definition
Create a service definition for each command you will use.

define service{
use local­service
host_name localhost
service_description Check Log
check_command check_log_fail
}

Nagios Server Logs

You should see examples in the logs that demonstrate the check is working. You can see the process of a check as it is
exhibited step by step in the log.

Apr 26 15:01:16 fw3 xinetd[2068]: START: nsca pid=4334 from=192.168.5.91

Apr 26 15:01:16 fw3 xinetd[2068]: EXIT: nsca status=0 pid=4334 duration=0(sec)
Apr 26 15:01:16 fw3 nagios: EXTERNAL COMMAND:
PROCESS_SERVICE_CHECK_RESULT;passnag;Log;0;Log check data initialized...
Apr 26 15:01:17 fw3 nagios: PASSIVE SERVICE CHECK: passnag;Log;0;Log check data initialized...
Apr 26 15:02:12 fw3 xinetd[2068]: START: nsca pid=4418 from=192.168.5.91
Apr 26 15:02:12 fw3 xinetd[2068]: EXIT: nsca status=0 pid=4418 duration=0(sec)
Apr 26 15:02:12 fw3 nagios: EXTERNAL COMMAND:
PROCESS_SERVICE_CHECK_RESULT;passnag;Log;0;Log check ok ­ 0 pattern matches found

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 119

Network Printers
Nagios allows you to monitor network printers so that you will easily be able to verify basic information about your
network printers. This will allow you to check if the printer is up and basic toner states. Why use Nagios for printers
if you can log into a network printer? It provides one central location for checking on servers, printers, routers, etc.
And it will notify according to your settings.

Edit the /usr/local/nagios/etc/nagios.cfg (RPM repository /etc/nagios/nagios.cfg) and uncomment the printer line.

#cfg_file=/usr/local/nagios/etc/objects/printer.cfg

Create the file for printers

Modify the template and enter the information for the printer that you want to
use. The information here is for an HP 4300dn printer.

###############################################################################
#
# HOST DEFINITIONS
#
##############################################################################

define host{
name generic­printer
use generic­host
check_period 24x7
check_interval 5
retry_interval 1
max_check_attempts 10
check_command check­host­alive
notification_period workhours
notification_interval 30
notification_options d,r
contact_groups admins
register 0 ; JUST A TEMPLATE
}

# Define a host for the printer we'll be monitoring

# Change the host_name, alias, and address to fit your situation

define host{
use generic­printer
host_name hp4300
alias HP LaserJet 4300dn
address 192.168.5.11
hostgroups network­printers
}

###############################################################################

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 120

#
# HOST GROUP DEFINITIONS
#
##############################################################################

# A hostgroup for network printers

define hostgroup{
hostgroup_name network­printers ; The name of the hostgroup
alias Network Printers ; Long name of the group
}

###############################################################################
#
# SERVICE DEFINITIONS
#
##############################################################################

# Create a service for monitoring the status of the printer

# Change the host_name to match the name of the host you defined above
# If the printer has an SNMP community string other than "public", change the #
check_command directive to reflect that

define service{
use generic­service ; from a template
host_name hp4300 ; host
service_description Printer Status ; service description
check_command check_hpjd!­C public ; command
normal_check_interval 10 ; Check 10 minutes
retry_check_interval 1 ; Re­check
}

# Create a service for "pinging" the printer occasionally. Useful for

# monitoring RTA, packet loss, etc.

define service{
use generic­service
host_name hp4300
service_description PING
check_command check_ping!3000.0,80%!5000.0,100%
normal_check_interval 10
retry_check_interval 1
}

Once you have created the printer.cfg file be sure to restart nagios.

service nagios restart

If you get errors you will need to fix them.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 121

The services to check for the printer are status and toner using the check_hpjd. This is a very common program that
is easy to use. There are other printer checking programs that you can find at http://www.nagiosexchange.org.

Here you can see the Ping status and the printer status for the HP 4300.

If you select the Printer Status in the web interface you will see additional information about the printer.

Checking Printers with SNMP

The check_snmp_printer_args.sh plugin is an excellent choice if you want generic options using SNMP (Simple
Network Management Protocol). In most cases this will provide all of the information that you need.

Download the plugin SNMP Printer Check:

http://exchange.nagios.org/directory/Plugins/Hardware/Printers/SNMP­Printer­Check/details

Here is an example of the output that is provided clearly indicating toner level, page count, maintenance kit level and
tray status.

Preparation
The script was written on a Windows machine so you will need to convert it. Install these two programs, bc is used by
the script.

yum install ­y dos2unix bc

Convert the program with:

dos2unix check_snmp_printer_args.sh

Make it executable and change ownership:

chmod 755 check_snmp_printer_args.sh

chown nagios:nagios check_snmp_printer_args.sh

Define the command in the commands.cfg

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 122

define command{
command_name check_snmp_printer_args
command_line $USER1$/check_snmp_printer_args.sh ­H $HOSTADDRESS$ $ARG1$
$ARG2$
}

Create service definitions based on options that are available.

CONSUM {<string> | TEST | ALL}

TEST will give you the exact names of all available consumables
CONSUMX <string>
TEST will give you the #'s of all trays available
DISPLAY Report contents of printer display
DEVICES Status of hardware modules
MESSAGES Event logs reported by the printer
MODEL ALL will give you all tray output at once.
PAGECOUNT How many pages this printer has processed (culmulative)
STATUS Overall status of the printer
TRAY {<number> | TEST | ALL} <number> will give you output for the
specified tray. A comma­separated list of values is possible as well.

Here you can see that two network printers are monitored each are HP, one is color and one is not. They both have
been easily detected and the plugin provides excellent information.

define service{
use generic­service
host_name hp4300
service_description Printer Consumables
check_command check_snmp_printer_args!­C public!­x "CONSUM ALL"
normal_check_interval 10
retry_check_interval 1
}
define service{
use generic­service
host_name hp4300
service_description Printer Tray 2
check_command check_snmp_printer_args!­C public!­x "TRAY 2"

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 123

normal_check_interval 10
retry_check_interval 1
}
define service{
use generic­service
host_name hp4300
service_description Printer Model
check_command check_snmp_printer_args!­C public!­x MODEL
normal_check_interval 10
retry_check_interval 1
}
define service{
use generic­service
host_name hp4300
service_description Printed Pages
check_command check_snmp_printer_args!­C public!­x PAGECOUNT
normal_check_interval 10
retry_check_interval 1
}
define service{
use generic­service
host_name hp4300
service_description Printer Messages
check_command check_snmp_printer_args!­C public!­x MESSAGES
normal_check_interval 10
retry_check_interval 1
}

Here is an example of an empty tray which sends a CRITICAL state.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 6: Monitoring 124

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 125

Chapter 7: Practical Exercises

The best way to learn anything with Nagios is to perform the task in a step­by­step manner so an understanding of the
context and options of the situation help build a higher level of understanding. The Nagios Certified Professional will
typically be engaged in recognizing and responding to problems discovered by Nagios as well as be responsible to
create reports and locate decision making data about the network and devices on the network. Therefore, these
exercises are designed to not only aid in preparing for the Nagios Certified Professional certification but also as a
exercise for those working at a helpdesk situation.

Exercise #1: Login and Research

Management has requested that you log into the Nagios interface and provide for them research on a plugin that could
be used to check for updates to a Linux server. The Linux servers are all running CentOS and management wants to
make sure that updates are a priority for those machines. Several other tasks were requested as well:

* verify that Nagios is the newest version

* recommend several options for plugins to check on updates to a CentOS system
* research the forum for ideas and problems related to the plugin choices

Login to Nagios
Point your browser to the IP Address of the Nagios server followed by “nagios”. Enter your username and password.
Here is an example of the location in the browser.

Locate the installed version and then click the highlighted link to verify the latest version is installed.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 126

Once you have checked for to verify that you are using the latest version, then return to the “Home” page and review
the links under “Get Started”, “Quick Links”, “Don't Miss” and “latest News”. One of the tasks requested was
research for a plugin so on the menu select “Nagios Exchange” under “Quick Links”.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 127

Follow the link to http://exchange.nagios.org/ and then select plugins to begin your search.

Once you get to the next page select “Operating Systems” and then “Linux” and start reviewing the plugins for
checking for updates. You will eventually get to a plugin that is highly rated, note the stars and easy to implement.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 128

From all indications this would be the plugin to recommend. Before you make that decision return to the “Home” for
Nagios and again use the menu to select the forum that is available for help.

Once you get to the “Support Resources” page select “General Support Forum” as your company has no t purchased a
support package.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 129

Once at the support forum choose the “Nagios Core” forum and then search to see if there are any problems using the
check_yum plugin that you located.

These steps allow you to fulfill the request made by management.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 130

Exercise #2: Responding to Problems

You have been notified of two problems on the network. The first problem is that a host is down, however you have
since been notified that someone is working on that problem. The second problem is that a service is flapping and
you need to disable the flap detection for that service. Third, you were told that a service is now up but you want to
reschedule a check to be performed immediately to verify.

Proceed to the “home” page and on the menu select “Hosts(unhandled)” as this will list the hosts that are not currently
responding to connections from Nagios.

Here you can see the “cisco827” is down. Click on the host.

Once you have selected the host, now select “Acknowledge this host problem” as you want to stop notifications and
communicate to the rest of those watching Nagios.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 131

Now you should enter the information that you want to relate to other administrators. In this case the IOS is being
updated so the router is down.

Once you hit “Commit” this information will now be made available on the web site so others know the problem is
taken is being handled. Now when you access the host you can see the check mark indicating someone has

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 132

acknowledged the problem and the comment icon tells them there is a text comment telling them more information.

Now you need to access a service that is stable currently but has a history of flapping. Management has requested that
you turn flapping off for this service. Choose the service and select “Disable flap detections for this service”.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 133

Now when you access the service the flap detection is turned off and it is listed on the service so other administrators
know as well. It can be turned back on at any time.

The final task is to recheck a service to make sure it is up. The first thing you do is check the scheduling to see where
the check is. Proceed to the menu and “System” and then “scheduling Queue”. As you review the queue you see that
your check will not happen for some time and you do not want to wait for it.

Since your check will not come up for a while, go to the menu and select the service you want to verify. Once you
open the service select “Re­schedule the next check of this service”.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 134

Once you click it you will need to “Commit”. Note, you could set the time specifically. The force option will check
immediately.

That completes all of the tasks in this exercise.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 135

Exercise #3: Reports

Management has requested a number of reports to verify activity on the network. Here is the list of reports that they
would like to see:

* overview of the entire system and how it is running currently

* summary of all host and service alerts over the last week
* uptime of a specific host as it is mission critical
* a graph of alerts for a specific host
* latency report for hosts and services

The organization has a power outage and once the power is back on the management asks you to show them the
current status of the network. Proceed to the menu and “Current Status” and “Tactical Overview”.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 136

This snapshot of the network not only shows the status of hosts and services but also the features that you are using on
the network, like flapping, notifications, eventhandlers, etc. Notice the “Monitoring Performance” provides latency
information, both for services and hosts.

In order to get a different picture on latency open up “System” and “Performance Info”.

The advantage is that there is more information available and it is easier to understand. For example, the “Tactical

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 137

Overview” provides these metrics for service latency:

0.00/37.57/0.449 sec.

Of course this is minimum/maximum/average. So it will provide a quick view of latency. However, the “Performance
Info” page provides that information as well as check time and percentage of change. The percentage of change can
be a way to predict problems before they actually occur.

The next aspect that management wants to see are all alerts for hosts and services over the last week. When you click
on “Reports” and “Alert Summary” you can set the time range for the list. In fact, there are a lot of options that can be
created for exactly the type of report needed.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 138

The report itself can be helpful not only in recognition of problems that have occurred but it can also be useful in
predicting future problems.

Management would also like to have a look at a graph showing the alerts for a specific host that is a mission critical
host. Once clicking on “Reports/Alerts Histogram” you can make the decision about using the report for a host or a
dervice.

Select the host from the list on the next screen and then select the time frame that you want to see.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 139

This report demonstrates a history of the problems related to this host. It shows the time it was down and the recovery
state.

Now management needs to see uptime on a host, in other words how available was this host for service over the last 30

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 140

days. Proceed to “Reports” and “Availability” and then select the host option.

Select the host from your list of hosts.

Once the host is selected choose the options that you want to see, in this example the availability of the host for the last
31 days is selected.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 141

The report provides a graph at the top which is green/red based on availability and then provides detailed information
below that. The availability of services are also listed on the chart.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 142

This completes all of the tasks for this exercise.

Exercise #4: Passive vs. Active Checks

The goal in this exercise is to recognize the difference in viewing active checks versus viewing passive checks because
they must be interpreted correctly. Your organization has a lot of active checks but it also has some passive checks and
you need to be able to understand the differences. First, remember that active checks are when Nagios takes the
initiative to execute the timetable for the check while passive checks are implemented solely on the client. Here is an
example of both active and passive checks. The active checks are NRPE, NTP and Nagios DNS. These three checks
are initiated by the Nagios server and the data is returned to Nagios.

You can also see the passive checks in two different situations. First, recognize that the “?” is the icon used in this
frontend, exfoliation (the default for Nagios Core), to indicate it is a passive check. So the first thing you know is all

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108
Chapter 7: Practical Exercises 143

passive checks will have the icon representing the fact that is is passive. Those passive checks listed which are in a
WARNIGN state also indicate that a check has not been received from the client for over an hour. These three have an
additional script tied to them which will provide a WARNING if the client has not sent a check for that service.
Again, the importance here is that the client initiates the check so that if for any reason the client does not send a
check the state of the passive check WILL NOT CHANGE. This is critical to understand and that is why the check
has a time limit set so the administrator understands that the check has not been received.

The other three passive checks which are “PENDING” indicate that a check has never been received. Again, if the
client does not send the checks, Nagios cannot effectively monitor that client.

Here is an example of one active check, SNMP Memory Usage, and three passive checks, again note the “?”. Here the
story is different in that the client has sent passive check information for these checks to Nagios.

In any evaluation of the checks that are being used on the network, the administrator must first discern if the check is
active or passive and then determine if the passive check is actually current. Discipline yourself to verify that the
passive check is current or you can incorrectly make the assumption about the real state of the passive check.

Copyright by Nagios Enterprises, LLC

Cannot be reproduced without written permission. P.O. Box 8154, Saint Paul, MN 55108