OpenNMS 101
Tarus Balog
tarus@opennms.org
http://www.opennms.org/Training
© 2017 The OpenNMS Group, Inc.
Module 5: Alarms
© 2017 The OpenNMS Group, Inc.
Alarms Implement Workflow
● Extremely powerful system for
● Reducing the number of duplicate events
● Performing automated actions to manage and
control alarms
● Performing general database activities
● Modeled on systems costing hundreds of
thousands of dollars
© 2017 The OpenNMS Group, Inc.
Event Reduction
Send a uei.class event three times
It gets reduced to one alarm with a count of 3
© 2017 The OpenNMS Group, Inc.
Event Reduction
© 2017 The OpenNMS Group, Inc.
Create an Alarm from an Event
<event>
<uei>uei.opennms.org/default/event</uei>
<event-label>OpenNMS-defined default event: event</event-label>
<descr><p>An event with no matching configuration was received
from interface %interface%. This event included the
following parameters:
%parm[all]%</p></descr>
<logmsg dest="logndisplay">An event with no matching configuration
was received from interface %interface%.</logmsg>
<severity>Indeterminate</severity>
<alarm-data reduction-key="%uei%:%dpname%:%nodeid%:%interface%"
alarm-type="3"/>
</event>
© 2017 The OpenNMS Group, Inc.
<alarm-data> Elements
● reduction-key: a string that uniquely identifies
the event. Can use the same %variables% as events
and notices.
● alarm-type: an optional value used in
automations.
1) Problem
2)Resolution
3)Unknown
● clear-key: an optional value to match an up
alarm with the reduction-key of a down alarm.
© 2017 The OpenNMS Group, Inc.
Automations
● Automations allow actions to be done to the
database, mainly focus on the alarms table.
● For example, the “cosmicClear” automation
matches “up” events with “down” events and
clears the down event.
● As a demonstration, let’s add the HTTP service to
our localhost node and see what happens.
© 2017 The OpenNMS Group, Inc.
Install a Web Server
● Run
yum install httpd
● Start the web server
systemctl start httpd
● Rescan “localhost”
© 2017 The OpenNMS Group, Inc.
Automation Demonstration
● Navigate to Status → Alarms → Alarm Summary
● Run
systemctl stop httpd
● Refresh until the alarm appears
© 2017 The OpenNMS Group, Inc.
Resolve the Alarm
● Run
systemctl start httpd
● Refresh until the up alarm appears
© 2017 The OpenNMS Group, Inc.
So, How’d That Work?
● Automations are configured in the vacuumd-
configuration.xml file
●
An automation consists of:
●
A trigger (optional)
● An action
● An event to create (optional)
© 2017 The OpenNMS Group, Inc.
Some Automations
<automation name="cosmicClear" interval="30000" active="true"
trigger-name="selectResolvers"
action-name="clearProblems" />
<automation name="cleanUp" interval="60000" active="true"
action-name="deletePastClearedAlarms" />
<automation name="escalation" interval="30000" active="false"
trigger-name="selectSuspectAlarms"
action-name="escalateAlarm"
action-event="eventEscalated" />
© 2017 The OpenNMS Group, Inc.
cosmicClear Trigger
<automation name="cosmicClear" interval="30000" active="true"
trigger-name="selectResolvers"
action-name="clearProblems" />
<!-- Find all alarms that potentially clear problems -->
<trigger name="selectResolvers" operator=">=" row-count="1" >
<statement>
SELECT *, now() AS _ts
FROM alarms
WHERE alarmType=2
</statement>
</trigger>
© 2017 The OpenNMS Group, Inc.
cosmicClear Action
<automation name="cosmicClear" interval="30000" active="true"
trigger-name="selectResolvers"
action-name="clearProblems" />
<!-- New and optimized version of clearing problems -->
<action name="clearProblems" >
<statement>
UPDATE alarms
SET severity=2, firstautomationtime =
COALESCE(firstautomationtime, ${_ts}), lastautomationtime = ${_ts}
WHERE alarmType=1
AND severity > 2
AND lastEventTime <= ${lastEventTime}
AND reductionKey = ${clearKey}
</statement>
</action>
© 2017 The OpenNMS Group, Inc.
cleanUp Action
<automation name="cleanUp" interval="60000" active="true"
action-name="deletePastClearedAlarms" />
<!-- ticket state of 3 is closed -->
<action name="deletePastClearedAlarms" >
<statement>
DELETE FROM alarms
WHERE severity <= 3
AND COALESCE(lastautomationtime, lasteventtime) < now()
- interval '5 minutes'
AND (alarmacktime IS NULL AND (tticketState IS NULL OR
tticketState = 3))
</statement>
</action>
© 2017 The OpenNMS Group, Inc.
Adding Notes to Alarms
You can add notes to alarms: Sticky and Journal
© 2017 The OpenNMS Group, Inc.
Sticky Notes
Sticky notes are like Post-Its™ - tied to a particular alarm
© 2017 The OpenNMS Group, Inc.
Journal Notes
Journal notes are tied to all alarms with a particular
reduction-key
© 2017 The OpenNMS Group, Inc.
Exercise #5: Make New Alarms
● Edit the Class.events.xml file
● Add alarm data to the “unhappiness” event and set it
as Type 1.
● Add alarm data to the “happiness” event and set it to
Type 2 and have it resolve the “unhappiness” event
● Reload the event configuration
● Test
© 2017 The OpenNMS Group, Inc.
Get the new Class.events.xml file and Reload
# wget -N https://www.opennms.org/Training/Config/Exercise%205/Class.events.xml
--2016-12-05 04:02:45-- https://www.opennms.org/~tarus/Class/Config/Exercise
%205/Class.events.xml
Resolving www.opennms.org (www.opennms.org)... 104.131.77.90,
2604:a880:800:10::1b1a:7001
Connecting to www.opennms.org (www.opennms.org)|104.131.77.90|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1438 (1.4K) [application/xml]
Saving to: ‘Class.events.xml’
100%[=============================================>] 1,438 --.-K/s in 0s
2016-12-05 04:02:45 (84.0 MB/s) - ‘Class.events.xml’ saved [1438/1438]
# send-event.pl uei.opennms.org/internal/reloadDaemonConfig -p 'daemonName Eventd'
© 2017 The OpenNMS Group, Inc.
unhappiness Event with Alarm Data
<event>
<uei>uei.opennms.org/class/unhappiness</uei>
<event-label>OpenNMS defined event: The OpenNMS Class is not happy</event-label>
<descr>
<p>This event is sent when the OpenNMS Class is unhappy.</p>
<ul>
<li>I can't get no</li>
<li>I can't get no</li>
<li>I can't get no</li>
<li>Satisfaction</li>
</ul>
</descr>
<logmsg dest='logndisplay'>
<p>OpenNMS Class is NOT Happy! </p>
</logmsg>
<severity>Major</severity>
<alarm-data reduction-key="%uei%"
alarm-type="1"
auto-clean="false"/>
</event>
© 2017 The OpenNMS Group, Inc.
happiness Event with Alarm Data
<event>
<uei>uei.opennms.org/class/happiness</uei>
<event-label>OpenNMS defined event: The OpenNMS Class is so happy</event-label>
<descr>
<p>This event is sent when the OpenNMS Class is happy.</p>
<ul>
<li>Dance, Everybody Dance!</li>
<li>Life is Good!</li>
<li>This is Fun!</li>
</ul>
</descr>
<logmsg dest='logndisplay'>
<p>OpenNMS Class is Happy! </p>
</logmsg>
<severity>Normal</severity>
<alarm-data reduction-key="%uei%"
alarm-type="2"
clear-key="uei.opennms.org/class/unhappiness"
auto-clean="false"/>
</event>
© 2017 The OpenNMS Group, Inc.
Test the Automation
Send the “down” event and watch what happens:
send-event.pl uei.opennms.org/class/unhappiness
Then send the “up” event and watch what happens:
send-event.pl uei.opennms.org/class/happiness
© 2017 The OpenNMS Group, Inc.
