Worker Cookbook
- A Simple Plugin Example
- A More Realistic Plugin Example
- Share Data between Stages
- Override one Stage of an Action
- Add to one Stage of an Action
- Run code before an Action
- Run code at the end of an Action
- Act on data before it hits the database
- How does Worker Priority work?
- The Execution Order of Plugin Workers
In your deployment.yml:
site_local_files: true
extra_worker_plugins: ['X::Demo']The following file would live at:
/home/netdisco/nd-site-local/lib/App/NetdiscoX/Worker/Plugin/Demo.pmpackage App::NetdiscoX::Worker::Plugin::Demo;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
register_worker({ phase => 'main' }, sub {
print "I am a worker.\n";
return Status->done("I have completed my work!");
});
true;Then you can run:
$ ND2_LOG_PLUGINS=1 ~/bin/netdisco-do demo -DWhich results in something like:
[33131] 2017-11-30 11:05:37 info App::Netdisco version 2.036012_003 loaded.
[33131] 2017-11-30 11:05:37 info demo: started at Thu Nov 30 11:05:37 2017
[33131] 2017-11-30 11:05:37 debug loading worker plugin App::NetdiscoX::Worker::Plugin::Demo
[33131] 2017-11-30 11:05:37 debug => running workers for phase: main
[33131] 2017-11-30 11:05:37 debug -> run worker main/_base_/0
I am a worker.
[33131] 2017-11-30 11:05:37 info demo: finished at Thu Nov 30 11:05:37 2017
[33131] 2017-11-30 11:05:37 info demo: status done: I have completed my work!package App::NetdiscoX::Worker::Plugin::Discover::Neighbors::Routed;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use App::Netdisco::Transport::SNMP;
use aliased 'App::Netdisco::Worker::Status';
use App::Netdisco::Util::Device qw/get_device is_discoverable/;
use App::Netdisco::JobQueue 'jq_insert';
register_worker({ phase => 'main', driver => 'snmp' }, sub {
my ($job, $workerconf) = @_;
my $device = $job->device;
return unless $device->in_storage and $device->has_layer(3);
my $snmp = App::Netdisco::Transport::SNMP->reader_for($device)
or return Status->defer("discover failed: could not SNMP connect to $device");
my $ospf_peers = $snmp->ospf_peers || {};
return Status->info(" [$device] neigh - no OSPF peers")
unless (scalar values %$ospf_peers);
my $count = 0;
foreach my $ip (values %$ospf_peers) {
my $peer = get_device($ip);
next if $peer->in_storage or not is_discoverable($peer);
next if vars->{'queued'}->{$ip};
jq_insert({
device => $ip,
action => 'discover',
subaction => 'with-nodes',
});
$count++;
vars->{'queued'}->{$ip} += 1;
debug sprintf ' [%s] queue - queued %s for discovery (peer)', $device, $ip;
}
return Status->info(" [$device] neigh - $count peers added to queue.");
});
true;Use the vars stash variable that
Dancer provides. This is a HashRef which
can store any data you like for the lifetime of the complete action. It becomes
available when you use Dancer ':syntax', and will not persist between actions.
For example:
package App::Netdisco::Worker::Plugin::Vlan;
vars->{'port'} = get_port($job->device, $job->port)or, as in the plugin example above:
package App::Netdisco::Worker::Plugin::Discover::Neighbors;
vars->{'queued'}->{$ip} = true;
# and then
package App::Netdisco::Worker::Plugin::Discover::Neighbors::Routed;
next if vars->{'queued'}->{$ip};You need to:
-
Use the same stage namespace or a child namespace
-
Use the same phase in the sequence
-
Set a higher priority
-
Return Status
done()
So for example to override the Arpnip::Nodes stage, your package must be
named App::NetdiscoX::Worker::Plugin::Arpnip::Nodes (or a child namespace).
Your worker config hash must have a higher priority, either using the
priority key or a driver key mapping to a higher priority. Be sure to run
at the same phase. Return Status→done() from your worker.
package App::NetdiscoX::Worker::Plugin::Arpnip::Nodes::MyOverride;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
register_worker({ phase => 'main', priority => 1000 }, sub {
return Status->done("I have completed my work!");
});
true;You need to:
-
Use the same stage namespace or a child namespace
-
Use the
userphase -
Optionally pin to a specific
drivertype (eg snmp) -
Return Status
info()
package App::NetdiscoX::Worker::Plugin::Arpnip::Nodes::MyExtra;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
register_worker({ phase => 'user' }, sub {
return Status->info("I have completed my work!");
});
true;You need to:
-
Use the same stage namespace or a child namespace
-
Use the
earlyphase -
Optionally pin to a specific
drivertype (eg snmp) -
Return Status
info()
package App::NetdiscoX::Worker::Plugin::Arpnip::MyInjection;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
register_worker({ phase => 'early' }, sub {
return Status->info("I will be run just before an arpnip!");
});
true;You need to:
-
Use the same stage namespace or a child namespace
-
Use the
latephase -
Optionally pin to a specific
drivertype (eg snmp) -
Return Status probably
info()
package App::NetdiscoX::Worker::Plugin::Arpnip:MyAddon;
use Dancer ':syntax';
use App::Netdisco::Worker::Plugin;
use aliased 'App::Netdisco::Worker::Status';
register_worker({ phase => 'late' }, sub {
return Status->info("I will be run after an arpnip!");
});
true;This is a special case and only currently available in the arpnip::nodes
action, which caches node data in vars and uses the store phase to write
to the database.
This allows you to intercept the data in vars during the user phase, which
occurs between the main and store phases.
You can simply read the cache, or even edit it, so long as you maintain the same data structure. Ask the Netdisco devs if you’d like this feature for any other actions.
As you can see in the examples in this document, priority may be set
explicitly using the priority key or implicitly using the driver key, or
not at all (which is actually zero). Workers are run in order from highest to
lowest priority and stop running when one priority level happens to return
Status done().
The mapping of driver to priority level is:
restconf: 500
netconf: 400
eapi: 300
cli: 200
snmp: 100Therefore setting a priority such as 1000 would cause a worker to override any built-in driver.
There are several dimensions to this: action, stage, phase, driver, and priority.
Action is the scheduled job or netdisco-do command, and is taken from the
Perl Package namespace in which the worker is registered, being the component
following Plugin:: in the namespace.
Stage is the Perl Package namespace component(s) following the Action. For
example the Arpnip::Nodes worker is stage nodes. Child packages of this
namespace get folded up into the same stage (there are no sub-stages).
Phase is one of check, early, main, user, store, or late (in that
order). Not all phases are used in all workers.
Driver and Priority are effectively the same - the driver being a shorthand for a given priority number (as documented above). Workers are run in descending order of priority from highest to lowest.
Here is the pseudocode runtime order:
-
For each Phase in order:
check,early,main,user,store, andlate-
Run each Stage (in asciibetical order), starting with the "root" action namespace
-
Run workers from highest priority to lowest
-
Abort if there are
checkworkers but none has returned Statusdone() -
Stop running when the previous higher priority level had a worker return Status
done()
-
-
-
This last line can be a little tricky to understand. Let’s say there are
workers available for each of the netconf, cli, and snmp drivers; that would
be priorities 400, 200, and 100 in practice. If the user has configured
device_auth settings for all drivers then Netdisco runs the priority 400
workers (netconf) first, and if none has returned done() runs the priority
200 workers (cli) next, and if none has returned done() runs the priority
100 workers (snmp). The result of the job overall is the best status across
all workers, which may be done, info, defer, or error.