JKFlow

JKFlow is an XML-configurable report module for FlowScan that analyzes NetFlow data exported from Cisco (and other) routers. It lets you isolate parts of your network into directions — defined by source and destination subnets, sites, or router/interface groups — and measure traffic volume, protocols, services, and quality-of-service parameters for each of them.

Homepage: https://jkflow.sourceforge.net/ SourceForge project: https://sourceforge.net/projects/jkflow/

• XML-driven configuration — no code changes needed to add or modify monitored traffic segments.
• Directions — recursive, nested source/destination subnet (or AS, or router/interface) definitions, each with its own protocol, service, and QoS monitoring.
• Sites — named groups of subnets so you can reference Belgium, England, etc. instead of repeating subnet lists.
• Definesets — reusable configuration templates (<set>) that can be shared across multiple directions to avoid duplicating monitoring definitions.
• Fast subnet matching using the C-coded Net::Patricia module, plus hash-based lookups for protocols and services, so lookup time stays flat regardless of how many directions or entries are configured.
• Autogenerated, per-direction evaluation code — JKFlow only generates the counting logic each direction actually needs, avoiding the performance cost of unused features.
• RRDTool integration — per-direction round-robin database (RRD) files for graphing traffic over time.
• Scoreboarding — IP-address and port-based "top talkers" tracking, with HTML report generation (including aggregated scoreboards as of 3.5.x).
• Router/interface groups, in addition to plain subnet-based directions, for monitoring by router or interface.
• Application grouping — combine several services under one label (e.g. web = http + https, mail = pop + imap + smtp).
• Special traffic classes — stateful FTP session tracking, multicast, ToS/DSCP breakdowns, and catch-alls for unmatched services/protocols.
• Ships with JKGrapher, a companion CGI script that renders the collected RRD data as traffic graphs (see the Flash demo).

The current stable release is 3.5.2, which added aggregated scoreboarding. Versions 1 and 2 are deprecated — use 3.x. Experimental SMP-enabled versions exist but are known to have synchronization issues. You may also be interested in JKFlow-python, a standalone python conversion of JKFlow with support for nfdump/nfcapd.

• Perl 5.8+
• FlowScan 1.006+, plus a flow collector such as cflowd or flow-capture
• RRDTool 1.0.42+ (and its bundled RRDs Perl module)
• The following CPAN Perl modules:
  • Net::Patricia (>= 1.010) — fast subnet lookups
  • XML::Simple — XML configuration parsing
  • HTML::Table — Top Talkers / scoreboard HTML reports
  • Boulder::Stream (1.30)
  • ConfigReader::DirectiveStyle (0.5)
  • Cflow (1.051) — must match your flow collector (cflowd vs. flow-tools)
• A web server (for JKGrapher.pl, run as a CGI script) if you want browsable HTML/graph reports

1. Install the required Perl modules, e.g. via CPAN:

   perl -MCPAN -e shell
   cpan> install Boulder::Stream
   cpan> install Net::Patricia
   cpan> install ConfigReader::DirectiveStyle
   cpan> install Cflow
   cpan> install HTML::Table

   ConfigReader::DirectiveStyle sometimes installs its files flat rather than into a ConfigReader/ subdirectory. If that happens, create the directory and move the files in yourself:

   mkdir /usr/lib/perl5/site_perl/5.8.0/ConfigReader
   mv DirectiveStyle.pm Values.pm Spec.pm ConfigReader/

   (Adjust the path for your Perl's actual site_perl location and version.)

2. Place the files:

   • JKFlow.pm and JKFlow.xml → /usr/local/bin (or wherever your FlowScan installation expects report modules; update the paths inside JKFlow.pm if you install elsewhere)
   • JKGrapher.pl → your web server's cgi-bin directory (e.g. /var/www/cgi-bin)

3. Point FlowScan at JKFlow in flowscan.cf:

   FlowFileGlob /var/flows/flows/flows.*:*[0-9]
   ReportClasses JKFlow
   WaitSeconds 30
   

4. On SELinux-enabled systems (Fedora/RHEL/CentOS), make sure the CGI script is allowed to run (temporarily setenforce 0 to test, then configure the appropriate booleans/contexts for production).

JKFlow is configured entirely through an XML file. The root elements are:

<config>
  <definesets>...</definesets>
  <sites>...</sites>
  <routergroups>...</routergroups>
  <all>...</all>
  <directions>...</directions>
  <rrddir>/var/flows/reports/rrds</rrddir>
  <scoredir>/var/flows/score</scoredir>
  <sampletime>300</sampletime>
</config>

<scoredir> is mandatory; the rest are included as needed for the functionality you want.

<config>
  <all localsubnets="10.0.0.0/8">
    <application name="web">80/tcp,8080/tcp</application>
    <protocols>tcp,udp,icmp</protocols>
    <otherservices/>
    <total/>
  </all>
  <rrddir>/var/flows/reports/rrds</rrddir>
  <scoredir>/var/flows/score</scoredir>
  <sampletime>300</sampletime>
</config>

Named groups of subnets, so directions can refer to a site instead of repeating subnet lists:

<sites>
  <site name="Belgium" subnets="10.10.0.0/16"/>
  <site name="England" subnets="10.30.0.0/16"/>
</sites>

<direction name="Belgium-England" from="Belgium" to="England">
  ...
</direction>

Reusable blocks of monitoring configuration you can drop into multiple directions with <set>:

<definesets>
  <defineset name="Common Services">
    <application name="web">80/tcp,8080/tcp</application>
    <protocols>tcp,udp,icmp</protocols>
    <total/>
  </defineset>
</definesets>

<direction name="Belgium-Holland">
  <set name="Common Services"/>
</direction>

• fromsubnets / tosubnets — source/destination subnets for a direction
• nofromsubnets / notosubnets — subnets to exclude
• fromas / toas — match by source/destination Autonomous System number
• samplerate — account for sampled NetFlow (sampling ratio)

• <application name="...">port/proto,...</application> — group several services under one label (e.g. web, mail)
• <services>...</services> — individual port/protocol definitions, e.g. 22-23/tcp,25/tcp,102/tcp
• <protocols>tcp,udp,icmp,ospf</protocols> — track specific IP protocols
• <ftp/> — stateful FTP session monitoring
• <multicast/> — multicast traffic
• <tos/> — Type of Service breakdown (BE and others)
• <dscp/> — DSCP value breakdown (BE, EF, AFxx, CSx)
• <otherservices/> / <otherprotocols/> — catch-alls for traffic that didn't match a defined service/protocol
• <total/> — aggregate total traffic for the direction (required if you want JKGrapher to graph it)

JKGrapher.pl, run as a CGI script, reads the RRD files JKFlow produces and renders traffic graphs per direction/site/router group. See the Flash demo for an example of the interface.

A full manual, "JKFlow — Flexible XML configurable report module for FlowScan," is available as a PDF: https://jkflow.sourceforge.net/eindwerk.pdf

• Latest release (3.5.2): https://sourceforge.net/projects/jkflow/files/jkflow3/
• Project page: https://sourceforge.net/projects/jkflow/

GNU General Public License v2 (GPLv2).

Jurgen Kobierczynski (SourceForge user jurgenk), project registered on SourceForge in February 2004.