Skip to content

atgreen/Whistler

Repository files navigation

Whistler

A Lisp compiling to eBPF.

Whistler is a Common Lisp dialect for writing eBPF programs. It compiles s-expressions to eBPF bytecode and emits valid ELF object files your kernel loads directly. The compilation pipeline has zero dependency on C, clang, or LLVM.

(defmap pkt-count :type :array
  :key-size 4 :value-size 8 :max-entries 1)

(defprog count-packets (:type :xdp :license "GPL")
  (incf (getmap pkt-count 0))
  XDP_PASS)

This compiles to 11 eBPF instructions and a valid BPF ELF object file.

Why Whistler

eBPF programs are small, verifier-constrained, and pattern-driven. Whistler is built for that shape:

  • No C toolchain. The compiler is self-contained Common Lisp (~7,000 lines). No LLVM, no kernel headers, no libelf. The ELF writer is hand-rolled.
  • Real metaprogramming. eBPF code is full of recurring patterns: parse headers, validate packet bounds, look up state. In Whistler, those become hygienic macros instead of preprocessor tricks.
  • Compiler-aware abstractions. Struct accessors, protocol helpers, and map operations are part of the language, so the compiler optimizes them intentionally rather than recovering patterns after C lowering.
  • Automatic CO-RE. Struct identity is preserved through the pipeline. CO-RE relocations are emitted automatically.
  • Interactive development. Full REPL. Compile, load, attach, inspect maps, iterate — all from one Lisp image.
  • bpftrace-compatible frontend. The same binary runs bpftrace-syntax scripts (whistler bpftrace -e 'kprobe:vfs_read { @[comm] = count(); }') with the surface language documented below — no separate bpftrace install needed.

If you already have a C/libbpf workflow, Whistler is not trying to replace it wholesale. It targets cases where you want a language and compiler designed around eBPF itself.

Side-by-side

Whistler C + clang BCC (Python) Aya (Rust) bpftrace
Toolchain size ~3 MB (SBCL) ~200 MB ~100 MB ~500 MB ~50 MB
Compile-time metaprogramming Full CL macros #define Python strings proc_macro none
Output ELF .o ELF .o JIT loaded ELF .o JIT loaded
Self-contained compiler yes no (needs LLVM) no (needs kernel headers) no (needs LLVM) no
Interactive development REPL no yes no yes
Code quality vs clang -O2 matches or beats baseline n/a comparable n/a

Getting started

Three entry points depending on what you're after:

  1. Run a bpftrace scriptsudo whistler bpftrace -e '<script>'. No separate language to learn; if you already know bpftrace, just point whistler bpftrace at your script. See bpftrace frontend.

  2. Interactive Lispmake repl-loader, then work in whistler-loader-user. Use with-bpf-session to compile, load, attach, and inspect from one image. This is the intended default for writing Whistler programs directly.

  3. CLI artifactwhistler compile X.lisp -o X.bpf.o produces a .bpf.o for CI, packaging, or external tooling.

Requirements

  • SBCL (Steel Bank Common Lisp) 2.0+
  • Linux with kernel 5.3+ (for bounded loop support)
  • FiveAM (for tests only)

Build

make        # build standalone binary
make test   # run test suite (requires FiveAM)
make repl   # compiler REPL in whistler-user
make repl-loader  # compiler + loader REPL in whistler-loader-user
./whistler doctor  # check local eBPF dev prerequisites

Compile an example

# Using the REPL:
make repl
* (load "examples/synflood-xdp.lisp")
* (compile-to-elf "synflood.bpf.o")
Compiled 1 program (74 instructions total), 2 maps → synflood.bpf.o

# Or from the command line:
./whistler compile examples/count-xdp.lisp -o count.bpf.o

Load into the kernel

ip link set dev eth0 xdp obj count.bpf.o sec xdp
bpftool map dump name pkt_count
ip link set dev eth0 xdp off

Permissions

# Allow BPF program loading and perf event attachment
sudo setcap cap_bpf,cap_perfmon+ep /usr/bin/sbcl

# Allow reading tracepoint format files (for deftracepoint)
sudo chmod a+r /sys/kernel/tracing/events/sched/sched_switch/format

Generate userspace headers

Whistler generates matching struct definitions for your userland code from the same defstruct declarations used in the BPF program:

./whistler compile probes.lisp --gen c        # C header
./whistler compile probes.lisp --gen c go rust python lisp  # multiple
./whistler compile probes.lisp --gen all      # all supported

Examples

Packet counter

Whistler: 11 instructions. clang -O2: 11 instructions.

WhistlerC + clang
(defmap pkt-count :type :array
  :key-size 4 :value-size 8
  :max-entries 1)

(defprog count-packets
    (:type :xdp :license "GPL")
  (incf (getmap pkt-count 0))
  XDP_PASS)
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>

char __license[] SEC("license") = "GPL";

struct {
  __uint(type, BPF_MAP_TYPE_ARRAY);
  __type(key, __u32);
  __type(value, __u64);
  __uint(max_entries, 1);
} pkt_count SEC(".maps");

SEC("xdp")
int count_packets(struct xdp_md *ctx) {
  __u32 key = 0;
  __u64 *val = bpf_map_lookup_elem(
                 &pkt_count, &key);
  if (val)
    __sync_fetch_and_add(val, 1);
  return XDP_PASS;
}

Port blocker

Whistler: 25 instructions. clang -O2: 26 instructions.

WhistlerC + clang
(defmap drop-count :type :array
  :key-size 4 :value-size 8
  :max-entries 1)

(defprog drop-port
    (:type :xdp :license "GPL")
  (with-tcp (data data-end tcp)
    (when (= (tcp-dst-port tcp) 9999)
      (incf (getmap drop-count 0))
      (return XDP_DROP)))
  XDP_PASS)
#include <linux/bpf.h>
#include <linux/if_ether.h>
#include <linux/ip.h>
#include <linux/tcp.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_endian.h>

char __license[] SEC("license") = "GPL";

struct {
  __uint(type, BPF_MAP_TYPE_ARRAY);
  __type(key, __u32);
  __type(value, __u64);
  __uint(max_entries, 1);
} drop_count SEC(".maps");

SEC("xdp")
int drop_port(struct xdp_md *ctx) {
  void *data = (void *)(long)ctx->data;
  void *end  = (void *)(long)ctx->data_end;
  if (data + sizeof(struct ethhdr)
      + sizeof(struct iphdr)
      + sizeof(struct tcphdr) > end)
    return XDP_PASS;
  struct ethhdr *eth = data;
  if (eth->h_proto != htons(ETH_P_IP))
    return XDP_PASS;
  struct iphdr *ip = data + sizeof(*eth);
  if (ip->protocol != IPPROTO_TCP)
    return XDP_PASS;
  struct tcphdr *tcp = (void *)ip
                       + sizeof(*ip);
  if (ntohs(tcp->dest) == 9999) {
    __u32 key = 0;
    __u64 *val = bpf_map_lookup_elem(
                   &drop_count, &key);
    if (val)
      __sync_fetch_and_add(val, 1);
    return XDP_DROP;
  }
  return XDP_PASS;
}

SYN flood mitigation

Whistler: 65 instructions. clang -O2: 68 instructions.

WhistlerC + clang
(defmap syn-counter :type :hash
  :key-size 4 :value-size 8
  :max-entries 32768)
(defmap syn-stats :type :array
  :key-size 4 :value-size 8
  :max-entries 3)

(defconstant +syn-threshold+ 100)

(defprog synflood
    (:type :xdp :license "GPL")
  (with-tcp (data data-end tcp)
    (when (= (logand (tcp-flags tcp)
                     #x12)
             +tcp-syn+)
      (incf (getmap syn-stats 0))
      (let ((src (ipv4-src-addr
                   (+ data
                      +eth-hdr-len+))))
        (if-let (p (map-lookup
                     syn-counter src))
          (if (> (load u64 p 0)
                 +syn-threshold+)
            (progn
              (incf (getmap syn-stats 1))
              (return XDP_DROP))
            (atomic-add p 0 1))
          (progn
            (incf (getmap syn-stats 2))
            (setf (getmap syn-counter
                          src) 1))))))
  XDP_PASS)
#include <linux/bpf.h>
#include <linux/if_ether.h>
#include <linux/ip.h>
#include <linux/tcp.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_endian.h>

char __license[] SEC("license") = "GPL";

struct {
  __uint(type, BPF_MAP_TYPE_HASH);
  __type(key, __u32);
  __type(value, __u64);
  __uint(max_entries, 32768);
} syn_counter SEC(".maps");

struct {
  __uint(type, BPF_MAP_TYPE_ARRAY);
  __type(key, __u32);
  __type(value, __u64);
  __uint(max_entries, 3);
} syn_stats SEC(".maps");

static void bump_stat(void *map, __u32 idx) {
  __u64 *val = bpf_map_lookup_elem(
                 map, &idx);
  if (val)
    __sync_fetch_and_add(val, 1);
}

#define SYN_THRESHOLD 100
#define TCP_SYN 0x02
#define TCP_ACK 0x10

SEC("xdp")
int synflood(struct xdp_md *ctx) {
  void *data = (void *)(long)ctx->data;
  void *end  = (void *)(long)ctx->data_end;
  if (data + sizeof(struct ethhdr)
      + sizeof(struct iphdr)
      + sizeof(struct tcphdr) > end)
    return XDP_PASS;
  struct ethhdr *eth = data;
  if (eth->h_proto != htons(ETH_P_IP))
    return XDP_PASS;
  struct iphdr *ip = data + sizeof(*eth);
  if (ip->protocol != IPPROTO_TCP)
    return XDP_PASS;
  struct tcphdr *tcp = (void *)ip
                       + sizeof(*ip);
  __u8 flags = ((__u8 *)tcp)[13];
  if (!(flags & TCP_SYN)
      || (flags & TCP_ACK))
    return XDP_PASS;

  bump_stat(&syn_stats, 0);

  __u32 src = ip->saddr;
  __u64 *count = bpf_map_lookup_elem(
                   &syn_counter, &src);
  if (count) {
    if (*count > SYN_THRESHOLD) {
      bump_stat(&syn_stats, 1);
      return XDP_DROP;
    }
    __sync_fetch_and_add(count, 1);
  } else {
    bump_stat(&syn_stats, 2);
    __u64 init = 1;
    bpf_map_update_elem(&syn_counter,
      &src, &init, BPF_ANY);
  }
  return XDP_PASS;
}

Userspace loader

whistler/loader is a pure Common Lisp BPF loader — no libbpf, no CFFI. It loads .bpf.o files, creates maps, attaches programs (kprobe, uprobe, tracepoint, XDP, TC, cgroup), and consumes ring buffers from SBCL.

Inline BPF sessions

Write BPF programs and userspace code in the same Lisp form. The BPF code compiles at macroexpand time, and the bytecode is embedded as a literal:

(whistler/loader:with-bpf-session ()
  ;; Kernel side, compiled to eBPF at macroexpand time
  (bpf:map counter :type :hash :key-size 4 :value-size 8 :max-entries 1024)
  (bpf:prog trace (:type :kprobe :section "kprobe/__x64_sys_execve" :license "GPL")
    (incf (getmap counter 0))
    0)

  ;; Userspace side, normal CL at runtime
  (bpf:attach trace "__x64_sys_execve")
  (loop (sleep 1)
        (format t "count: ~d~%" (bpf:map-ref counter 0))))

One file, one language. No intermediate artifacts or separate build steps.

bpftrace frontend

whistler/bpftrace runs scripts written in bpftrace's surface language. It parses the script, lowers it through the same SSA/regalloc pipeline the rest of Whistler uses, and attaches the resulting BPF programs via the pure-CL loader. Same self-contained binary — no clang, no LLVM, no libbpf.

sudo ./whistler bpftrace \
  -e 'tracepoint:syscalls:sys_enter_openat
        { @[comm] = count(); }'
^C
@[Hyprland]:        1
@[code]:            17
@[ptyxis]:          43
...

CLI options

whistler bpftrace matches bpftrace's day-to-day workflow flags:

Flag Behaviour
-e PROGRAM Inline script text
-l [PATTERN] List kernel probes matching glob — whistler bpftrace -l 'kprobe:tcp_*'
-p PID Inject /pid == PID/ predicate on every probe
-c 'CMD' Spawn CMD ptrace-stopped, attach probes, resume; exit when child exits
--dump Print generated Whistler forms and exit (no kernel load)
-V Version
-h Help
-- --NAME[=VALUE] Named params consumed by getopt(NAME, default) in the script (matches bpftrace's script.bt -- --foo=5 convention)

If whistler is invoked under the name bpftrace (e.g. via a ln -sf whistler /usr/local/bin/bpftrace), it dispatches as if the user typed whistler bpftrace …. Drop-in for scripts that hard-code the bpftrace command name.

-c uses PTRACE_TRACEME/SIGSTOP to stop the child at the exec entry, attaches probes, then PTRACE_DETACHs — matching bpftrace's behaviour so short-lived commands have probes live for their full lifetime.

Surface language coverage

Almost everything you'd write in a typical bpftrace script:

  • Probes: kprobe, kretprobe, kfunc, kretfunc, uprobe, uretprobe, tracepoint, profile, interval, BEGIN, END. Wildcards (kprobe:tcp_*) and multi-target specs (kprobe:foo,kprobe:bar).
  • Aggregations: count(), sum(x), avg(x), min(x), max(x), stats(x), hist(x), lhist(x, min, max, step).
  • Async actions: printf (with %-16s/%05d flags), print(@m [, top [, div]]) (top-N + value scaling), clear(@m), zero(@m), delete(@m[k]), time(), cat("/path"), join(argv), system("cmd") (shells out userspace), errorf / warnf (printf to stderr), fail("msg") (compile-time abort), exit().
  • String / address builtins: str(ptr [, n]), kstr(ptr [, n]), ksym(addr), usym(addr), ntop([af,] addr) (literal or runtime family), reg("ip"|"sp"|…), syscall_name(id), signal_name(N), macaddr(ptr), path(struct path *) (bpf_d_path), kaddr("name") (kallsyms lookup), pton("…") (compile-time IP parse), strerror(N).
  • String operations: strncmp(s1, s2, n), strcontains(haystack, needle), len(s), assert(cond, msg) (errorf+exit on fail), static_assert(cond, msg) (compile-time abort).
  • Misc helpers: socket_cookie(sk), cgroupid("/path") (compile-time stat), signal(N) (send to current task), override(retval) (kprobe error injection), jiffies(), is_err(p), kptr(p) / uptr(p) (identity), cpid() / has_cpid() (from -c).
  • Compile-time type predicates: is_str, is_ptr, is_array, is_integer, is_unsigned_integer, is_literal — fold to 0/1 for if comptime (is_str($x)) { … } macro dispatch.
  • CLI integration: getopt(NAME, default) reads whistler bpftrace script.bt -- --NAME[=VALUE] parameters and returns the parsed value. Bool, int, and string defaults all flow correctly through printf %s, $v = …, and @m[k] = ….
  • Script configuration: top-level config = { KEY=VALUE; …} blocks. Honored knobs: print_maps_on_exit, max_strlen, max_map_keys, on_stack_limit, missing_probes (warn / ignore / error), str_trunc_trailer, stack_mode (bpftrace / perf / raw).
  • Variables: pid, tid, uid, gid, comm, pcomm, nsecs, cpu, cgroup, elapsed, retval, curtask, args, probe, func, arg0..arg9 (kprobe args 6+ via stack reads on x86-64, registers on arm64), kstack, ustack, $local, @global, and @map[k1, k2, …]. Each is also valid with empty parens (pid(), comm(), etc.) per bpftrace's stdlib convention.
  • Symbolic constants: AF_INET, O_RDONLY, IPPROTO_TCP, etc. — resolved from kernel BTF enums + a curated #define table, no C headers required.
  • Struct access: curtask->pid, ((struct sock_common *)arg0)->skc_family (BTF-resolved field offsets, scalar fields).
  • Control flow: if/else, ternary ?:, filter predicates /expr/, while loops, user-defined fn.

The script-side text below produces a working opensnoop:

sudo ./whistler bpftrace \
  -e 'tracepoint:syscalls:sys_enter_openat
        { printf("%-16s %s\n", comm, str(args->filename)); }'

Userspace stack frames are symbolized via a pure-CL ELF / DWARF .debug_line reader (whistler/symbolize), so @[ustack] renders with real symbol names + file:line when debuginfo is installed.

Documentation

The full language reference, compilation model, and API details are in The Whistler Book.

Author

Whistler was created by Anthony Green.

License

MIT

The compiler itself is MIT-licensed. BPF programs compiled by Whistler typically use license "GPL" in their defprog because the kernel requires GPL for BPF programs calling GPL-only helpers.

About

A lisp that compiles to eBPF

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors