BirdNET-STM32

Bird sound classification for edge deployment on the STM32N6570-DK development board with neural processing unit (NPU).

Read the full BirdNET-STM32 documentation for setup, training, conversion, evaluation, and deployment guides.

A compact DS-CNN trained on raw waveforms or spectral features, quantized to INT8 with post-training quantization or quantization-aware fine-tuning, and deployed using ST's X-CUBE-AI toolchain. The standalone firmware supports raw waveform, hybrid STFT, and precomputed-mel deployment paths. In a verified 24 kHz, 2.5-second raw configuration, inference takes 12–13 ms on the NPU and about 84 ms total including SD-card input; exact timing depends on the model and SD card.

# Install
git clone https://github.com/birdnet-team/birdnet-stm32.git
cd birdnet-stm32
python3.12 -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"

# Train (the defaults are the release recipe: raw frontend, 2.5 s at 24 kHz)
python -m birdnet_stm32 train \
  --data_path_train data/train

# Convert to quantized TFLite
python -m birdnet_stm32 convert \
  --checkpoint_path checkpoints/best_model.keras \
  --model_config checkpoints/best_model_model_config.json \
  --data_path_train data/train

# Evaluate
python -m birdnet_stm32 evaluate \
  --model_path checkpoints/best_model_quantized.tflite \
  --model_config checkpoints/best_model_model_config.json \
  --data_path_test data/test --pooling lme

# Apply an explicit operational release profile to the INT8 artifact
python -m birdnet_stm32 measure-operational \
  --model_path checkpoints/best_model_quantized.tflite \
  --model_config checkpoints/best_model_model_config.json \
  --data_path_test data/test --gate_profile gate.json \
  --report_json report/operational.json

# Deploy to STM32N6570-DK (requires config.json; see config.example.json)
python -m birdnet_stm32 deploy

# On-board integration test (requires SD card with test audio)
python -m birdnet_stm32 board-test

The board-test command runs inference entirely on the STM32N6570-DK. It reads WAV files from the SD card, applies the model-specific preprocessing on the board (peak normalization for raw, STFT for hybrid, or STFT + mel for librosa), and runs the model on the NPU. WAV files must match the model's sample rate, recorded in _model_config.json; mismatches are skipped.

Prepare the SD card as follows:

1. Format as FAT32.
2. Create an audio/ directory at the root.
3. Copy .wav files (mono or stereo, 16-bit PCM) into audio/. Each file should be at least as long as the model's chunk duration (default 2.5 s).
4. Insert the SD card into the STM32N6570-DK board slot.

See the full documentation for detailed guides on dataset preparation, training, conversion, evaluation, and deployment.

Trained and converted models are published as release assets — grab a bundle from the latest release.

Every file shares one basename, BirdNET_Tiny_N6_<REGION>_<SPECIES>_V<VERSION>, with a _Raw or _Hybrid suffix from 1.3 on, when a release ships both frontends:

• _Raw takes 2.5 s of audio; the whole pipeline runs on the NPU (75 ms per file on the board). This is the reference release model.
• _Hybrid takes a spectrogram the firmware computes on the Cortex-M55; it has the better INT8 accuracy, at 148 ms per file. On any other device, compute its input exactly as specified in Spectrogram Input.

A bundle whose model was exported split also carries <basename>_INT8_backbone.tflite and <basename>_INT8_classifier.tflite (each with a .gz, plus the backbone's .fingerprint.json and the head's own labels). The firmware runs a single network, so _INT8.tflite is still what you flash; the pair exists so a species-list change can be pushed as a few kilobytes instead of the whole model. See the conversion guide.

The TFLite model, config, and labels are a single contract — keep them together and never mix files across bundles. Models are licensed under the Apache License 2.0; the bundle also carries the acceptable use policy.

Everything the firmware needs is in the bundle — no extra downloads.

1. Install the toolchain (X-CUBE-AI, STM32CubeProgrammer/IDE, ARM GNU) and copy config.example.json to config.json, filling in your local tool paths.

2. Prepare an SD card with test audio as described above, matching the sample rate in _model_config.json.

3. Compile, flash, and run on the NPU:

   # The globs below pick files out of the bundle, so this does not depend on
   # which region or version you downloaded.
   BUNDLE=~/Downloads/<bundle>
   
   python -m birdnet_stm32 board-test \
     --model_path    "$BUNDLE"/*_INT8.tflite \
     --model_config  "$BUNDLE"/*_model_config.json \
     --labels        "$BUNDLE"/*_labels.txt

The command generates the N6 binary, flashes the board over serial, runs inference on every WAV on the SD card, and streams the top predictions back over UART. See the deployment guide for toolchain setup and troubleshooting.

• Audio frontends: raw (waveform → learned Gabor quadrature filterbank, default), hybrid (linear STFT + learned mel mixer), and librosa (precomputed mel). All three are supported by the standalone firmware. hybrid and librosa take an optional --input_compression (sqrt, log) applied before the first INT8 tensor. The mfcc and log_mel modes were removed in 1.2.0.
• Magnitude scaling: pwl (learned piecewise-linear, quantization-friendly) and none (pass-through ablation baseline). pcen and db were removed in 1.2.0 — dB's log op creates exactly the wide dynamic range INT8 cannot hold, and PCEN was never used by a release.
• Model: DS-CNN with configurable width (--alpha) and depth (--depth_multiplier) built from plain depthwise separable blocks
• Augmentation: Dirichlet multi-source mixup with multi-label union targets for overlapping vocalizations, SpecAugment (on by default), smart crop for long recordings
• Optimization: linear warmup into cosine LR decay, Adam/SGD/AdamW, gradient clipping (on by default), mixed precision (FP16). Standard training checkpoints track exact validation cMAP; QAT selects matching Keras/TFLite artifacts using actual converted INT8 file cMAP, including the starting checkpoint
• QAT: native Keras 3 quantization-aware fine-tuning via --qat — uses the converter's exact calibration manifest to simulate the INT8 input, per-channel kernels, fused activation boundaries, and otherwise-opaque raw-frontend internals; frozen-teacher KL plus mean and configurable worst-sample cosine consistency protect probability calibration and lower-tail parity while exact converted INT8 validation cMAP selects the deployment checkpoint
• Linear probing: --linear_probe freezes a pretrained backbone and trains only the classifier head

• Post-training quantization: INT8 internals, float32 I/O, per-channel (default) or per-tensor
• Dynamic range quantization: --quantization dynamic — no calibration data needed
• Backbone/classifier split: --split_head also emits the backbone (audio to embeddings) and the classifier head (embeddings to scores) as separate gated artifacts with deterministic .gz copies, so a species-list change can be pushed over a narrowband satellite link as a few kB instead of the whole model. The head is calibrated on the quantized backbone's embeddings, and the chained pair must clear the same parity gates as the monolithic model
• Validation: mean and tail cosine similarity, MSE, MAE, and Pearson r on a deterministic held-out calibration split
• Atomic quality gate: failed conversions never promote a release-looking .tflite
• ONNX export: --export_onnx uses the Keras 3 exporter, ONNX checker, and ONNX Runtime parity gate (requires tf2onnx, onnx, and onnxruntime)

• Pooling: avg, max, LME (log-mean-exponential)
• Metrics: ROC-AUC, cmAP, mAP, precision, recall, F1
• Species AP report: per-species AP with bootstrap 95% CI (--species_report)
• DET curve: detection error tradeoff (--det_curve, --save_det_plot)
• Latency measurement: per-chunk inference timing (--benchmark_latency)
• Benchmark JSON: structured report for experiment tracking (--benchmark)
• HTML report: self-contained evaluation report (--report_html)

• X-CUBE-AI / stedgeai: generate → flash → validate pipeline
• Board test: standalone on-device inference (board-test) — reads WAV from SD, performs frontend-specific preprocessing, runs the NPU, and captures results over UART

• Source code: MIT License
• Model artifacts (checkpoints, .tflite/.onnx exports, labels, model config): Apache License 2.0
• Third-party firmware sources: STMicroelectronics (BSD-3-Clause) and ChaN's FatFs retain their original licenses — see firmware/THIRD_PARTY_LICENSES.md.
• STM tools and scripts: see respective documentation for license details.

@article{kahl2025birdnetstm32,
  title={A quantization-friendly audio classification pipeline for embedded bioacoustics on microcontroller NPUs},
  author={Kahl, Stefan and Marshall, Isabella and Chaopricha, Patrick T. and Aceto, Jordan and Klinck, Holger},
  year={2025}
}

See CONTRIBUTING.md for guidelines. AI-assisted contributions are welcome — keep PRs focused and review every line.

BirdNET is built to support biodiversity research and conservation. See ACCEPTABLE_USE.md for how we expect the code and models to be used, including uses we consider unacceptable and guidance on branding derivative work.

Our work in the Cornell K. Lisa Yang Center for Conservation Bioacoustics is made possible by the generosity of K. Lisa Yang to advance innovative conservation technologies to inspire and inform the conservation of wildlife and habitats.

The development of BirdNET is supported by the German Federal Ministry of Research, Technology and Space (FKZ 01|S22072), the German Federal Ministry for the Environment, Climate Action, Nature Conservation and Nuclear Safety (FKZ 67KI31040E), the German Federal Ministry of Economic Affairs and Energy (FKZ 16KN095550), the Deutsche Bundesstiftung Umwelt (project 39263/01) and the European Social Fund.

BirdNET is a joint effort of partners from academia and industry. Without these partnerships, this project would not have been possible. Thank you!