The previous release introduced a performance regression for users with many
flows from a single exporter. This is fixed in this release.

• 🩹 docker: restart geoip container on boot
• 🌱 inlet: make load-balancing algorithm for Kafka partitions configurable
  (random or by-exporter) and revert back to random by default (like before 2.0.3)
• 🌱 orchestrator: add kafka→manage-topic flag to enable or disable topic management
• 🌱 cmd: make akvorado healthcheck use an abstract Unix socket to check service liveness

Docker image: docker pull ghcr.io/akvorado/akvorado:2.0.4
Full changelog: v2.0.3...v2.0.4

This release contains some important fixes to prevent flow corruption under heavy load.

• 💥 config: skip-verify is false by default in TLS configurations for
  ClickHouse, Kafka and remote data sources (previously, verify was set to
  false by default)
• 🩹 inlet: keep flows from one exporter into a single partition
• 🩹 outlet: provide additional gracetime for a worker to send to ClickHouse
• 🩹 outlet: prevent discarding flows on shutdown
• 🩹 outlet: enhance scaling up and down workers to avoid hysteresis
• 🩹 outlet: accept flows where interface names or descriptions are missing
• 🩹 docker: update Traefik to 3.6.1 (for compatibility with Docker Engine 29)
• 🌱 common: enable block and mutex profiling
• 🌱 outlet: save IPFIX decoder state to a file to prevent discarding flows on start
• 🌱 config: rename verify to skip-verify in TLS configurations for
  ClickHouse, Kafka and remote data sources (with inverted logic)
• 🌱 config: remote data sources accept a specific TLS configuration
• 🌱 config: gNMI metadata provider has been converted to the same TLS
  configuration than ClickHouse, Kafka and remote data sources.
• 🌱 docker: update Kafka to 4.1.1
• 🌱 docker: update Kafbat to 1.4.2

Docker image: docker pull ghcr.io/akvorado/akvorado:2.0.3
Full changelog: v2.0.2...v2.0.3

The modification of the default value of inlet→kafka→queue-size should
prevent packet drops on busier setups.

• 💥 config: stop shipping demo exporter configurations from the orchestrator
• ✨ inlet: load-balance incoming UDP packets to all workers using eBPF on
  Linux (check docker/docker-compose-local.yaml to enable)
• 🩹 inlet: fix akvorado_inlet_flow_input_udp_in_dropped_packets_total metric
• 🩹 console: fix completion tooltip being obscured with Firefox
• 🌱 inlet: increase default kafka→queue-size value to 4096 to prevent packet drops
• 🌱 outlet: be more aggressive when increasing the number of workers
• 🌱 outlet: cap the number of workers to the number of Kafka partitions
• 🌱 console: add auth→logout-url and auth→avatar-url to configure
  logout and avatar URLs when not provided as headers
• 🌱 docker: update Vector to 0.50.0.

Docker image: docker pull ghcr.io/akvorado/akvorado:2.0.2
Full changelog: v2.0.1...v2.0.2

• 🩹 inlet: disable kernel timestamping on Linux kernel older than 5.1
• 🩹 outlet: fix gNMI metadata provider exiting too early
• 🩹 doc: fix documentation for SNMPv3 configuration
• 🌱 inlet: add support for RFC 5103 (bidirectional flows)
• 🌱 outlet: handle discard and multiple interfaces for expanded sFlow samples

Docker image: docker pull ghcr.io/akvorado/akvorado:2.0.1
Full changelog: v2.0.0...v2.0.1

There is a companion blog post to this release.

This release introduces a new component: the outlet. Previously, ClickHouse was
fetching data directly from Kafka. However, this required pushing the protobuf
schema using an out-of-band method. This makes cloud deployments more complex.
The inlet now pushes incoming raw flows to Kafka without decoding them. The
outlet takes them, decodes them, enriches them, and pushes them to ClickHouse.
This also reduces the likelihood of losing packets.

This change should be transparent on most setups but you are encouraged to
review the new proposed configuration in the quickstart tarball and update
your own configuration to move the appropriate configuration from the inlet to
the outlet service.

As it seems a good time as any, Zookeeper is removed from the docker compose
setup. ClickHouse Keeper is used instead when setting up a cluster. Kafka is now
using the KRaft mode. You need to recreate the Kafka container:

# docker compose down --remove-orphans
# docker compose rm --volumes kafka
# docker volume rm akvorado_akvorado-kafka
# docker compose pull
# docker compose up -d

The documentation has been updated, notably the troubleshooting section.

If you use the monitoring stack, note that the Docker Compose file was split
into docker-compose-prometheus.yml for metrics, and docker-compose-loki.yml
for logs. You need to update your .env. Also, metric scraping is now done by
Grafana Alloy instead of Prometheus andx you need to fix the ownership of the
Prometheus volume:

# docker compose run --user root --entrypoint="/bin/sh -c" prometheus "chown -R nobody:nobody /prometheus"

• ✨ outlet: new service
• ✨ orchestrator: automatic restart of the orchestrator service on configuration change
• 💥 inlet: flow rate limiting feature has been removed
• 💥 docker: rename docker-compose-monitoring.yml to docker-compose-prometheus.yml
• 💥 docker: enforce a specific IPv4 subnet (in the reserved class E)
• 💥 common: be stricter on results returned from remote sources
• 💥 docker: switch to Apache Kafka 4.1 with KRaft mode
• 💥 docker: switch from Prometheus to Grafana Alloy for scraping metrics
• 💥 docker: use profiles to optionally enable Prometheus, Loki, and Grafana
  (if you were already using them, you also need to enable the profile)
• 🩹 console: display missing images in documentation
• 🩹 console: ensure main table is used when required even when there is no data
• 🩹 console: fix deletion of saved filters
• 🩹 console: fix intermittent failure when requesting previous period
• 🩹 docker: move healthcheck for IPinfo updater into Dockerfile to avoid
  "unhealthy" state on non-updated installations
• 🌱 cmd: make akvorado version shorter (use -d for full output)
• 🌱 inlet: improve performance of classifiers
• 🌱 outlet: decode IPFIX ingressPhysicalInterface and egressPhysicalInterface
• 🌱 outlet: improve performance of the BMP routing provider
• 🌱 console: submit form on Ctrl-Enter or Cmd-Enter while selecting dimensions
• 🌱 orchestrator: move ClickHouse database settings from clickhouse to clickhousedb
• 🌱 build: accept building with a not up-to-date toolchain
• 🌱 build: build with Go 1.25 and use bundled toolchain
• 🌱 build: modernize JavaScript build with Oxlint and Rolldown-Vite
• 🌱 build: switch from NPM to PNPM for JavaScript build and reduce dependencies
• 🌱 config: listen to 4739 for IPFIX on inlet service
• 🌱 docker: stop spawning demo exporters by default
• 🌱 docker: build a linux/amd64/v3 image to enable optimizations
• 🌱 docker: build a linux/arm/v7 image
• 🌱 docker: add IPv6 configuration
• 🌱 docker: switch from Provectus Kafka UI (unmaintained) to Kafbat UI
• 🌱 docker: switch to Prometheus Java Agent exporter for Kafka
• 🌱 docker: update ClickHouse to 25.8 (not mandatory)
• 🌱 docker: update Prometheus to 3.5.0
• 🌱 docker: update Traefik to 3.4 (not mandatory)
• 🌱 docker: update node-exporter to 1.9.1
• 🌱 docker: add Loki to the observability stack
• 🌱 docker: add cAdvisor to the observability stack
• 🌱 docker: add examples to enable authentication and TLS
• 🌱 docker: change default log level for ClickHouse from trace to information
• 🌱 docker: enable HTTP compression for Traefik
• 🌱 docker: enable access log for Traefik
• 🌱 docker: expose Kafka UI (read-only) to the public endpoint
• 🌱 docker: expose Traefik Dashboard (read-only) to the public endpoint
• 🌱 docker: expose metrics to the public endpoint
• 🌱 documentation: document how to tune TCP receive buffer for BMP routing provider
• 🌱 documentation: document how to update the database schema for installations before 1.10.0

Docker image: docker pull ghcr.io/akvorado/akvorado:2.0.0
Full changelog: v1.11.5...v2.0.0

• 💥 docker: use profiles to optionally enable Prometheus, Loki, and Grafana
  (if you were already using them, you also need to enable the profile)
• 🩹 console: display missing images in documentation
• 🩹 console: ensure main table is used when required even when there is no data
• 🩹 docker: fix broken /metrics endpoint for inlet
• 🌱 outlet: improve performance of the RIB for the BMP routing provider
• 🌱 build: accept building with a not up-to-date toolchain
• 🌱 docker: update ClickHouse to 25.8 (not mandatory)
• 🌱 docker: update Kafka to 4.1.0 (not mandatory)
• 🌱 docker: update Loki to 3.5.4 (not mandatory)
• 🌱 docker: enable HTTP compression for Traefik

Docker image: docker pull ghcr.io/akvorado/akvorado:2.0.0-beta.6
Full changelog: v2.0.0-beta.5...v2.0.0-beta.6

If you use the monitoring stack, note that the Docker Compose file was split
into docker-compose-prometheus.yml for metrics, and docker-compose-loki.yml
for logs. You need to update your .env. Also, metric scraping is now done by
Grafana Alloy instead of Prometheus andx you need to fix the ownership of the
Prometheus volume:

# docker compose run --user root --entrypoint="/bin/sh -c" prometheus "chown -R nobody:nobody /prometheus"

If you upgrade from a previous 2.0.0-beta version, you need again to recreate
the Kafka container:

# docker compose down --remove-orphans
# docker compose rm --volumes kafka
# docker volume rm akvorado_akvorado-kafka
# docker compose pull
# docker compose up -d

• 💥 docker: update Kafka data volume mount path
• 💥 docker: switch from Prometheus to Grafana Alloy for scraping metrics
• 🩹 outlet: fix crash when scaling down and up the Kafka workers
• 🩹 outlet: move gRPC metrics for BioRIS provider in the routing namespace
• 🌱 config: listen to 4739 for IPFIX on inlet service
• 🌱 docker: enforce bridge name
• 🌱 docker: add Loki to the observability stack
• 🌱 docker: add cAdvisor to the observability stack
• 🌱 docker: update Prometheus to 3.5.0
• 🌱 docker: update node-exporter to 1.9.1
• 🌱 docker: stop spawning demo exporters by default
• 🌱 build: build with Go 1.25

Docker image: docker pull ghcr.io/akvorado/akvorado:2.0.0-beta.5
Full changelog: v2.0.0-beta.4...v2.0.0-beta.5

This is likely the last beta before releasing 2.0.0.

• 🌱 outlet: improve performance of the BMP routing provider
• 🌱 documentation: document how to tune TCP receive buffer for BMP routing provider
• 🌱 documentation: document how to update the database schema for installations before 1.10.0

Docker image: docker pull ghcr.io/akvorado/akvorado:2.0.0-beta.4
Full changelog: v2.0.0-beta.3...v2.0.0-beta.4

• 💥 docker: enforce a specific IPv4 subnet (in the reserved class E)
• 🌱 docker: add IPv6 configuration
• 🌱 outlet: dynamically adjust the number of Kafka workers to ensure better performance from ClickHouse
• 🌱 outlet: insert asynchronously when flow count is low
• 🌱 outlet: decode IPFIX ingressPhysicalInterface and egressPhysicalInterface
• 🌱 docker: expose Kafka UI (read-only) to the public endpoint
• 🌱 docker: expose Traefik Dashboard (read-only) to the public endpoint
• 🌱 docker: add examples to enable authentication and TLS

Docker image: docker pull ghcr.io/akvorado/akvorado:2.0.0-beta.3
Full changelog: v2.0.0-beta.2...v2.0.0-beta.3

• 💥 common: be stricter on results returned from remote sources
• 🌱 outlet: commit records from Kafka after queuing them to ClickHouse
• 🌱 docker: build a linux/amd64/v3 image to enable some optimizations
• 🌱 docker: build a linux/arm/v7 image
• 🌱 docker: change default log level for ClickHouse from trace to information
• 🌱 docker: switch from Provectus Kafka UI (unmaintained) to Kafbat UI
• 🌱 docker: expose metrics and Kafka UI (read-only) to the public endpoint

Docker image: docker pull ghcr.io/akvorado/akvorado:2.0.0-beta.2
Full changelog: v2.0.0-beta.1...v2.0.0-beta.2