Go applications expose a useful set of runtime metrics, but raw /metrics output does not make it easy to spot GC pressure, scheduler latency, memory growth, or file descriptor exhaustion. This post covers a go-mixin for Prometheus and Grafana that adds a dashboard and alerts for the Go runtime.

The mixin is available on GitHub. It currently ships with one Grafana dashboard and three alert rules:

• Go / Overview - A dashboard for runtime CPU usage, scheduler latency, garbage collection, heap churn, mutex contention, cgo activity, and file descriptor pressure.
• GoHighGcCpu - Alerts when a Go process spends too much CPU time in garbage collection.
• GoHighSchedulerLatency - Alerts when runnable goroutines wait too long to be scheduled.
• GoHighFdUsage - Alerts when a Go process is close to its file descriptor limit.

The repo also includes generated dashboard JSON and Prometheus rule files, so you can either vendor the mixin into your Jsonnet setup or import the generated files directly.

Over the years I've written several blog posts covering different parts of Django observability - Django Monitoring with Prometheus and Grafana, Django Development and Production Logging, Celery Monitoring with Prometheus and Grafana, and Django Error Tracking with Sentry. Each post covers one piece: wiring up django-prometheus, configuring structlog, deploying the Celery exporter, setting up distributed tracing. The problem is that wiring all of it by hand across every project is repetitive and easy to get wrong.

django-o11y bundles those patterns into a single installable package. One DJANGO_O11Y settings dict gets you traces, structured logs, Prometheus metrics, and optional Pyroscope profiling - with all four signals correlated on trace_id.

Headscale is an open source, self-hosted control server compatible with the Tailscale clients. It lets you run your own Tailnet and have full control over users, nodes, keys, and routing policies without relying on Tailscale’s hosted control plane. This post introduces the tailscale-exporter and shows how to collect Headscale metrics via the Headscale gRPC API, and visualize everything in Grafana using dashboards and alerts bundled in the mixin.

Envoy is a popular open source edge and service proxy that's widely used in modern cloud-native architectures. Envoy gateway is a controller that manages Envoy proxies in a Kubernetes environment. Monitoring Envoy and Envoy gateway is crucial for ensuring the reliability and performance of your applications. In this blog post, we'll explore how to monitor Envoy and Envoy gateway using Prometheus and Grafana and we'll also introduce a new monitoring-mixin for Envoy.

With the retirement of ingress-nginx, many users are looking for alternatives for ingress controllers. Envoy gateway is a great option for those who want to leverage the power of Envoy in their Kubernetes clusters. I recently migrated from ingress-nginx and you can read more about it here.

Syncthing is a popular open-source file synchronization tool that allows users to securely sync files across multiple devices. Monitoring Syncthing is crucial to ensure that your files are being synchronized correctly and to identify any potential issues. In this guide, we'll explore how to monitor Syncthing using Prometheus and Grafana.

Tailscale is a popular VPN solution that allows you to create secure, encrypted connections between devices. It is based on the WireGuard protocol and is designed to be easy to use and configure. Recently, I've started using Tailscale more extensively both in my personal projects and at work. As a result, I wanted to visualize my Tailnet in Grafana to get better insights into its performance and usage. This post introduces the tailscale-exporter, a tool I built to collect Tailnet metrics directly from the Tailscale API. I’ll also show how to enable scraping of Tailscale client metrics and visualize everything in Grafana for complete observability across your Tailnet.

Cluster autoscaler is a popular tool for automatically adjusting the size of a Kubernetes cluster based on the current workload. It helps ensure that your applications have enough resources to run efficiently while minimizing costs by scaling down unused nodes. However, monitoring the cluster autoscaler is crucial to ensure that it is functioning correctly and that your applications are running smoothly.

KEDA is a tool that provides event-driven autoscaling for Kubernetes, allowing you to scale your applications based on external metrics. It uses the Kubernetes Horizontal Pod Autoscaler (HPA) to adjust the number of pods in a deployment based on metrics like CPU usage, memory usage, or custom metrics from external sources. It also supports scaling based on event sources like message queues, databases as a job and defines a new Custom Resource Definition (CRD) called ScaledJob to configure the scaling behavior. Monitoring KEDA effectively is crucial to ensure that your autoscaling policies are working as expected and that your applications are performing optimally.

Not long ago, I shared how to monitor Kubernetes costs using OpenCost, Prometheus, and Grafana. In this follow-up, I want to highlight a specific feature: setting up proactive budget alerts using OpenCost and Prometheus. It's a simple but powerful way to catch cost overruns early.

Kubernetes events offer valuable insights into the activities within your cluster, providing a comprehensive view of each resource's status. While they're beneficial for debugging individual resources, they often face challenges due to the absence of aggregation. This can lead to issues such as events being garbage collected, the necessity to view them promptly, difficulties in filtering and searching, and limited accessibility for other systems. The blog post explores configuring Loki with Alloy to efficiently scrape Kubernetes events and visualize them in Grafana.