echo-server

A TCP echo server (RFC 862) with a tiny HTTP health listener, shipped as a distroless container image. It runs as a sidecar in Kubernetes so AWS NLB and ALB target group health checks always have a port to hit, no matter what the main container is doing.

The binary is static, built from the Go standard library alone with no third-party dependencies, and is a few megabytes in size.

An NLB target group health check is a TCP handshake (or an HTTP GET) against a port on the target. When the application listens on a port that cannot be probed that way (UDP, a protocol that greets first, a port that is only open after a warm-up), or when several target groups share a pod, a dedicated health check port is simpler than teaching the application about health checks. This sidecar provides that port:

• TCP on :12000 for NLB TCP health checks: connect, optionally send bytes, get them back.
• HTTP on :12001 for ALB health checks, which support only HTTP and HTTPS, and for NLB HTTP checks: every GET returns 200 and {"status":"ok"}.

Health checks close their connections abruptly and arrive from every load balancer node in every availability zone, so the server treats resets, immediate closes and idle connections as normal, logs them at debug level only, and never writes anything unsolicited.

$ docker run -p 12000:12000 -p 12001:12001 ghcr.io/eldhugr/echo-server:latest

In another terminal:

$ nc localhost 12000
ping
ping
^C
$ curl -i localhost:12001/
HTTP/1.1 200 OK
Content-Length: 16
Content-Type: application/json
Date: Sat, 05 Sep 2026 18:46:32 GMT

{"status":"ok"}

To build and run from source:

$ go run ./cmd/echo-server --log.level=debug

The binary starts with no arguments. Every flag can also be set through an environment variable named after it: ECHO_ followed by the flag name in upper case with dots and dashes replaced by underscores. A flag on the command line wins over the environment. Flags accept both --name and -name, and --help prints them all with their environment variables and defaults.

Port 12000 is the port the previous Python sidecar listened on, so existing target groups and security group rules keep working; 12001 is the HTTP listener next to it. Both are above 1024 because the container runs as an unprivileged user (uid 65534, nobody) with no capabilities and cannot bind lower ports.

Either listener can be switched off by setting its address to an empty string, on the command line (--tcp.listen-address="") or in the environment (ECHO_TCP_LISTEN_ADDRESS=). A pod that only sits behind an NLB with a TCP check needs just the TCP listener; one that only sits behind an ALB needs just the HTTP listener. Switching both off is a configuration error: the process prints the usage and exits 2 without starting.

Logs go to stderr. --log.format=logfmt uses the log/slog text handler, json the JSON handler.

Logging. At info the server logs only lifecycle events: starting echo-server, one listening line per listener and echo-server stopped on exit. Everything a health check does is logged at debug: connection opened with the remote address, connection closed with the remote address, bytes echoed, duration and a reason, and for the HTTP listener one request line with method, path, remote address and user agent (ELB-HealthChecker/2.0 for AWS). Run with --log.level=debug to confirm that checks arrive from the load balancer node IPs.

Benign errors. The connection close reasons are peer closed (FIN), peer reset (RST or broken pipe, which is how NLB health checks usually end), idle timeout and server shutdown. None of them is a warning. Any other connection error is logged at warn as connection error. The HTTP server's internal error log (for example TLS handshakes from a misconfigured HTTPS check) is routed to debug. A failing Accept is logged at warn and retried with exponential backoff (5ms doubling up to 1s); the server never exits because of it, since a crash-looping sidecar would make the whole pod unhealthy.

Idle timeout. The deadline is re-armed on every read and write, so --tcp.idle-timeout measures inactivity, not connection lifetime. TCP keepalives (30s) are enabled on accepted connections so peers that disappear without closing are detected.

Connection limit. A slot is taken before each Accept, so when --tcp.max-connections is reached the server simply stops accepting. New connections still complete their TCP handshake in the kernel backlog, so a TCP health check passes even when the server is saturated, and they are served as soon as a slot frees up. Reaching the limit is logged at warn at most once a minute.

Shutdown and exit codes. On SIGTERM or SIGINT both listeners close, open echo connections and in-flight HTTP requests get up to --shutdown-timeout to finish, then whatever is left is closed and logged at warn; for the echo listener that line includes the number of connections that were force-closed. A clean shutdown exits 0. A listener that fails to start (for example a port that is already in use) is logged at error, the other listener is drained and the process exits 1. An invalid flag or environment variable, or a configuration with both listeners disabled, prints the usage and exits 2.

The HTTP listener answers GET and HEAD on any path with 200, Content-Type: application/json and {"status":"ok"} (no body for HEAD); other methods get 405 with an Allow header. The Host header is ignored: ALB sends the target IP, NLB sends the load balancer node IP.

Run echo-server as a native sidecar: an initContainers entry with restartPolicy: Always (Kubernetes 1.29 or newer). It starts before the main container, is restarted if it exits and keeps running until the main containers have stopped, so the health check port stays open for the whole life of the pod. On older clusters put it in the regular containers: list instead; it then runs as an ordinary sidecar without the startup and shutdown ordering guarantees.

Expose the health check port as a containerPort and give the sidecar its own readiness probe (tcpSocket on 12000, or httpGet on 12001 when the TCP listener is off) so the pod is not registered as a target before the sidecar is listening. The image runs as uid 65534 and needs no writable files, so a securityContext with runAsNonRoot, readOnlyRootFilesystem, all capabilities dropped and the RuntimeDefault seccomp profile works, which is what the Pod Security Standards restricted profile requires. A few millicores and 16Mi of memory are plenty.

Point the load balancer health check at the sidecar rather than the application port. With the AWS Load Balancer Controller, a LoadBalancer Service for an NLB takes the service.beta.kubernetes.io/aws-load-balancer-healthcheck-protocol: tcp and service.beta.kubernetes.io/aws-load-balancer-healthcheck-port: "12000" annotations (or protocol http, port 12001 and path / for an HTTP check; NLB accepts 200-399 by default). An Ingress for an ALB takes alb.ingress.kubernetes.io/healthcheck-protocol: HTTP, alb.ingress.kubernetes.io/healthcheck-port: "12001" and alb.ingress.kubernetes.io/healthcheck-path: /. ALB target groups do not support TCP health checks, which is what the HTTP listener is for. If a listener is moved to another port, change the containerPort, the probe and the health check annotation to match, or the pod never becomes Ready and the check is refused.

The health check port must be reachable from the load balancer nodes. The controller adds the security group rule for Services and Ingresses it manages. With a TargetGroupBinding the health check itself is configured on the target group outside Kubernetes, and spec.networking.ingress[].ports must list both the traffic port and the health check port (leaving ports unset allows all TCP ports).

The image has no shell. To poke at a running sidecar, attach an ephemeral container:

$ kubectl debug -it pod/<pod> --image=busybox:1.37 --target=echo-server -- sh
/ # nc 127.0.0.1 12000
/ # wget -qO- http://127.0.0.1:12001/

For a local image with a busybox shell, build from the debug distroless variant:

$ docker build --build-arg DISTROLESS_TAG=debug -t echo-server:debug .
$ docker run --rm -it --entrypoint sh echo-server:debug

--log.level=debug prints every connection and request.

$ CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" ./cmd/echo-server
$ go test -race ./...
$ docker build -t echo-server .

The module has no dependencies outside the Go standard library, so there is no go.sum and nothing to download. -s -w strips the symbol table and DWARF data; together with -trimpath and CGO_ENABLED=0 this is what the Dockerfile runs. The image is cross-compiled in golang:1.27.1-alpine3.24 and runs on gcr.io/distroless/static-debian13 as uid 65534 (nobody) with no shell. Tagged releases (vX.Y.Z) are built as multi-arch images (linux/amd64, linux/arm64) and pushed to ghcr.io/eldhugr/echo-server by CI.

Apache 2.0