Skip to content
Mmx233Public

About

Robust, QUIC-backed NAT traversal tool

Topics

Resources

Stars

5 stars

Watchers

0 watching

Forks

Repository files navigation

QMux

Lisense Test Status Release Artifact Hub Dockerhub

A high-availability L4 (transport layer) NAT traversal tool built on the QUIC protocol.

QMux operates at Layer 4 of the OSI model, enabling secure TCP/UDP tunneling through NAT/firewalls by establishing QUIC connections between clients and servers. It works at the transport layer, forwarding raw TCP streams without inspecting application-layer protocols. It supports multi-server configurations with automatic load balancing, mTLS or token authentication, and TLS session resumption for fast reconnections.

Get Started

Installation

From Binary

Download the latest release from GitHub Releases.

From Source

go install github.com/Mmx233/QMux@latest

Docker

docker pull mmx233/qmux

Helm

Get Repo Info

helm repo add qmux https://mmx233.github.io/QMux
helm repo update

Install Chart

Set the role and its QMux configuration directly through chart values. See charts/qmux/values.yaml for the complete options.

helm install qmux-client qmux/qmux \
  --namespace qmux \
  --create-namespace \
  --set mode=client \
  --set 'config.client.server.servers[0].address=qmux.example.com:8443' \
  ...

Quick Start

1. Generate Certificates

QMux uses mTLS for authentication by default. Generate the required certificates for testing:

qmux generate certs -s your-server-domain.com

The -s flag specifies the server DNS name for the certificate (required). You can specify multiple names:

qmux generate certs -s example.com -s qmux.example.com

Certificates are valid for 10 years by default. Use -y to customize:

qmux generate certs -s example.com -y 5  # 5 years validity

This creates in pwd:

  • certs/ca.crt - CA certificate
  • certs/server.crt / certs/server.key - Server certificate
  • certs/client.crt / certs/client.key - Client certificate

2. Generate Configuration Files

# Generate server config
qmux generate config server -o server.yaml

# Generate client config
qmux generate config client -o client.yaml

3. Configure Example Server

Edit server.yaml:

admin_address: "127.0.0.1:9090" # Optional; omit to disable admin endpoints

tcp_copy_buffer_size: 131072 # Bytes; shared by this server root's TCP relays

listeners:
  - quic_addr: "0.0.0.0:8443"    # QUIC control port
    traffic_addr: "0.0.0.0:8080" # Traffic forwarding port
    protocol: "both"              # tcp, udp, or both
    capacity:
      max_client_generations: 16
      max_pending_registrations: 128
      max_tcp_connections: 128
      max_pending_tcp_setups: 128
      max_tcp_connections_per_generation: 100
      max_pending_tcp_setups_per_generation: 16
      max_udp_sessions: 1024
      max_udp_sessions_per_generation: 256
      max_udp_sender_queued_frames_per_generation: 256
      max_udp_sender_queued_backing_bytes_per_generation: 524288
      max_udp_fragment_groups: 4096
      max_udp_fragment_backing_bytes: 67108864

auth:
  method: "mtls"
  ca_cert_file: "./certs/ca.crt"

tls:
  server_cert_file: "./certs/server.crt"
  server_key_file: "./certs/server.key"
  auto_reload: false

4. Configure Example Client

Edit client.yaml:

admin_address: "127.0.0.1:9090" # Optional; omit to disable admin endpoints

tcp_copy_buffer_size: 131072 # Bytes; shared by this client root's TCP relays

capacity:
  max_local_udp_sessions: 256
  max_udp_fragment_groups_per_handler: 4096
  max_udp_fragment_backing_bytes_per_handler: 67108864

server:
  servers:
    - address: "your-server-ip:8443"
      server_name: "your-server-domain.com"

local:
  host: "127.0.0.1"
  port: 3000  # Your local service port

auth:
  method: "mtls"

tls:
  ca_cert_file: "./certs/ca.crt"
  client_cert_file: "./certs/client.crt"
  client_key_file: "./certs/client.key"
  auto_reload: false
Key Scope Default When full
tcp_copy_buffer_size Client root: all TCP relays; server traffic-manager root: all listeners and TCP relays (bytes) 131072 N/A
max_client_generations Server listener: client generations 16 Reject the new client registration.
max_pending_registrations Server listener: registrations not yet committed 128 Close the new registration connection.
max_tcp_connections Server listener: pending and active TCP flows 128 Close the new TCP connection.
max_pending_tcp_setups Server listener: TCP flows still setting up 128 Close the new TCP connection.
max_tcp_connections_per_generation Server listener: pending and active TCP flows on one client generation 100 Try another eligible generation; otherwise close the new TCP connection.
max_pending_tcp_setups_per_generation Server listener: pending TCP setups on one client generation 16 Try another eligible generation; otherwise close the new TCP connection.
max_udp_sessions Server listener: server-side UDP sessions 1024 Drop the datagram that would create a new session.
max_udp_sessions_per_generation Server listener: server-side UDP sessions on one client generation 256 Try another eligible generation; otherwise drop the new-session datagram.
max_udp_sender_queued_frames_per_generation Server listener: queued and worker-held frames for one client generation 256 Drop the new datagram batch.
max_udp_sender_queued_backing_bytes_per_generation Server listener: queued and worker-held full pooled datagram backing for one client generation (bytes) 524288 Drop the new datagram batch.
max_udp_fragment_groups Server listener: incomplete UDP fragment groups 4096 Drop the fragment that would create a new group.
max_udp_fragment_backing_bytes Server listener: retained UDP fragment backing (bytes) 67108864 Drop the fragment that would exceed the limit.
max_local_udp_sessions Client process: local UDP sessions shared by all UDP handlers 256 Drop the datagram that would create a new local session.
max_udp_fragment_groups_per_handler Client UDP handler (connection generation): incomplete fragment groups 4096 Drop the fragment that would create a new group.
max_udp_fragment_backing_bytes_per_handler Client UDP handler (connection generation): retained fragment backing (bytes) 67108864 Drop the fragment that would exceed the limit.

All sizes are integer bytes. Omitting any listed setting or using 0 selects its default; negative values are rejected, and there is no unlimited setting. The TCP copy pools are immutable per root; a standard CLI invocation creates one root for its selected client or server role. max_local_udp_sessions is one process-wide client budget, while each client handler has independent fragment limits. Server capacity settings are independent per listeners entry, and sender limits include queued and worker-held frames for each client generation. When a gate is full, QMux rejects or drops only new work; it does not evict registered generations, established TCP connections, or existing UDP sessions.

The per-generation TCP gates are independent: max_tcp_connections_per_generation (100 by default) counts pending and active connections, while max_pending_tcp_setups_per_generation (16 by default) counts only connections still being established. A pending TCP setup remains counted through backend dial and the NewConn ACK round trip, so size this limit for peak concurrent setup latency; success activates it and failure or the setup deadline releases it.

For multi-listener or multi-generation deployments, aggregate memory bounds multiply by the number of active client handlers, listeners, and client generations. Calculate the aggregate budget from the actual topology and workload, then validate the limits under that topology.

5. Run

# Start server (on public server)
qmux run server -c server.yaml

# Start client (on machine behind NAT)
qmux run client -c client.yaml

Now external traffic to your-server-ip:8080 will be forwarded to your local service on port 3000.

The optional admin_address enables unauthenticated GET /healthz, GET /readyz, and GET /metrics endpoints on both server and client, so bind it only to loopback or a protected management network. Server readiness requires every configured route to be listening with an eligible client for each enabled protocol. Client readiness requires a healthy registered connection to every configured server endpoint. Failed readiness returns 503 not ready; liveness returns 200 ok while the admin listener is running. These endpoints do not probe the local service; use a business-port blackbox check for the end-to-end data path.

GET /metrics exposes Prometheus metrics for runtime, connections, and traffic. QMux metrics use the qmux_server_ and qmux_client_ prefixes.

Authentication

QMux uses mTLS by default. In this mode the server and client verify one another with certificates, as shown in the quick-start configuration above.

Token authentication is available when distributing a client certificate is undesirable. Generate a machine-random secret with at least 256 bits of entropy, for example with openssl rand -base64 32, and configure the same value on both sides:

# server.yaml
auth:
  method: "token"
  token: "replace-with-the-generated-secret"

tls:
  server_cert_file: "./certs/server.crt"
  server_key_file: "./certs/server.key"
# client.yaml
auth:
  method: "token"
  token: "replace-with-the-generated-secret"

tls:
  ca_cert_file: "./certs/ca.crt"

The token itself is never sent over the wire. After the TLS 1.3 handshake completes, the client sends an HMAC-SHA512 registration proof bound to that connection through the TLS exporter. The client still validates the server certificate using ca_cert_file, while token mode neither requires nor sends a client certificate. A proof captured from one connection cannot authenticate another connection, including a resumed TLS session.

Session ticket key rotation

By default, QMux leaves session ticket key rotation to Go. In the Go 1.27 baseline, Go rotates keys every 24 hours and expires them after 7 days; Go owns this automatic policy.

Set tls.session_ticket_encryption_key_rotation_interval to a positive duration to use QMux's custom rotation. tls.session_ticket_encryption_key_rotation_overlap is the number of old keys retained: omission or null uses the fixed default of 7 old keys (8 total), while numeric 0 retains no old keys. The overlap setting is invalid when the interval is zero or omitted.

Performance

Test Environment

Component Specification
QMux hosts Separate c6i.large server/client and separate c6i.4xlarge server/client
Traffic hosts Separate c6i.4xlarge load generator and backend
Network Same-AZ private VPC; ENA
Software QMux v2.2.1; Go 1.27.1; Amazon Linux 2023; stripped Linux/amd64/v1 build
Security Mutual TLS; TLS 1.3; one QUIC connection per client

Transport Throughput

20 s × 3; medians. UDP: 1,200-byte/P1 at ≤0.1% median loss.

QMux server/client TCP forward P1 TCP forward P4 TCP reverse P1 TCP reverse P4 UDP forward delivered / loss
c6i.large 3.777 Gbit/s 3.457 Gbit/s 3.716 Gbit/s 3.492 Gbit/s 499.7 Mbit/s / 0.053%
c6i.4xlarge 5.845 Gbit/s 5.792 Gbit/s 5.866 Gbit/s 5.520 Gbit/s 999.8 Mbit/s / 0.019%

Small-Packet UDP

QMux server/client Payload / parallel Delivered Packet rate Loss
c6i.large 256-byte / P4 100.0 Mbit/s 48.8 kpps 0%
c6i.4xlarge 256-byte / P4 499.7 Mbit/s 244.0 kpps 0.064%

TCP reached host bandwidth limits; formal UDP points had no host drops.

HTTP Service Workloads

Request path: user → QMux server → QMux client → nginx. Concurrency 32; 30 s × 3; medians.

QMux server/client Workload Requests/s (min-max) p50 p99 Errors / requests
c6i.large 1 KiB, keep-alive 50,145 (50,095-50,783) 0.624 ms 1.059 ms 0 / 4,530,822
c6i.large 1 KiB, new connection 8,009 (8,005-8,046) 3.940 ms 6.521 ms 0 / 721,859
c6i.large 64 KiB, keep-alive 5,570 (5,122-5,578), 2.920 Gbit/s 5.728 ms 7.910 ms 3 / 501,084
c6i.4xlarge 1 KiB, keep-alive 70,011 (69,986-70,159) 0.452 ms 0.676 ms 1 / 6,304,755
c6i.4xlarge 1 KiB, new connection 24,495 (24,461-24,532) 1.266 ms 2.355 ms 0 / 2,204,724
c6i.4xlarge 64 KiB, keep-alive 9,109 (8,495-9,287), 4.776 Gbit/s 3.350 ms 4.790 ms 21 / 835,257

Low-rate keep-alive socket errors remained in the three-run samples and are reported rather than discarded.

Ten-Minute Stability

One 600 s connection per row. Delivered/loss: final 5 min. CPU: full run, client+server. RSS: maximum client+server sum across three checkpoints.

QMux server/client Workload Final 5 min delivered Loss / retransmits Combined CPU-s/GiB Maximum combined RSS
c6i.large TCP forward P4 3.360 Gbit/s 8 tail retransmits 7.73 42.0 MiB
c6i.large UDP 1,200 B P1, 500 Mbit/s offered 499.552 Mbit/s 0.0900% loss 43.01 38.0 MiB
c6i.4xlarge TCP forward P4 5.699 Gbit/s 1 tail retransmit 7.12 46.2 MiB
c6i.4xlarge UDP 1,200 B P1, 1 Gbit/s offered 999.919 Mbit/s 0.0079% loss 32.85 40.4 MiB

Client Failover

2 × c6i.4xlarge clients; least-connections; 100 new TCP connections/s; SIGKILL after ten clean 1 s bins; recovery = three clean bins.

Metric Result
Confirmed recovery time 30.165 s median (26.149-31.238 s)
Start of stable zero-error period 28.116 s median (24.100-29.189 s)
Failed requests per 5,000-request trial 30 median (28-42)

New TCP flows only; established connections are not migrated.

High Available Architecture

System Overview

flowchart TD
    subgraph External["External Traffic"]
        direction LR
        User1[Users :80]
        User2[Users :443]
    end

    subgraph LBLayer["External LB / DNS"]
        direction LR
        LB1[LB :80]
        LB2[LB :443]
    end

    subgraph ServerCluster["QMux Server Cluster"]
        direction LR
        S1[Server 1<br/>QUIC:8443,8444<br/>Traffic:80,443]
        S2[Server 2<br/>QUIC:8443,8444<br/>Traffic:80,443]
    end

    subgraph ClientLayer["QMux Client Pods"]
        direction LR
        subgraph ClientGroup1["Client Group A (Port 80)"]
            C1A[Client Pod 1]
            C1B[Client Pod 2]
        end
        subgraph ClientGroup2["Client Group B (Port 443)"]
            C2A[Client Pod 1]
            C2B[Client Pod 2]
        end
    end

    subgraph Services["Internal Services"]
        direction LR
        SVC1[Service A<br/>:3000]
        SVC2[Service B<br/>:5000]
    end

    User1 --> LB1
    User2 --> LB2
    LB1 --> S1 & S2
    LB2 --> S1 & S2

    S1 & S2 <-.->|QUIC :8443| C1A & C1B
    S1 & S2 <-.->|QUIC :8444| C2A & C2B

    C1A & C1B --> SVC1
    C2A & C2B --> SVC2
Loading

Data Tunnel Flow

sequenceDiagram
    participant User as External User
    participant LB as L4 Load Balancer
    participant Server as QMux Server
    participant Pool as Connection Pool
    participant Client as K8s Client Pod
    participant SVC as K8s Service

    User->>LB: TCP Connection
    LB->>Server: Forward to healthy server
    Server->>Pool: BeginTCPAdmission()
    Pool-->>Server: TCPAdmission

    loop Retryable candidate attempts within setup budget
        Server->>Pool: Next() - Round Robin / Least Connection
        Pool-->>Server: TCPLease + ClientConn
        Server->>Client: OpenStream() via existing QUIC conn
        Server->>Client: NewConnMsg{ConnID, Protocol, SourceAddr}

        Client->>SVC: Dial internal service
        alt Setup succeeds
            SVC-->>Client: Connected
            Client->>Server: NewConnAckMsg{ConnID}
            Server->>Pool: TCPLease.Commit()

            Note over User,SVC: Bidirectional Tunnel Established

            par Data Flow
                User->>LB: Request Data
                LB->>Server: Forward
                Server->>Client: Stream Data
                Client->>SVC: Forward
            and
                SVC->>Client: Response
                Client->>Server: Stream Data
                Server->>LB: Forward
                LB->>User: Response
            end
            Server->>Pool: TCPLease.Release()
        else Candidate setup fails
            Server->>Pool: TCPLease.Release()
            alt Retryable, budget remains, and another candidate is available
                Note over Server,Pool: Continue with Next()
            else Deadline, shutdown, or candidates exhausted
                Server-->>LB: Close current public TCP request
            end
        end
    end
Loading

About

Robust, QUIC-backed NAT traversal tool

Topics

Resources

Stars

5 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages