Warning
AX and several of its features are in heavy development. We are actively refining our core concepts, protocols, and specifications, and will likely introduce major breaking changes prior to a stable release.
Declare an agentic task with workspaces and model specifications. AX sandboxes it, wires up its workspace, and helps running it at scale.
AX is a highly efficient, declarative orchestrator to run millions of autonomous agent workloads in a cluster. It runs on top of Agent Substrate for sandboxed execution and is built to run millions of tasks per cluster. If you have used Kubernetes, ax will feel similar.
# task.yaml
apiVersion: ax.io/v1alpha1
kind: Workspace
metadata:
name: golang
spec:
setup:
git:
- repo: https://github.com/golang/go.git
ref: "my-fix"
files:
- path: "AGENTS.md"
content: |
- Run `go test` to ensure that tests are passing.
- Do not modify the public API.
---
apiVersion: ax.io/v1alpha1
kind: Task
metadata:
name: test
spec:
profile:
image: "ghcr.io/my-org/my-coding-agent@sha256:..."
goal:
text: "Fix what's broken please."
workspace:
name: golang
debug: true # enables `ax ssh`Then apply it, resume it, and look over the agent's shoulder:
ax apply -f task.yaml
ax resume task test
ax describe task test
ax ssh test -- ls -al /workspaceAgents are a new kind of workload. They are neither stateless microservices nor run-to-completion batch jobs. They accumulate state, need strict isolation, call out to model APIs and tool servers, and can burn money in a loop if nobody is watching. AX gives you three small primitives that handle all of that declaratively:
| You want to... | AX gives you |
|---|---|
| Run untrusted agent code in an isolated sandbox with CPU/memory limits | Task |
| Pre-wire Git repos, MCP servers, and skill packages so every agent starts warm | Workspace |
| Configure which LLM the platform itself uses, with credentials from a Kubernetes secret | Model |
| Pause an idle agent and pick up exactly where it left off | ax suspend / ax resume |
| Shell into a running agent to see what it is doing | ax ssh |
Everything is expressed as ax.io/v1alpha1 manifests and applied with a single command.
If you are deploying to a Kubernetes cluster (or see docs/installation.md for the full version-pinned audit recipe), you need:
- A Kubernetes 1.36+ cluster with Agent Substrate installed (see the Substrate README)
- Go and
kubectl - Podman or Docker, authenticated to a container registry your cluster can pull from
Substrate lands in the ate-system namespace and exposes its Control API at api.ate-system.svc.cluster.local:443. Verify it is up before moving on:
kubectl get svc api -n ate-systemmake install # installs `ax` from this checkout into $(go env GOPATH)/binPoint AX_IMAGE_REPO at your registry, then build the ax-server and ax-task-runner images and push them to it:
export AX_IMAGE_REPO=<your-registry> # e.g. gcr.io/my-project
make imagesUse ax install so ax-system is labeled (ate.dev/api-access=true) and ax-server is registered in Substrate's caller allowlists before deploying PostgreSQL and ax-server:
ax install \
--server-image "$AX_IMAGE_REPO/ax-server:latest" \
--default-task-image "$AX_IMAGE_REPO/ax-task-runner:latest"This deploys PostgreSQL and the ax-server image you just pushed, configured to use your ax-task-runner image for tasks that do not set spec.profile.image. Everything lands in the ax-system namespace.
Every task runs as a Substrate actor, and an actor needs a free worker from a WorkerPool. (ax local start creates one automatically; on an existing cluster neither ax install nor the Substrate installer creates one for you, so without a pool the first ax resume fails with ResourceExhausted: no free workers available.)
Create a small gVisor pool from examples/workerpool.yaml (its header explains how to apply it), or follow the WorkerPool step in the installation guide. Then wait until kubectl get workerpools -A shows the workers ready.
From your ax checkout:
ax apply -f examples/simple.yaml # the smallest useful Task
ax resume task simple-task # new tasks start Suspended; resume schedules the actor and starts it
ax get tasks
# NAME ATESPACE STATE ACTOR AGE
# simple-task default Working simple-task 1m
ax ssh simple-task -- ls -la /workspace # poke around inside the sandbox
ax suspend task simple-task # checkpoint and pause
ax resume task simple-task # pick up where it left offThe first resume also waits for the workspace to initialize (cloning its git repos), so it can take a while; if the CLI gives up first, the task keeps coming up in the background — ax get tasks tells you where it is.
Want to see the whole lifecycle end to end? Run ./demo.sh. It applies a custom workspace, resumes the task, runs commands over ax ssh, and suspends it again.
| Guide | Read it to... |
|---|---|
| Local Installer | Walk through ax local, ax ctx, ax install, and promoting tasks unchanged from local Kind to a remote GKE cluster. |
| Concepts | Learn what a Task, Workspace, and Model each do, and how a task moves through phases and conditions. |
| Manifests | Write your own YAML, with an annotated example of every kind. |
| Sandbox | See what the runner does on boot and what your command can rely on: metadata server, guest services, environment. |
| Runners | Understand the contract between the control plane and the task container, and build your own runner image to replace the default. |
| Networking | Reach a running task through the atenet router from the cluster, your laptop, or a gRPC client. |
| Observability | Collect ax-server's logs, metrics, traces, and events, and configure where they go. |
| Architecture | Understand how the control plane fits together, plus the API reference. |
| Development | Build, test, and ship changes to AX itself. |
| Roadmap | See planned milestones across core specs, actor architecture, agentic environments, and governance. |
ax talks to the control plane over gRPC. It is deliberately kubectl-shaped: apply, get, describe, delete, plus a few agent-specific verbs.
# Apply anything (multi-document YAML, file or stdin)
ax apply -f examples/simple.yaml
# Tasks
ax get tasks # list
ax get tasks -a my-atespace # list in another atespace
ax get task task123 # full spec + live status as YAML
ax describe task task123 # human-readable detail
ax suspend task task123 # checkpoint actor state and pause
ax resume task task123 # resume a suspended task
ax promote task task123 --to prod # copy the task and what it binds to the cluster of kube context prod, and resume it there
ax delete task task123
# Shell into the running sandbox
ax ssh task123 # interactive shell (task needs spec.debug: true)
ax ssh task123 -- ls -la /workspace # one-off command
ax ssh task123 -- python3 main.py
# Workspaces, gateways, and models follow the same pattern
ax get workspaces
# NAME ATESPACE GIT-REPOS MCP-SERVERS
# default-workspace default 1 1
ax describe workspace default-workspace
ax delete workspace default-workspace
ax get gateways
# NAME ATESPACE EGRESS-RULES
# review-bot-gw default 3
ax describe gateway review-bot-gw
ax delete gateway review-bot-gw
ax get models
# NAME ATESPACE PROVIDER MODEL
# default-model default google gemini-3.8-flash
ax describe model default-model
ax delete model default-model
# Environment & connection plumbing
ax local start --server-image <img> # create/resume local Kind + Substrate + AX cluster
ax local status # check local environment, Substrate, and AX health
ax local stop # stop local environment while keeping state
ax local delete # destroy local environment and clean up owned context
ax install / ax uninstall # install or remove AX on the active Kubernetes cluster
ax studio # launch the AX Studio local Web UI on 127.0.0.1:8080
ax ctx # list kube contexts and active AX tunnel status
ax ctx <name> # switch active Kubernetes context in kubeconfig
ax tunnel list # background tunnels (state lives in ~/.ax/tunnels)
ax tunnel stop
ax versionax follows your active Kubernetes context. Switch clusters and ax resolves and tunnels to that cluster's control plane in the background.
kubectx staging-cluster
ax get tasks
kubectx prod-cluster
ax get tasks
# Or target a context without switching
ax --context=dev-cluster get tasks| Flag | Description | Default |
|---|---|---|
-a, --atespace |
Atespace scope for the command | default |
-n, --namespace |
Kubernetes namespace where AX is installed | ax-system |
--context |
Kubernetes context to target | active kubectx / current-context |
--server |
Control plane address, bypassing auto-detection | derived from kube context, or $AX_SERVER |
--trace |
Trace the command and print its trace ID to stderr. Its RPCs send a sampled W3C traceparent, and CLI spans are exported over OTLP when OTEL_EXPORTER_OTLP_ENDPOINT is set. Without it, the CLI sends no trace context and ax-server samples on its own. See CLI traces |
off |
Apache License 2.0. See LICENSE for details.