Run large language models on an HPC cluster, and serve them under an SLA.
ALMA turns a SLURM cluster into a managed LLM service. Administrators register a model and launch it as a
cluster job; ALMA places it on a node with free GPUs, opens an SSH tunnel to it, and publishes it on an
OpenAI-compatible gateway. Consumers get an API key governed by a Service Level Agreement — the gateway
enforces the rate limit written in that SLA and answers every request with live X-RateLimit-* headers.
Nothing about the cluster is exposed: the public surface is HTTPS at alma.us.es, with the API under
/engine/*.
flowchart TB
subgraph pub["Public"]
browser["Browser"]
client["API client<br/>apikey header"]
end
caddy["<b>Caddy</b><br/>:443 TLS, HSTS<br/>serves alma-landing"]
front["<b>alma-frontend</b><br/>React + Vite dashboard"]
back["<b>alma-backend</b><br/>:2045 REST API"]
gw["<b>nginx SLA gateway</b><br/>:8080 apikey auth + limits"]
tel["<b>alma-telemeter</b><br/>:2047 /internal/rate-limit"]
redis[("Redis<br/>counters, AOF")]
subgraph cluster["HPC cluster (private network)"]
ctrl["<b>slurm-controller</b><br/>:3000 sbatch / squeue / scancel"]
vllm["<b>vLLM</b><br/>node:9000-9099"]
end
browser -->|"HTTPS"| caddy
caddy --> front
front -->|"REST"| back
back -->|"SSH tunnel"| ctrl
ctrl -->|"submits job"| vllm
client -->|"HTTPS /engine/*"| caddy
caddy -->|"proxy, apikey passed through"| gw
gw -->|"auth_request"| tel
tel <-->|"INCR + TTL"| redis
gw -->|"127.0.0.1:port<br/>one SSH tunnel per model"| vllm
back -.->|"writes nginx conf + OAS,<br/>reloads on change"| gw
back -.->|"opens / reconciles tunnels"| vllm
The control plane (solid, left) launches and supervises models. The data plane (solid, right) serves inference. The dotted edges are what connects them: the backend generates the gateway configuration and keeps one SSH tunnel alive per running model, so publishing a model and routing to it are the same act.
| Repository | What it does | Stack |
|---|---|---|
| alma-frontend | Admin and user dashboard: cluster metrics, model and SLA management, users, chat portal. RBAC for Admin, Researcher, Student and Viewer | React 18, Vite, Tailwind, Radix / shadcn / MUI, Recharts, Vitest + Playwright |
| alma-backend | REST API on :2045. Model lifecycle, SLA generation, nginx configuration, SSH tunnel reconciliation, model auto-restart, operator alerts, multi-device sessions |
Node, Express, NeDB, Docker, Vitest + Newman |
| slurm-controller | Thin HTTP API on :3000 over sbatch, squeue, scancel plus job logs. Runs as a SLURM job on the cluster; alma-backend is its only consumer |
Node, Express, mustache job templates |
| alma-telemeter | Rate-limit accounting behind nginx auth_request on :2047. Atomic per-key, per-endpoint counters in Redis, dynamic X-RateLimit-* headers, OAS telemetry UI |
Express 5, ioredis, chokidar, winston, Jest |
| intelligent-proxy-manager | Controller for SLA creation and updates in ALMA CORE: the Caddy + nginx configuration pipeline and its integration tests | Node, Caddy, nginx, Vitest |
| alma-landing | Public landing page, served as static files by Caddy | Static |
| .github | This profile and organization-wide defaults | — |
slurm-controller first, then alma-backend.
The controller has no CI/CD — it is resubmitted to the cluster by hand — while the backend deploys automatically on a tag. So a backend release lands against whatever controller is already running, and if that one is stale the backend degrades rather than fails: duplicate-launch protection falls back to the local job record, the auto-restart loop stops launching, and the frontend shows an amber SLURM — degraded indicator.
curl -s 'localhost:2045/api/health/slurm?refresh=1' | jq .capabilitiesstatus is one of OK, DEGRADED (stale controller), MISROUTED (the base URL points at some other
service) or UNREACHABLE. capabilities.controllerCommit should match git rev-parse --short HEAD in the
cluster checkout. The distinction between stale and misrouted matters: both 404 the same endpoint, and
only the controller's X-Request-ID header tells them apart.
| Port | Service |
|---|---|
443 |
Caddy — TLS termination, landing page, /engine/* → gateway |
8080 |
nginx SLA gateway — apikey auth, rate limits, JSON error bodies |
2045 |
alma-backend REST API |
2047 |
alma-telemeter, internal rate-limit endpoint |
3000 |
slurm-controller, on the cluster, reached through an SSH tunnel |
9000-9099 |
vLLM remote ports, one per model, on the GPU nodes |
The gateway configuration is not written by hand. It is generated from an OpenAPI description plus SLA4OAI
agreements by sla-wizard and a chain of plugins:
json-errors → auth-request-ratelimit → custom-baseurl → nginx-strip → nginx-confd → sla-wizard core
Each SLA becomes rate limits, per-client routing and an API key in the emitted nginx configuration, which
alma-backend writes and reloads whenever a model or an agreement changes.
| Package | What it adds |
|---|---|
sla-wizard |
Automated configuration of API rates and limits (OpenAPI + SLA4OAI) for Envoy, HAProxy, NGINX and Traefik |
sla-wizard-nginx-confd |
Generates Nginx-Confd configuration files |
sla-wizard-plugin-auth-request-ratelimit |
Replaces nginx limit_req with auth_request-based rate limiting, enabling dynamic X-RateLimit headers via alma-telemeter |
sla-wizard-plugin-custom-baseurl |
Overrides the nginx proxy_pass target per endpoint using the x-nginx-server-baseurl OAS extension |
sla-wizard-plugin-json-errors |
Converts nginx HTML error responses (401, 403, 404, 500, 502, 503, 504) to JSON |
sla-wizard-plugin-nginx-strip |
Strips URL prefixes (x-nginx-strip) before proxying |
sla-wizard-plugin-sla-generator |
Generates SLAs from a CSV file |
These are published on npm and are not repositories of this organization.
Rate limiting stays correct when alma-telemeter is down: nginx keeps its own limit_req as a fallback and
serves static X-RateLimit-* values, so a client is throttled and informed either way.
git clone https://github.com/alma-org/alma-backend.git
cd alma-backend
cp .env.example .env # then edit it
docker compose up -d --buildThe frontend is the same three steps in
alma-org/alma-frontend, and slurm-controller is submitted to
the cluster with ./submit_api.sh rather than run locally.
Each repository's README carries the real procedure — deployment order, environment variables, the SSH tunnel model and the operational failure modes are documented there, particularly in alma-backend and slurm-controller.