Agnes Infrastructure Template

Template for deploying Agnes — an open-source AI Data Analyst platform — into your own GCP project.

Use this template to create your own private deployment repo:

gh repo create <your-org>/agnes-infra --template keboola/agnes-infra-template --private

Then follow the steps below.

• GCP project with billing enabled (you own it, you pay)
• gcloud CLI authenticated as Owner of that project
• terraform ≥ 1.5
• GitHub account (for private repo + Actions)

Download and run the bootstrap script from the public Agnes repo:

curl -fsSL https://raw.githubusercontent.com/keboola/agnes-the-ai-analyst/main/scripts/bootstrap-gcp.sh -o bootstrap-gcp.sh
chmod +x bootstrap-gcp.sh
./bootstrap-gcp.sh <YOUR_GCP_PROJECT_ID>

This creates:

• Service Account agnes-deploy@<project>.iam.gserviceaccount.com with Terraform-scoped roles
• GCS bucket agnes-<project>-tfstate with versioning
• Enables required APIs (compute, iam, secretmanager, storage, iamcredentials)
• Generates a SA JSON key — you'll need this in Step 3.

echo -n "<YOUR_KEBOOLA_STORAGE_TOKEN>" | gcloud secrets create keboola-storage-token \
    --data-file=- --replication-policy=automatic --project=<YOUR_GCP_PROJECT_ID>

gh secret set GCP_SA_KEY < path/to/agnes-deploy-<project>-key.json
rm path/to/agnes-deploy-<project>-key.json   # ⚠ NEVER COMMIT

Edit terraform/main.tf:

• backend.bucket = agnes-<your-project>-tfstate
• backend.prefix = <your-customer-name>

Copy terraform/terraform.tfvars.example → terraform/terraform.tfvars, fill in values.

Go to: Settings → Environments → New environment.

Create dev: no protection rules.

Create prod:

• Required reviewers: add yourself or your ops team
• Wait timer: 5 minutes
• Deployment branches: Selected branches and tags → main

cd terraform
export GOOGLE_APPLICATION_CREDENTIALS=path/to/agnes-deploy-key.json
terraform init
terraform plan
terraform apply

Or push the tfvars to main and let GitHub Actions do it (after step 4).

Output will show prod_ip.

PROD_IP=$(terraform output -raw prod_ip)
curl -X POST http://$PROD_IP:8000/auth/bootstrap \
    -H "Content-Type: application/json" \
    -d '{"email": "you@example.com", "password": "YOUR_STRONG_PASSWORD"}'

Then open http://<prod_ip>:8000/login and sign in.

Terraform is the source of truth. Do not hardcode IPs anywhere — they persist across VM replacement (static addresses), but documenting them manually goes stale when anyone recreates the infra.

cd terraform
export GOOGLE_APPLICATION_CREDENTIALS=path/to/agnes-deploy-key.json

# Single prod IP
terraform output -raw prod_ip
# 34.77.102.61

# Map of all instances (prod + every dev_instances entry)
terraform output -json instance_ips | jq
# {
#   "agnes-prod": "34.77.102.61",
#   "agnes-dev":  "34.77.94.14"
# }

# JWT secret reference (for app restarts after rotation)
terraform output -raw jwt_secret_name

# Daily backup policy (for 'gcloud compute snapshots list' filtering)
terraform output -raw backup_policy_id

Alternative (without Terraform, direct from GCP):

gcloud compute addresses list \
    --project=<YOUR_GCP_PROJECT_ID> \
    --filter="name~agnes-" \
    --format="table(name, address, status, users)"

Or via GCP Console: https://console.cloud.google.com/networking/addresses/list?project=<YOUR_GCP_PROJECT_ID>

When a VM is recreated (e.g. terraform apply -replace=module.agnes.google_compute_instance.vm["agnes-prod"]), the static IP is preserved — the same address is reattached to the new VM. URLs do not change.

If you have a custom domain, point a DNS A-record at the static IP once; all future VM replacements keep the same address.

Automatic via cron (every 5 minutes) if upgrade_mode = "auto" in tfvars. Otherwise, manual:

gcloud compute ssh agnes-prod --zone=... --project=... --command="sudo /usr/local/bin/agnes-auto-upgrade.sh"

Update the module ref in terraform/main.tf:

source = "github.com/keboola/agnes-the-ai-analyst//infra/modules/customer-instance?ref=infra-v1.X.Y"

Open a PR → terraform plan runs → review → merge → terraform apply runs (dev auto, prod gated).

Edit terraform.tfvars:

dev_instances = [
  { name = "agnes-dev",           image_tag = "dev" },
  { name = "agnes-alice-feature", image_tag = "dev-feature-alice-xyz" },
]

PR → plan → merge → new dev VM spawned. No GitHub env / SA changes needed.

In addition to Terraform-managed infrastructure, this template ships content that an Agnes server pulls from your fork: the initial analyst workspace, your curated marketplace, and the install-prompt template. Each lives in its own sub-tree and is registered with the Agnes server via a different admin endpoint. The three contracts are independent — the workspace parser only reads workspace/, the marketplace parser only reads .claude-plugin/ + plugins/, and the install-prompt loader only reads install-prompt/.

Shipped to analyst laptops on agnes init. The template ships a vendor-agnostic baseline — copy this fork into your own private infra repo and customize the placeholder content (CLAUDE.md rules, AGNES_WORKSPACE.md commands) to match your deployment.

Sync workflow: after editing under workspace/, an admin clicks Sync now at /admin/server-config on Agnes. Analysts pick up new content on their next agnes init --force.

Sync workflow: Agnes scheduler clones nightly at 03:00 UTC. Manual re-sync via Sync now at /admin/marketplaces.

The Mustache-style template Agnes renders into the one-page install prompt an analyst sees after admin onboarding. Variables ({{server_url}}, {{credential_email}}, …) are filled in server-side per tenant. Customize the wording to match your onboarding tone; the template references the connector skills directly so the analyst's first Claude Code session knows which connectors to enable.

.
├── terraform/                   # Infra-as-code
│   ├── main.tf                  #   Module reference; don't customize (PR upstream module changes instead)
│   ├── variables.tf             #   Input schema
│   ├── terraform.tfvars.example #   Copy to terraform.tfvars (gitignored)
│   └── .gitignore
├── .github/workflows/
│   ├── plan.yml                 # PR trigger: terraform plan, comment on PR
│   └── apply.yml                # main push: apply-dev → apply-prod (gated)
├── workspace/                   # Initial Workspace Template — shipped on `agnes init`
│   ├── CLAUDE.md                #   Project rules (customize for your team)
│   ├── AGNES_WORKSPACE.md       #   Human-readable workspace docs
│   └── .claude/
│       ├── settings.json        #   Hooks + permissions + statusLine
│       ├── CLAUDE.local.md      #   Stub for analyst personal notes
│       ├── commands/            #   /agnes-private + /update-agnes-plugins
│       └── skills/              #   Connector skills (asana, atlassian, gws)
├── .claude-plugin/
│   └── marketplace.json         # Curated marketplace manifest (skeleton ships empty)
├── plugins/                     # Curated Claude Code plugins (skeleton ships empty)
├── install-prompt/
│   └── template.md.tmpl         # Install-prompt template rendered per tenant
├── config/                      # Customer-specific overrides (instance.yaml, branding)
└── README.md

• GCP_SA_KEY (GitHub secret) — SA JSON key with Terraform-scoped roles
• keboola-storage-token (GCP Secret Manager, manual create) — only if data_source = keboola
• agnes-<customer>-jwt-secret (GCP Secret Manager, TF-managed) — auto-generated

• JWT secret rotates on terraform apply -replace=module.agnes.google_secret_manager_secret_version.jwt (invalidates all sessions).
• Keboola token rotation: gcloud secrets versions add keboola-storage-token + sudo docker compose restart on each VM.
• SA key rotation: generate new key, update GCP_SA_KEY secret, delete old key in GCP IAM.

File issues in keboola/agnes-the-ai-analyst.