Skip to content

Latest commit

 

History

History

Folders and files

README.md

Vivaria Jumphost

SSH jumphost for accessing internal services via Tailscale.

Users

User Shell Purpose
ssh-admin /bin/ash Administrative access, can add/remove keys
ssh-user /sbin/nologin Hawk-managed sandbox access only

Building and Deploying

The jumphost is the Pulumi component metr:core:Jumphost in infra/core/jumphost.py. When hawk:tailscaleAuthKeysSecretArn is configured on a full non-dev stack (normally stg), pulumi up builds this directory's Docker image, pushes it to the stack's ECR repository, and deploys the ECS Fargate service. Dev stacks reuse staging's jumphost references; they do not create another jumphost. There is no separate build script or Terraform/OpenTofu deployment.

# Log in to Hawk's S3-backed Pulumi state first; see AGENTS.md.
export PULUMI_FALLBACK_TO_STATE_SECRETS_MANAGER=true
export STACK=stg
pulumi up -s "$STACK"

Manual Testing

Set ENVIRONMENT to the owning stack name, normally stg:

ENVIRONMENT=stg
export AWS_REGION="$(pulumi stack output region -s "$ENVIRONMENT")"

Get NLB DNS

If the jumphost is not enabled for the stack, this output is empty.

NLB_DNS=$(pulumi stack output jumphost_nlb_public_dns -s "$ENVIRONMENT")

SSH as Admin

Pulumi generates the admin key and stores it in Secrets Manager; it is not one of your normal SSH keys. Retrieve it into a mode-0600 temporary file. The AWS CLI uses AWS_PROFILE when set, but no profile is required if your default credential chain can read the secret.

ADMIN_KEY_SECRET_ARN=$(pulumi stack output jumphost_admin_private_key_secret_arn -s "$ENVIRONMENT")
umask 077
ADMIN_KEY=$(mktemp)
trap 'rm -f "$ADMIN_KEY"' EXIT
aws secretsmanager get-secret-value \
  --secret-id "$ADMIN_KEY_SECRET_ARN" \
  --query SecretString --output text > "$ADMIN_KEY"
chmod 600 "$ADMIN_KEY"
ssh -i "$ADMIN_KEY" "ssh-admin@$NLB_DNS"

Human Evaluations via Hawk

For Hawk human-evaluation sandboxes, the jumphost's authorized_keys is managed automatically: hawk human eval start registers the human's public key when the sandbox starts, and hawk delete removes it. Users get a copy-paste-ready ssh -J command via hawk human eval ssh-command <eval-set-id> instead of constructing the ProxyJump manually.

Key Management

As ssh-admin:

# Add a public key for ssh-user
sudo /add-public-key.sh "ssh-rsa AAAA... user@example.com"

# Verify key was added
cat /home/ssh-user/.ssh/authorized_keys

# Remove a public key
sudo /remove-public-key.sh "user@example.com"

Test ssh-user Access

# Add your key
PUBLIC_KEY="$(cat ~/.ssh/id_ed25519.pub)"
printf '%s\n' "$PUBLIC_KEY" | ssh -i "$ADMIN_KEY" "ssh-admin@$NLB_DNS" \
  'read -r public_key; sudo /add-public-key.sh "$public_key"'

Troubleshooting

Check ECS Task Status

aws ecs describe-services --cluster $ENVIRONMENT-platform --services $ENVIRONMENT-jumphost \
  --query 'services[0].{running:runningCount,desired:desiredCount}'

View Logs

aws logs tail "/ecs/${ENVIRONMENT}-jumphost" --follow --since 5m

Host Key Changed Warning

After redeployment, clear the old host key:

ssh-keygen -R $NLB_DNS