Skip to content

mz0/gcp-model

Repository files navigation

Bring up

  1. Install Requirements: gcloud, terraform - see Terraform and Google Cloud CLI
  2. Authenticate and Set Up Project (here I re-use existing project b-twim in my account)
    PROJECT_ID=b-twim
    gcloud auth login
    gcloud config set project ${PROJECT_ID}
    gcloud services enable compute.googleapis.com
  3. Verify GPU Quota
  4. Create Google Compute Service Account and get its ID ("e-mail")
    gcloud iam service-accounts create "${PROJECT_ID}-sa" \
      --project=${PROJECT_ID} \
      --display-name="Compute Instance Service Account"
    
    gcloud iam service-accounts list --project=${PROJECT_ID} \
      --filter="displayName:Compute Instance Service Account" \
      --format="value(email)"
  5. Create Terraform Service Account (first time only)
    gcloud iam service-accounts create terraform \
      --display-name "Terraform admin account"
    
    gcloud projects add-iam-policy-binding ${PROJECT_ID} \
      --member="serviceAccount:terraform@${PROJECT_ID}.iam.gserviceaccount.com" \
      --role="roles/editor"
    
    gcloud iam service-accounts keys create terraform-key.json \
      --iam-account=terraform@${PROJECT_ID}.iam.gserviceaccount.com
  6. Initialize Terraform
    export GOOGLE_APPLICATION_CREDENTIALS="./terraform-key.json"
    
    terraform init
  7. Adjust terraform.tfvars (use sample ), preview (plan), and deploy
    terraform plan  -var-file="terraform.tfvars"
    terraform apply -var-file="terraform.tfvars"
  8. Verify image availability
    gcloud compute images list --project deeplearning-platform-release \
      --filter="name ~ 'common-cu12'" --sort-by ~creationTimestamp --limit 5

After Deployment

  • SSH into the VM
    gcloud compute ssh llm1server --zone=us-west1-b
  • Test the LLM
    curl http://${externalIP}:8000/v1/completions

Cleanup

terraform destroy

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors