DevBox

DevBox is a Kubernetes-based container development machine management system that provides developers with persistent, customizable development environments.

Features

Core Features

Instance Lifecycle Management: Create, start, stop, and delete development machine instances
State Persistence: Use nerdctl rootless to run containers on block storage, ensuring installed system packages are persisted
Fast Creation: Leverage block storage snapshot functionality to quickly clone large images from pre-configured snapshots (supports 30G-100G images)
Common Service Injection: Services such as SSH, JupyterLab, and Conda are automatically injected as Sidecar containers
Storage Mounting: Support for user-defined storage sizes and mount points
Network Access: JupyterLab is exposed via Ingress, SSH is accessed through a dedicated SSH Proxy service
Image Building: Support for saving current instances as images, or building new images via Dockerfile

Component Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        DevBox Operator                          │
├─────────────────────────────────────────────────────────────────┤
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐  │
│  │ DevBox          │  │ DevBoxRelease   │  │ DevBoxOpsRequest│  │
│  │ Controller      │  │ Controller      │  │ Controller      │  │
│  └─────────────────┘  └─────────────────┘  └─────────────────┘  │
├─────────────────────────────────────────────────────────────────┤
│                        SSH Proxy Service                         │
└─────────────────────────────────────────────────────────────────┘

CRD Documentation

DevBox represents a development machine instance. Two modes are supported:

Mode 1: Template Mode (Recommended, using DevBoxRelease)

apiVersion: devbox.scclabs.io/v1alpha1
kind: DevBox
metadata:
  name: my-devbox
spec:
  # Reference DevBoxRelease template
  devBoxReleaseRef: pytorch-cuda12
  state: Running
  resources:
    cpu: "4"
    memory: "16Gi"
    gpu: "1"
    gpuType: "nvidia.com/gpu"
  # ... other configurations

Advantages:

Simple configuration, just reference the template name
Administrators centrally maintain image templates
Supports centralized snapshot management

Mode 2: Direct Mode (Simplified, no DevBoxRelease required)

apiVersion: devbox.scclabs.io/v1alpha1
kind: DevBox
metadata:
  name: my-devbox-direct
spec:
  # Directly specify image and configuration
  image: nvcr.io/nvidia/pytorch:24.01-py3
  imagePullPolicy: IfNotPresent
  rootVolume:
    size: "100Gi"
    storageClassName: csi-ceph-rbd
    snapshotSource:
      snapshotName: pytorch-cuda12-snapshot
  state: Running
  resources:
    cpu: "4"
    memory: "16Gi"
  # ... other configurations

Advantages:

No need to create DevBoxRelease first
More intuitive configuration, all information in one resource
Suitable for single users or temporary use

Note: devBoxReleaseRef and image/rootVolume are mutually exclusive, only one mode can be chosen.

DevBoxRelease

DevBoxRelease defines available image templates, including snapshot information for fast creation.

apiVersion: devbox.scclabs.io/v1alpha1
kind: DevBoxRelease
metadata:
  name: pytorch-cuda12
spec:
  displayName: "PyTorch 2.0 with CUDA 12"
  image: nvcr.io/nvidia/pytorch:24.01-py3
  rootVolume:
    size: "100Gi"
    storageClassName: csi-ceph-rbd
    snapshotSource:
      snapshotName: pytorch-cuda12-snapshot
  containerRuntime:
    type: nerdctl
    enableBuildKit: true
  defaultResources:
    cpu: "4"
    memory: "8Gi"
    gpu: "1"
  category: AI
  tags:
    - pytorch
    - cuda
  requiresGPU: true

DevBoxOpsRequest

DevBoxOpsRequest is used to perform operations such as saving images, building images, etc.

apiVersion: devbox.scclabs.io/v1alpha1
kind: DevBoxOpsRequest
metadata:
  name: save-my-devbox
spec:
  devBoxRef: my-devbox
  type: SaveImage
  saveImage:
    targetImage: ghcr.io/myorg/my-custom-image:v1.0
    registrySecret: regcred
    message: "Added custom packages"

Quick Start

Prerequisites

Kubernetes 1.26+
kubectl configured
Block storage support (e.g., Ceph RBD, AWS EBS)
VolumeSnapshot support
Helm 3.x (optional, for Helm deployment)

Method 1: Install using Makefile

Install CRDs:

make install

Deploy Operator:

make deploy IMG=ghcr.io/scclabs/devbox-operator:latest

Deploy SSH Proxy:

kubectl apply -f deploy/kubernetes/ssh-proxy-deployment.yaml

Quick Test

After installation, you can use the quick test script to verify functionality:

# Use direct mode (recommended, simplest)
./hack/quick-test.sh

# Or use template mode
MODE=template ./hack/quick-test.sh

# Custom namespace and name
NAMESPACE=test DEVBOX_NAME=my-test ./hack/quick-test.sh

Or manually create a simple example:

# Direct mode (no Release required)
kubectl apply -f config/samples/devbox_v1alpha1_devbox-simple.yaml

# Check status
kubectl get devbox test-devbox-direct
kubectl get pods -l devbox.scclabs.io/name=test-devbox-direct

For detailed instructions, please refer to: Quick Start Guide

Method 2: Install using Helm

# Add Helm repository (if published)
# helm repo add devbox https://scclabs.github.io/devbox

# Or install directly from local
helm install devbox ./deploy/helm/devbox \
  --namespace devbox-system \
  --create-namespace \
  --set operator.image.repository=ghcr.io/scclabs/devbox-operator \
  --set operator.image.tag=latest

Helm Custom Configuration:

# View all configurable options
helm show values ./deploy/helm/devbox

# Use custom values file
helm install devbox ./deploy/helm/devbox \
  -f my-values.yaml \
  --namespace devbox-system

Create Development Machine

Quick Test (Recommended for Beginners)

Use a simple Ubuntu image for quick testing:

# Method 1: Use template mode
kubectl apply -f config/samples/devbox_v1alpha1_devboxrelease-simple.yaml
kubectl apply -f config/samples/devbox_v1alpha1_devbox-simple.yaml

# Method 2: Use direct mode (simpler, no Release required)
kubectl apply -f config/samples/devbox_v1alpha1_devbox-simple.yaml

Check status:

kubectl get devbox test-devbox
kubectl describe devbox test-devbox
kubectl get pods -l devbox.scclabs.io/name=test-devbox

For detailed instructions, please refer to: Quick Start Guide

Full Feature Example

Create DevBoxRelease (administrator operation):

kubectl apply -f config/samples/devbox_v1alpha1_devboxrelease.yaml

Create DevBox:

kubectl apply -f config/samples/devbox_v1alpha1_devbox.yaml

Check status:

kubectl get devbox
kubectl describe devbox my-devbox

Access Development Machine

JupyterLab: Access the configured Ingress hostname, e.g., https://my-devbox.example.com

SSH:

# Access via SSH Proxy
ssh devbox@my-devbox.default@<ssh-proxy-ip> -p 2222

# Username format: <username>@<devbox-name>.<namespace>

Stop/Start Development Machine

# Stop
apiVersion: devbox.scclabs.io/v1alpha1
kind: DevBoxOpsRequest
metadata:
  name: stop-my-devbox
spec:
  devBoxRef: my-devbox
  type: Stop

Or directly modify DevBox:

kubectl patch devbox my-devbox --type=merge -p '{"spec":{"state":"Stopped"}}'

Save Image

apiVersion: devbox.scclabs.io/v1alpha1
kind: DevBoxOpsRequest
metadata:
  name: save-image-request
spec:
  devBoxRef: my-devbox
  type: SaveImage
  saveImage:
    targetImage: ghcr.io/myorg/my-devbox:v1.0
    registrySecret: regcred
    message: "Installed custom packages"

Build Image

apiVersion: devbox.scclabs.io/v1alpha1
kind: DevBoxOpsRequest
metadata:
  name: build-image-request
spec:
  devBoxRef: my-devbox
  type: BuildImage
  buildImage:
    targetImage: ghcr.io/myorg/my-custom:v1.0
    dockerfile: Dockerfile
    context: /workspace/myproject
    push: true

Directory Structure

devbox/
├── api/v1alpha1/           # CRD type definitions
│   ├── devbox_types.go
│   ├── devboxrelease_types.go
│   └── devboxopsrequest_types.go
├── cmd/
│   ├── main.go             # Operator entry point
│   └── sshproxy/           # SSH Proxy service
├── internal/controller/    # Controller implementation
│   ├── devbox_controller.go
│   ├── devboxrelease_controller.go
│   └── devboxopsrequest_controller.go
├── pkg/sshproxy/          # SSH Proxy core logic
├── build/                  # Sidecar image builds
│   ├── sidecar-ssh/
│   ├── sidecar-jupyterlab/
│   └── sidecar-conda/
├── config/                 # Kubernetes configuration
│   ├── crd/               # CRD YAML
│   ├── samples/           # Example YAML
│   └── rbac/              # RBAC configuration
└── deploy/kubernetes/      # Deployment configuration

Development

Run Locally

# Install dependencies
go mod tidy

# Generate code
make generate

# Generate CRDs
make manifests

# Run locally
make run

Build Images

# Build Operator
make docker-build docker-push IMG=ghcr.io/scclabs/devbox-operator:latest

# Build SSH Proxy
docker build -t ghcr.io/scclabs/devbox-ssh-proxy:latest -f cmd/sshproxy/Dockerfile .

# Build Sidecar images
docker build -t ghcr.io/scclabs/devbox-sidecar-ssh:latest build/sidecar-ssh/
docker build -t ghcr.io/scclabs/devbox-sidecar-jupyterlab:latest build/sidecar-jupyterlab/
docker build -t ghcr.io/scclabs/devbox-sidecar-conda:latest build/sidecar-conda/

Testing

make test

Configuration

Environment Variables

Variable	Description	Default
`ENABLE_WEBHOOKS`	Enable Admission Webhooks	`true`
`METRICS_ADDR`	Metrics service address	`:8080`
`PROBE_ADDR`	Health check address	`:8081`

SSH Proxy Parameters

Parameter	Description	Default
`--listen-addr`	Listen address	`:2222`
`--host-key`	SSH Host Key file path	Auto-generated
`--default-namespace`	Default namespace	`default`
`--ssh-port`	DevBox SSH port	`22`

Advanced Configuration

GPU Support

Ensure the cluster has NVIDIA GPU Operator or Device Plugin installed, then configure in DevBox:

spec:
  resources:
    gpu: "1"
    gpuType: "nvidia.com/gpu"
  nodeSelector:
    nvidia.com/gpu.present: "true"
  tolerations:
    - key: nvidia.com/gpu
      operator: Exists
      effect: NoSchedule
  runtimeClassName: nvidia  # If using nvidia runtime

Storage Snapshots

Prepare base image and create VolumeSnapshot:

apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: pytorch-cuda12-snapshot
spec:
  volumeSnapshotClassName: csi-ceph-rbd-snapshot
  source:
    persistentVolumeClaimName: pytorch-base-pvc

Reference in DevBoxRelease:

spec:
  rootVolume:
    snapshotSource:
      snapshotName: pytorch-cuda12-snapshot

License

Apache License 2.0

Name		Name	Last commit message	Last commit date
Latest commit History 6 Commits
.devcontainer		.devcontainer
.github/workflows		.github/workflows
api/v1alpha1		api/v1alpha1
cmd		cmd
config		config
deploy/kubernetes		deploy/kubernetes
hack		hack
internal/controller		internal/controller
pkg/sshproxy		pkg/sshproxy
test		test
.dockerignore		.dockerignore
.gitignore		.gitignore
.golangci.yml		.golangci.yml
Dockerfile		Dockerfile
Makefile		Makefile
PROJECT		PROJECT
README.md		README.md
go.mod		go.mod
go.sum		go.sum

wutz/devbox

Folders and files

Latest commit

History

Repository files navigation

DevBox - Container Development Machine Kubernetes Operator