A distributed browser automation and security monitoring platform for detecting threats in web applications through instrumented browser sessions.
Merrymaker is a security monitoring platform that executes JavaScript in instrumented browser environments to capture and analyze security-relevant events. It consists of two main services:
- merrymaker-go: Core API, job queue, rules engine, and web UI
- puppeteer-worker: Browser automation worker that executes jobs and captures events
- Job Queue System - Reliable job processing with priority queues, retries, and scheduling
- Rules Engine - Distributed detection system for security threats and anomalies
- Unknown domain detection with allowlist support
- IOC (Indicator of Compromise) domain matching
- YARA rule scanning (planned)
- Site Management - Configure and schedule browser automation jobs per site
- Source Management - Store and version browser automation scripts
- Alert System - HTTP webhooks for security alert notifications
- Event Storage - Capture and query browser events (network, console, screenshots)
- Secrets Management - Encrypted storage for credentials and API keys
- Web UI - Modern interface for managing sites, sources, jobs, and alerts
- Multi-Service Architecture - Run HTTP API, rules engine, scheduler, and reaper independently or together
- Browser Automation - Execute JavaScript in instrumented Chromium instances
- Event Capture - Network requests/responses, console logs, screenshots, client-side monitoring
- File Capture - Capture and store response bodies with deduplication
- Flexible Storage - Memory, Redis, or cloud storage backends for captured files
- Job Processing - Reserve, execute, and report job status via REST API
- Configurable - YAML and environment variable configuration
- Go 1.24.6+
- Node.js 20.18.1+
- Bun 1.2+
- PostgreSQL 15+
- Redis 7+
- Docker & Docker Compose (for local development)
Use Bun for the merrymaker-go frontend build pipeline; Node.js remains required for the puppeteer-worker service and other Node-based tooling in this repository.
# Start PostgreSQL and Redis for development
cd services/merrymaker-go
make dev-db-up
# Wait for services to be healthy, then run migrations and seed data
make dev-db-seed
make dev-db-verify# Copy example environment file
cp services/merrymaker-go/.env.example services/merrymaker-go/.env
# Edit .env with your configuration
# Required: DB credentials, Redis connection, OAuth settings
# Local dev: set AUTH_MODE=mock to use the built-in mock identity provider (never enable in production)cd services/merrymaker-go
# Build frontend assets (requires Bun)
cd frontend
bun install
bun run build
cd ..
# Run all services (HTTP API, rules engine, scheduler, reaper)
go run ./cmd/merrymaker
# Or run specific services
SERVICES=http go run ./cmd/merrymaker # HTTP API only
SERVICES=http,rules-engine go run ./cmd/merrymaker # HTTP + rules engine
SERVICES=scheduler go run ./cmd/merrymaker # Scheduler only
SERVICES=reaper go run ./cmd/merrymaker # Cleanup worker only
SERVICES=http,scheduler,rules-engine,reaper go run ./cmd/merrymaker # Explicit full setDefault behavior (no SERVICES set) starts all available components. On Windows PowerShell use $env:SERVICES="http"; go run ./cmd/merrymaker; on cmd.exe use set SERVICES=http && go run .\cmd\merrymaker.
The web UI will be available at http://localhost:8080
cd services/puppeteer-worker
# Install dependencies
npm install
# Configure worker
cp .env.example .env
# Edit .env: set MERRYMAKER_API_BASE=http://localhost:8080
# Start worker
npm run worker┌─────────────────────────────────────────────────────────────┐
│ Merrymaker Platform │
├─────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ HTTP API │ │ Rules Engine │ │ Scheduler │ │
│ │ │ │ │ │ │ │
│ │ • Web UI │ │ • Unknown │ │ • Cron jobs │ │
│ │ • REST API │ │ domains │ │ • Site scans │ │
│ │ • Auth │ │ • IOC match │ │ • Recurring │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
│ │ │ │ │
│ └─────────────────┼────────────────┘ │
│ │ │
│ ┌─────────────────┴──────────────────┐ │
│ │ │ │
│ ┌────▼─────┐ ┌────▼─────┐ │
│ │PostgreSQL│ │ Redis │ │
│ │ │ │ │ │
│ │• Jobs │ │• Caches │ │
│ │• Events │ │• Locks │ │
│ │• Sites │ │• Files │ │
│ └──────────┘ └──────────┘ │
│ │
└───────────────────────────┬─────────────────────────────────┘
│
│ REST API
│
┌───────────▼───────────┐
│ Puppeteer Worker(s) │
│ │
│ • Reserve jobs │
│ • Execute scripts │
│ • Capture events │
│ • Report results │
└───────────────────────┘
- Job Creation: Sites schedule jobs via cron expressions or manual triggers
- Job Execution: Puppeteer workers reserve jobs, execute browser scripts
- Event Capture: Workers capture network, console, screenshot events
- Event Processing: Rules engine analyzes events for security threats
- Alert Generation: Detected threats trigger alerts sent to HTTP webhooks
Core Entities:
- Sites: Monitored web applications with scheduling configuration
- Sources: Versioned browser automation scripts
- Jobs: Queued work items with priority, status, and payload
- Events: Browser events (network, console, screenshots) linked to jobs
- Alerts: Fired security alerts with context and metadata
- Rules Engine: Detection rules (unknown domains, IOC matching, YARA)
- Secrets: Encrypted credentials for authentication
- HTTP Alert Sinks: Webhook destinations for alert notifications
Configuration via environment variables (.env file):
# Database
DB_HOST=localhost
DB_PORT=5432
DB_NAME=merrymaker
DB_USER=merrymaker
DB_PASSWORD=merrymaker
# Redis
REDIS_HOST=localhost
REDIS_PORT=6379
# Authentication
AUTH_MODE=oauth # oauth or dev
OAUTH_ISSUER=https://auth.example.com
OAUTH_CLIENT_ID=your-client-id
OAUTH_CLIENT_SECRET=your-secret
ADMIN_GROUP=merrymaker-admins
USER_GROUP=merrymaker-users
# Service Configuration
SERVICES=http,rules-engine,scheduler,reaper # Comma-separated list
HTTP_ADDR=:8080
# Rules Engine
RULES_ENGINE_CONCURRENCY=1
RULES_ENGINE_BATCH_SIZE=100
RULES_ENGINE_AUTO_ENQUEUE=true
# Scheduler
SCHEDULER_CONCURRENCY=1
SCHEDULER_BATCH_SIZE=10
SCHEDULER_INTERVAL=30s
# Reaper
REAPER_INTERVAL=5m
REAPER_PENDING_MAX_AGE=1h
REAPER_COMPLETED_MAX_AGE=168h
REAPER_FAILED_MAX_AGE=168h
REAPER_BATCH_SIZE=1000
# Additional guidance: services/merrymaker-go/README.md#reaper-configurationConfiguration via environment variables or YAML:
# Worker Configuration
MERRYMAKER_API_BASE=http://localhost:8080
WORKER_JOB_TYPE=browser
WORKER_LEASE_SECONDS=30
WORKER_WAIT_SECONDS=25
WORKER_HEARTBEAT_SECONDS=10
# Browser Configuration
PUPPETEER_HEADLESS=true
PUPPETEER_TIMEOUT=30000
# File Capture
FILE_CAPTURE_ENABLED=true
FILE_CAPTURE_TYPES=script,document
FILE_CAPTURE_STORAGE=redis
FILE_CAPTURE_MAX_SIZE=10485760
# Event Shipping
SHIPPING_BATCH_SIZE=50
SHIPPING_MAX_BATCH_AGE=5000See services/puppeteer-worker/README.md for detailed configuration options.
merrymaker/
├── services/
│ ├── merrymaker-go/ # Core Go service
│ │ ├── cmd/ # Entry points
│ │ ├── internal/ # Domain, application, infrastructure packages
│ │ │ ├── domain/ # Business entities and value objects
│ │ │ ├── ports/ # Interfaces for persistence/external systems
│ │ │ ├── service/ # Scheduler, rules engine, reaper orchestration
│ │ │ ├── core/ # Shared infrastructure primitives
│ │ │ ├── data/ # Repository implementations (Postgres, Redis, etc.)
│ │ │ ├── adapters/ # Integrations (auth, job runners, notifications)
│ │ │ ├── http/ # HTTP handlers, middleware, routes
│ │ │ ├── bootstrap/ # Composition root and wiring
│ │ │ ├── migrate/ # Migration helpers and SQL files
│ │ │ ├── mocks/ # Generated gomock stubs
│ │ │ └── testutil/ # Test harnesses and utilities
│ │ ├── frontend/ # Bun-based HTMX frontend build
│ │ ├── assets.go # Embed bindings for frontend assets
│ │ └── Makefile # Dev orchestration
│ │
│ └── puppeteer-worker/ # Browser automation worker
│ ├── src/ # TypeScript source
│ ├── config/ # Configuration schemas
│ └── scripts/ # Build scripts
│
└── build/ # Build artifacts
Static assets for merrymaker-go are embedded via assets.go, replacing the legacy web/ directory used in earlier releases.
# Merrymaker Go tests
cd services/merrymaker-go
make test # Run all tests
make test-integration # Integration tests only
# Puppeteer worker tests
cd services/puppeteer-worker
npm testcd services/merrymaker-go
# Development database
make dev-db-up # Start dev PostgreSQL
make dev-db-down # Stop dev PostgreSQL
make dev-db-seed # Seed with sample data
make dev-db-verify # Verify seeded data
# Test database
make test-db-up # Start test PostgreSQL
make test-db-down # Stop test PostgreSQL
make test # Run tests (auto-starts test DB)For rapid iteration with automatic rebuilds and restarts:
cd services/merrymaker-go
# Install Air (one-time setup, pinned version)
go install github.com/air-verse/air@v1.63.0
# Start full dev environment (DB + live-reload)
make dev-full
# Or start live-reload only (no DB)
make devAir watches Go source files, frontend styles, templates, and vendor scripts. Edit any file and see changes in ~1-2 seconds without manual rebuilds or restarts. The DEV=true environment variable enables hot-reloading of templates and static assets from disk.
cd services/merrymaker-go/frontend
# Install dependencies
bun install
# Development build (watch mode)
bun run watch
# Production build
bun run build
# Lint and format
bun run lint
bun run formatJobs:
GET /api/jobs/{jobType}/reserve_next?lease={seconds}&wait={seconds}- Reserve next jobPOST /api/jobs/{jobId}/heartbeat?extend={seconds}- Extend job leasePOST /api/jobs/{jobId}/complete- Mark job completePOST /api/jobs/{jobId}/fail- Mark job failedGET /api/jobs/{jobId}/status- Get job statusGET /api/jobs/{jobId}/events- Get job events
Events:
POST /api/events/bulk- Bulk insert events
Sites, Sources, Secrets, Alerts:
- See web UI or OpenAPI documentation (planned)
{
"script": "await page.goto('https://example.com'); await screenshot();",
"source_id": "optional-source-uuid"
}Or URL-only format:
{
"url": "https://example.com"
}- Puppeteer Worker - Worker configuration and usage
- Frontend Build - Frontend build system