Consult is a web application that combines AI with human oversight to process public consultation responses at scale to inform public policy. Once consultation responses are uploaded to the app, the AI identifies themes across the responses using the themefinder package. Users review and finalise these themes — selecting, editing, or creating new ones — before AI assigns the finalised themes to individual responses. The results are presented in a dashboard for users to analyse and draw insights from.

The repository is split into a Django REST backend (backend/), an Astro and Svelte frontend (frontend/), AI processing pipelines that run on AWS Batch (pipeline-sign-off/, pipeline-mapping/), Lambda functions that sync pipeline results to the database (lambda/), and Terraform infrastructure (terraform/).

Installation instructions assume using a Mac with Homebrew.

• Docker Desktop
• uv (instructions)
• nvm (instructions)
• GraphViz (brew install graphviz), used for generating database diagrams
• pre-commit (brew install pre-commit)
• Postegres(optional) (brew install postgresql) if you are getting psycopg2 error during make setup

Make sure that you have coreutils installed: brew install coreutils

Also note that you will need to add a personal access token on github with read:packages access and then use this to log in on the command line before running the end-to-end tests:

echo $PASSWORD | docker login ghcr.io -u <username> --password-stdin

git clone git@github.com:i-dot-ai/consult.git
cd consult
make install
make setup

The make install command installs the correct Python and Node versions and all dependencies. And make setup creates .env files from templates and sets up the database with dummy data and an admin user (email@example.com / admin).

make serve

This starts the backend (API server + RQ workers) at http://localhost:8000 and the frontend (Astro dev server) at http://localhost:3000.

You can also run them separately with make backend and make frontend.

To generate new migrations after changing models:

make migrations

To apply migrations:

make migrate

Running make migrate also regenerates the entity-relationship diagram at docs/erd.png (requires graphviz). The current schema:

Run backend tests:

make test-backend

Run frontend tests:

make test-frontend

Run end-to-end tests:

docker compose up -d postgres # postgres must be running already
make test-end-to-end

If you are getting error while running e2e that the frontend is failing to start during the docker spin up its likely because of the timeout module that is missing and you will need to run

brew install coreutils

The scripts/ directory contains CLI tools for preparing a consultation's data for the ThemeFinder pipeline:

# Generate an opinionated Q.U. workbook template with live in-sheet validation:
make build-consultation-template

# Validate a Q.U. workbook against response data, build the ThemeFinder
# input layout, and upload it to S3:
make setup-consultation name=my_consultation

See scripts/README.md for the full pipeline walkthrough and scripts/setup_consultation_checks.md for the list of validation rules.

This project includes VSCode configuration files to ensure consistent development experience:

• .vscode/settings.json - Workspace settings for formatting, linting, and language support
• .vscode/extensions.json - Recommended extensions for the project

When you open the project in VSCode, you'll be prompted to install recommended extensions. These include:

• Python - Python language support with uv integration
• Ruff - Python linter and formatter
• ESLint - JavaScript/TypeScript linter
• Prettier - JavaScript/TypeScript code formatter
• Astro - Astro framework support
• Svelte - Svelte framework support
• Tailwind CSS IntelliSense - Tailwind CSS tooling

The workspace settings are configured to:

• Format code on save (using appropriate formatter per language)
• Auto-fix ESLint issues on save
• Enable TypeScript support in Svelte files

You can override these settings in your User Settings if you prefer different personal configurations. See the VSCode settings documentation for more information on the settings hierarchy.