Kalavai is an open source platform that unlocks computing from spare capacity. It aggregates resources from multiple sources to increase your computing budget and run large AI workloads.

Kalavai's goal is to make using AI workloads in real applications accessible and affordable to all.

• Increase GPU utilisation from your devices (fractional GPU).
• Multi-node, multi-GPU and multi-architecture support (AMD and NVIDIA).
• Aggregate computing resources from multiple sources: home desktops, work computers, multi cloud VMs, raspberry pi's, Mac, etc.
• Ready-made templates to deploy common AI building blocks: model inference (vLLM, llama.cpp, SGLang), GPU clusters (Ray, GPUStack), automation workflows (n8n and Flowise), evaluation and monitoring tools (Langfuse), production dev tools (LiteLLM, OpenWebUI) and more.
• Easy to expand to custom workloads

• November: Kalavai is now opening a managed service to create and manage AI workloads on a fleet of GPUs. We are inviting Beta Testers for early access. If you are interested Apply here
• September: Kalavai now supports Ray clusters for massively distributed ML.
• August 2025: Added support for AMD GPUs (experimental)
• July 2025: Added support for GPUStack clusters for managed LLM deployments (experimental).
• June 2025: Native support for Mac and Raspberry pi devices (ARM).
• May 2025: Added support for diffusion pipelines (experimental)
• April 2025: Added support for workflow automation engines n8n and Flowise (experimental)
• March 2025: Added support for AI Gateway LiteLLM

We currently support out of the box the following AI engines:

• vLLM: most popular GPU-based model inference.
• llama.cpp: CPU-based GGUF model inference.
• SGLang: Super fast GPU-based model inference.
• n8n (experimental): no-code workload automation framework.
• Flowise (experimental): no-code agentic AI workload framework.
• Speaches: audio (speech-to-text and text-to-speech) model inference.
• Langfuse (experimental): open source evaluation and monitoring GenAI framework.
• OpenWebUI: ChatGPT-like UI playground to interface with any models.
• diffusers (experimental)
• RayServe inference.
• GPUstack (experimental)

Not what you were looking for? Tell us what engines you'd like to see.

Kalavai is at an early stage of its development. We encourage people to use it and give us feedback! Although we are trying to minimise breaking changes, these may occur until we have a stable version (v1.0).

• Get a free Kalavai account and access unlimited AI.
• Full documentation for the project.
• Join our Substack for updates and be part of our community
• Join our discord community

The kalavai-client is the main tool to interact with the Kalavai platform, to create and manage both local and public pools and also to interact with them (e.g. deploy models).

The client is a python package and can be installed with one command:

pip install kalavai-client

You can create and manage your pools with the kalavai GUI or the Command Line Interface (CLI). For a quick start, get a pool going with:

kalavai pool start

And then start the GUI:

kalavai gui start

This will expose the GUI and the backend services in localhost. By default, the GUI is accessible via http://localhost:49153.

Check out our getting started guide for next steps on how to add more workers to your pool, or use our managed seeds service for over-the-internet AI pools.

Check out our use cases documentation for inspiration on what you can do with Kalavai:

• Multi-GPU LLM
• Fine tune
• Autoscaling deployments
• BYO Model Gateway
• Easy LLMs with GPUstack
• Production GPU fleets

Anything missing here? Give us a shout in the discussion board. We welcome discussions, feature requests, issues and PRs!

• Join the community and share ideas!
• Report bugs, issues and new features.
• Help improve our compatibility matrix by testing on different operative systems.
• Follow our Substack channel for news, guides and more.
• Community integrations are template jobs built by Kalavai and the community that makes deploying distributed workflows easy for users. Anyone can extend them and contribute to the repo.

Name: DOCKER_HUB_USERNAME
Value: Your Docker Hub username or organization name.

Name: DOCKER_HUB_TOKEN
Value: The Personal Access Token you copied from Docker Hub.

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install python3.10 python3.10-dev python3-virtualenv python3-venv
virtualenv -p python3.10 env
source env/bin/activate
sudo apt install  python3.10-venv python3.10-dev -y
pip install -U setuptools
pip install -e .[dev]

bash publish.sh build

To run the unit tests, use:

python -m unittest