Local AI with Vulkan support Community Container for Nextcloud All-In-One

This container is used in the Nextcloud All-In-One AI backend for Nextcloud Assistant. It works with the Caddy community container as a reverse proxy.

Compared to the default LocalAI container, this container provides:

• Automatic configuration of Nextcloud Assistant.
• Hardware acceleration support with Vulkan.
• Easy access to the local AI web interface.

You need:

• A Nextcloud instance running on a server with x86_64 architecture.
• At least 7GB of available storage space.
• Ensure that port 10078 is not used by another program (use sudo netstat -tulpn to list all used ports).
• Deploy the Caddy community container as a reverse proxy. (Other solutions are possible, see: Use Your Own Reverse Proxy).

For hardware acceleration (optional but recommended), you need:

• A GPU compatible with Vulkan. Run vulkaninfo in the terminal to check if it is enabled.
• Enable DRI device in AIO. Add --env NEXTCLOUD_ENABLE_DRI_DEVICE=true to the container.

See how to use community containers.

After installation on Nextcloud, go to https://ai.$NC_DOMAIN and log in with:

• API key: Retrieve the password from the AIO interface

From the Local AI web interface, you can download and manage models. By default, no models are installed, so you need to download them manually. See Recommended Models below.

After downloading the models, configure the Local AI server in Nextcloud:

1. Go to https://$NC_DOMAIN/settings/admin/ai
2. Set the Local AI server URL to http://nextcloud-aio-local-ai:10078
3. Setup API key authentication with the API key from above.
4. Configure the models you want to use in the Nextcloud Assistant settings.

Feedback and suggestions are welcome!

Redirect HTTPS (or HTTP) traffic from ai.$NC_DOMAIN (or another subdomain) to port 10078 of the nextcloud-aio-local-ai container over HTTP.

Example with Caddyfile syntax:

https://ai.{$NC_DOMAIN}:443 {
    reverse_proxy http://{$LOCAL_AI_HOSTNAME}:10078
}