Happy Friday! This is the v1.2.0 release of LlamaChat, and the big update this week is support for configuring ✨ model hyperparameters ✨, alongside a bunch of other tweaks and improvements.

🔥 New

• You can now configure the model hyperparameters (including context size and sampling parameters like top-p and top-k values) for all of your chat sources in Settings > Sources. These parameters still default to sensible values for each model type, allowing you to tweak them as you see fit (#13).
• You can now configure the number of threads that LlamaChat runs text generation on from Settings > General (#13).
• You can now configure whether the model for each chat source is fully loaded into memory during prediction, which can improve performance for smaller models. If you're familiar with llama.cpp this controls the --mlock parameter. (#4)

🫧 Improved

• You can now import older .ggml files directly into LlamaChat without conversion, thanks to some upstream changes made to llama.cpp. (#3)
• The chat view has been revamped with slicker message bubbles and animations. It also automatically scrolls to the bottom when new messages are added. (#18)
• The File menu has been improved: you can now add new sources with ⌘N.
• The Add Chat Source flow has been improved to make it (almost) pixel-perfect, and a dedicated Cancel button has been added to make it clearer how to exit (previously this could be done with Esc). (#7)

🐞 Bug Fixes

• Previously when converting PyTorch checkpoints directly in LlamaChat, an intermediary converted (but un-quantized) .ggml artefact was left on the filesystem. This has now been fixed, and any of these artefacts left by previous versions of LlamaChat are automatically cleaned up by on launch. (#10)

❤️ Sponsorships

• Sponsorship positions have been opened to help support the continued development of LlamaChat. Any support is much appreciated, and more info can be found on the sponsorship page.

• Adds the Sparkle framework to support automatic updates going forward
• Makes minor adjustments to the release scripts in Scripts/
• Fixes #5
• Updates the README

This release fixes a few niggly issues, as well as an issue related to Chat Sources (#1):

• Fixes persistence of Chat Sources (#1) - this was broken if there were any sources without avatars
• Makes text in message bubbles selectable (#2)
• Other minor changes to the repo and setup since v1.0 tag

This is the v1.0 release of LlamaChat, which allows you to run LLaMA-compatible model files in a native macOS chat-style app.

LlamaChat currently supports models from:

• The original LLaMA project: https://github.com/facebookresearch/llama
• Stanford's Alpaca: https://github.com/tatsu-lab/stanford_alpaca
• NomicAI's GPT4All: https://github.com/nomic-ai/gpt4all

LlamaChat supports models in both the raw PyTorch checkpoint format (.pth) as well as the .ggml format, since LlamaChat is powered by the ggml and llama.cpp and llama.swift libraries.

Features

• Import .pth and .ggml models with support for pre-converting .pth files directly within the app. Note that some manual intervention may be necessary in the case of outdated .ggml model files; please see the llama.cpp repository for more.
• Chat with LLaMA-compatible models in a native macOS chat-style interface. Messages are stored between sessions.
• Support for fun Llama and Alpaca avatars to make the chat experience more fun ✌️
• Inspect and debug model context including raw tokens.
• Clear model context and chat history on demand.