Eel is "Eric's infErence Library".

This is simple LLM inference with a thin python wrapper.

Create a new python environment, e.g.

python -m venv .venv

and with that environment activated, install the dependencies:

pip install -r requirements.txt

Make the C code:

make

Run scripts/download_models.sh to download the models.

Run scripts/main.py data/Qwen2.5-0.5B-Instruct. This will run a timing test and output something like:

(eel) eric@macbook eel % scripts/main.py data/Qwen2.5-0.5B-Instruct
Prompt time: 1.5839s
Total time: 3.9726s
Certainly! A large language model, or LLM, is a type of artificial intelligence that can generate human-like text based on a given input. These models are designed to be highly accurate, creative, and adaptable, making them useful in a wide range of applications, including but not limited to:

1. **Chatbots**:

At present, there is just a simple C implementation using only 32-bit floats.

On MacOS, matrix-vector multiplies use Accelerate BLAS, but the performance improvement from that over simple hand-coded matrix-vector multiply is minimal.