forked from michaelwhitford/llm-repl
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathconfig.example.edn
More file actions
128 lines (117 loc) · 7.06 KB
/
Copy pathconfig.example.edn
File metadata and controls
128 lines (117 loc) · 7.06 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
{;; llm-repl configuration.
;;
;; Copy this file to ~/.config/llm-repl/config.edn and edit it there.
;; If more than one config exists, later ones win:
;; built-in defaults < ~/.config/llm-repl/config.edn < ./config.edn
;; < the file named by the LLM_REPL_CONFIG environment variable
;; Providers — where completions come from.
;; :llamacpp a local llama.cpp server (OpenAI-compatible /v1)
;; :codex an OpenAI ChatGPT subscription, via OAuth. Log in once with
;; `escapement login codex`; the token is stored in
;; ~/.escapement/openai-auth.json
:providers {:local {:provider/kind :llamacpp}
:openai {:provider/kind :codex}}
;; Models — the roster. Open a session on one with (open! :s {::model :name}).
;; For :llamacpp providers :model/port is required — the server's port.
;; Optional, with defaults:
;; :model/host "localhost" — where the server runs. Name a
;; LAN box, or the host gateway from inside a
;; container (podman: host.containers.internal,
;; docker: host.docker.internal)
;; :model/http-timeout-ms 300000 — raise it for slow or thinking models
;; :model/max-output-tokens 8192
;; :model/slots — pin sessions to llama.cpp KV-cache slots for
;; faster reuse; a bad pin only costs a
;; re-prefill, never a wrong answer
:models {:qwen36-35b-a3b {:model/provider :local
:model/port 5100
:model/http-timeout-ms 900000
:model/max-output-tokens 16384}
:gemma-4-31b-it {:model/provider :local
:model/port 5102}
:gpt-5.4 {:model/provider :openai}
:gpt-5.5 {:model/provider :openai}}
;; Thinking models (Qwen and friends): reasoning is ON by default. Turn it
;; off per session at the repl with (open! :s {::thinking false}) — or fork
;; the same tape both ways and compare. Only :llamacpp providers can reach
;; this switch; plain OpenAI-compatible endpoints ignore it.
;; Preamble — text placed at the top of every system prompt.
;; Set it here for everything, or per provider, per model, per session.
;; The most specific level wins and fully replaces the others:
;; session > model > provider > this top-level key
;; A missing key inherits from the level above. false or "" means "no
;; preamble". A value is a string, or {:file "~/path/to/text.txt"}.
:preamble "Be precise and concise. Say when you are unsure. Prefer runnable examples over prose."
;; Examples:
;; :providers {:local {:provider/preamble "all local models boot with this"}}
;; :models {:qwen36-35b-a3b {:model/preamble {:file "~/.config/llm-repl/qwen-boot.txt"}}}
;; at the repl: (open! :s {::preamble "just this session"})
;; (fork! :s :bare {::preamble? false}) ; same tape, no preamble
;; The WHOLE prompt stack resolves the same way (same chain, same value
;; shapes — string | {:file path} | false). Every layer is replaceable, so
;; a host can swap the entire stack for its own idiom (e.g. nucleus
;; lambda-notation prompt files):
;;
;; :system-prompt — the system voice. Default: "You are a precise
;; assistant." Chain: session :system > :model/system-prompt >
;; :provider/system-prompt > this key.
;; :orientation — the environment template appended when :tools is armed
;; (tells the model WHERE IT LIVES; the literal {slug} is substituted
;; with the session's slug at call time). Default: the built-in repl
;; orientation. Chain: session :orientation > :model/orientation >
;; :provider/orientation > this key. false ≡ armed but unoriented.
;;
;; :system-prompt {:file "~/.config/llm-repl/system.λ"}
;; :orientation {:file "~/.config/llm-repl/orientation.λ"}
;; Unknown top-level keys are REJECTED at load (a typo fails loud instead
;; of silently shaping a mystery roster). Embedding hosts put their own
;; keys under :ext — the one open slot:
;; :ext {:anima/whatever …}
;; The model a session gets when (open! :s) doesn't name one.
:default-model :qwen36-35b-a3b
;; Tools — arm sessions with the self-eval tool by default.
;; true every registered tool (today: clojure_eval — the model
;; can evaluate Clojure in the repl that hosts it, and
;; drive the session commands through it)
;; [:clojure/eval] an explicit whitelist
;; absent or nil no tools; arm per session instead:
;; (open! :s {::tools true})
;; Armed sessions also get a system-prompt line telling the model where it
;; lives. Disarm any one session with (open! :s {::tools nil}) — or fork the
;; same tape both ways and compare.
;; :tools true
;; Trace durability — the daemon's flight recorder (./.llm-repl/).
;; Every committed completion's full request/response, tool results,
;; compact! originals, a transcript of every receipt, and a tape snapshot
;; per session — so tapes SURVIVE daemon/container restarts (auto-recovered
;; at boot, loudly) and every assistant turn has its generation on disk.
;; ON by default for the daemon; --plain (the debug hatch) never traces.
;; The dir self-writes a .gitignore: captured requests carry the resolved
;; preamble, so traces must never reach a repo.
;; :ring-bytes caps the in-memory send-ring (EVERY physical send, incl.
;; tapeless bounce!/trampoline! successes — memory-only by design; query
;; via eval: trace/sends, trace/ring-stats). Default 8MiB; 0/false ≡ off.
;; :trace {:enabled? true :dir ".llm-repl" :ring-bytes 8388608}
;; nREPL attach port. 0 picks a free port. Either way the chosen port is
;; written to .nrepl-port so editors and clients can auto-connect.
;; :bind (default "127.0.0.1") is the listen address. "0.0.0.0" opens the
;; attach surface beyond loopback — nREPL is unauthenticated eval, so only
;; do that behind a wall (e.g. a container publishing loopback-only; see
;; docker/config.edn, which pairs it with a fixed port).
:nrepl {:port 0}
;; :attach — make `bb llm-repl` (no flag) auto-attach its TUI to a core
;; running ELSEWHERE (e.g. the container) instead of starting a local one.
;; The clojure half evals where that core lives (next to /work and the
;; model); prose turns fire there too — the render surface is just remote.
;; Attach is a CONTRACT: if this is set and the target is unreachable, the
;; repl ERRORS AND EXITS (a fresh local session would silently mask a down
;; container as lost state). Unset ≡ local; there is no silent fallback.
;; Shapes:
;; "127.0.0.1:7899" host:port
;; "7899" bare port (host defaults to 127.0.0.1)
;; {:host "…" :port 7899} map form
;; true read ./.nrepl-port (the mounted /work seam —
;; zero-config when run from the bind-mounted dir)
;; --attach on the CLI forces/overrides this; --headless/--plain ignore it.
;; :attach "127.0.0.1:7899"
}