Launchers

These launchers make local llama.cpp models usable from Codex, OpenCode, and Claude Code without repeating provider flags every time. Each wrapper solves a different harness-specific problem: Codex needs local model metadata plus provider routing, OpenCode needs model ids qualified to its configured provider, and Claude Code needs a llama.cpp-scoped settings file.

All four launchers acquire a shared router lease. They join an existing router, or start a loopback router when none is reachable. After the last managed consumer exits, the auto-started instance stays resident for a 10-minute grace period so the next harness can reuse its loaded model; run ,llama-cpp stop to end it earlier or set LLAMA_CPP_GRACE_SECONDS=0 for immediate shutdown. Running ,llama-cpp serve first remains supported, and that manually started process is left running.

Mental model

Codex has two layers. The transparent ,codex wrapper supplies catalog metadata for the llama.cpp router ids, while ,codex-llama-cpp supplies the provider routing flags that point Codex at llama-server.

OpenCode reads providers from ~/.config/opencode/opencode.jsonc, so its launcher only normalizes model selection and passes the rest through.

Claude Code has one global autoCompactWindow, but cloud claude-opus-5-5 and 262144-token local models need different values. The llama.cpp launcher picks a model-scoped additive settings file: base local models and Qwen3.6 use 200000. Plain cloud Claude sessions and their subagents use the 600000 from the user settings. Those additive files keep thinking off and pin each local model to high without changing local context windows.

Using it

No separate server command is required for these launchers. Start any harness directly; use ,llama-cpp serve only when you want the router to remain available independently of harness sessions.

Which llama-server the launchers run

Every launcher reaches the router through ,llama-cpp run, so they all inherit one binary resolution order, implemented in both main.sh (serve) and lifecycle.py (run):

Order	Source	Used when
1	$LLAMA_CPP_SERVER_BIN	set (absolute/relative path, or a bare name looked up on PATH)
2	$LLAMA_CPP_PRISM_ROOT/bin/llama-server	that file exists and is executable
3	llama-server on PATH	otherwise (Homebrew llama.cpp)

When none resolves, the error names all three sources. Homebrew llama.cpp stays installed and is the fallback; building the fork never replaces it.

Optional PrismML fork build (,llama-cpp build-prism)

The optional PrismML fork build remains available for compatible future weights. The current Qwen3.6 and Nemotron roster does not require it. The fork ships no macOS release binaries, so build-prism builds it from source:

,llama-cpp build-prism                                   # newest prism-* tag
,llama-cpp build-prism --tag prism-b10687-5d80cff        # pin a tag
,llama-cpp build-prism --force                           # rebuild the same tag

The command is idempotent: it clones on first run and fetches tags afterwards, picks --tag or the newest prism-* tag by creation date, and exits with up to date (<tag>) when bin/.tag already records that tag and bin/llama-server exists. Otherwise it checks the tag out, configures a Release build (LLAMA_BUILD_TESTS=OFF, LLAMA_BUILD_EXAMPLES=OFF, LLAMA_BUILD_TOOLS=ON, LLAMA_CURL=ON; Metal is the macOS default), builds the three targets, installs them plus any shared libraries they need, records the tag, and prints llama-server --version.

Layout under $LLAMA_CPP_PRISM_ROOT (default ~/.llama.cpp/prism):

Path	Contents
llama.cpp/	fork checkout, parked on a prism-* release tag
llama.cpp/build/	cmake build tree
bin/llama-server, llama-cli, llama-bench	installed binaries
bin/.tag	tag the installed binaries were built from

After a successful build, step 2 of the resolution order picks the fork up automatically — no launcher flag or env var change is needed.

Codex launcher metadata

Codex only has first-class model metadata for slugs present in its model catalog; unknown local slugs use fallback metadata and emit a warning. This repo ships a transparent ,codex wrapper plus a small local catalog for the llama.cpp models.

The wrapper injects -c model_catalog_json="$HOME/.codex/llama-cpp-model-catalog.json" when the selected model is one of the llama.cpp router ids, in either --model <id> or --model=<id> form.

Other Codex invocations execute the real Codex binary directly.

Codex launcher (,codex-llama-cpp)

The ,codex shim above only supplies catalog metadata; Codex still needs the provider routing flags to reach llama-server. ,codex-llama-cpp bakes those in so you don't type them every time.

The wrapper injects:

Codex config key	Value
model_providers.llama-cpp.base_url	http://${LLAMA_CPP_HOST}:${LLAMA_CPP_PORT}/v1
model_providers.llama-cpp.name	llama.cpp
model_provider	llama-cpp

The ~/bin/,codex shim still injects catalog metadata. Pass --model / -m nemotron-3.5 to pick the model.

The wrapper adds its default --model $CODEX_LLAMA_CPP_MODEL only when you did not pass one, so there is no duplicate flag.

,codex-llama-cpp                          # default model nemotron-3.5
,codex-llama-cpp --model qwen3.6-35b-a3b       # Unsloth Qwen3.6 35B A3B
,codex-llama-cpp -m qwen3.6-35b-a3b exec "..."  # one-shot

OpenCode launcher (,opencode-llama-cpp)

OpenCode reads providers from ~/.config/opencode/opencode.jsonc; there is no per-invocation provider override.

The llama-cpp provider is declared in readonly_opencode.jsonc and flows through the merge hook unchanged:

Field	Value
Provider id	llama-cpp
Base URL	http://127.0.0.1:8080/v1
Models	llama.cpp router ids

Each model declares limit.context matching the router: 262144. limit.output=32000 preserves OpenCode's native output allowance and enables automatic compaction before the context fills.

The provider id avoids a dot (llama-cpp, not llama.cpp) because OpenCode's SDK derives an incorrect lookup key from dotted ids.

Pass --model/-m with a bare router id — the wrapper qualifies it to llama-cpp/<id> — or the full llama-cpp/<id>. With no --model, it defaults to llama-cpp/$OPENCODE_LLAMA_CPP_MODEL (nemotron-3.5).

Any subcommand/args pass through.

,opencode-llama-cpp                            # interactive TUI, default model nemotron-3.5
,opencode-llama-cpp --model qwen3.6-35b-a3b run "…"  # Unsloth Qwen3.6 35B A3B

Claude Code launcher (,claude-llama-cpp)

Claude Code compacts conversation history at autoCompactWindow tokens.

Context	Desired value
Cloud claude-opus-5-5	600000 from settings.json
Local llama.cpp	compact below server context so llama.cpp does not reject the prompt

Those needs conflict on a single global setting.

Solution: a dedicated llama.cpp-scoped settings file loaded via claude --settings <file> (layers additively on top of ~/.claude/settings.json), wired through a thin wrapper.

The wrapper:

• exports ANTHROPIC_BASE_URL=http://${LLAMA_CPP_HOST:-127.0.0.1}:${LLAMA_CPP_PORT:-8080}.
• sets ANTHROPIC_API_KEY=$LLAMA_CPP_API_KEY.
• defaults the key to sk-no-key-required because llama.cpp accepts unauthenticated local requests unless started with --api-key.
• invokes claude --settings ~/.claude/settings.llama-cpp*.json "$@".

Pass --model / -m nemotron-3.5 to pick the model.

The wrapper injects its default --model $CLAUDE_LLAMA_CPP_MODEL only when you did not pass one, so there is no duplicate flag.

Variable	Default	Purpose
LLAMA_CPP_HOST	127.0.0.1	Same as ,llama-cpp
LLAMA_CPP_PORT	8080	Same as ,llama-cpp
LLAMA_CPP_API_KEY	sk-no-key-required	Sent as ANTHROPIC_API_KEY (Claude Code uses this for bearer auth)
CLAUDE_LLAMA_CPP_MODEL	nemotron-3.5	Default model; overridden by a caller --model/-m, empty to skip
CLAUDE_LLAMA_CPP_SETTINGS	model-derived	Point at an alternate llama.cpp settings file

The wrapper clears inherited CLAUDE_CODE_AUTO_COMPACT_WINDOW and CLAUDE_CODE_MAX_CONTEXT_TOKENS so an outer hosted session cannot override the local budget. It derives the settings file from the effective model before --. nemotron-3.5 uses ~/.claude/settings.llama-cpp.json with autoCompactWindow=200000; qwen3.6-35b-a3b and qwen3.6-35b-a3b-instruct use ~/.claude/settings.llama-cpp.qwen3.6.json, which also sets autoCompactWindow=200000.

200000 leaves a ~62k token buffer under the 262144-token server context.

env.CLAUDE_CODE_ATTRIBUTION_HEADER=0 stops Claude Code from prepending a per-request x-anthropic-billing-header that would miss the llama.cpp KV cache. Claude Code 2.1.220 copies --settings env into process.env and treats "0" as off.

,claude-llama-cpp                           # interactive session, default model nemotron-3.5
,claude-llama-cpp --model qwen3.6-35b-a3b   # Unsloth Qwen3.6 35B A3B
,claude-llama-cpp -p "summarize README.md"  # one-shot prompt

Cloud Claude sessions are unaffected — plain claude ... still reads only ~/.claude/settings.json, where autoCompactWindow is 600000.

Sources and verification

• home/exact_lib/exact_,llama-cpp/main.sh → ~/lib/,llama-cpp/main.sh (serve binary resolution and build-prism)
• home/exact_lib/exact_,llama-cpp/lifecycle.py → ~/lib/,llama-cpp/lifecycle.py (run binary resolution)
• home/exact_bin/executable_,codex → ~/bin/,codex
• home/exact_lib/exact_,codex/main.py → ~/lib/,codex/main.py
• home/dot_codex/readonly_llama-cpp-model-catalog.json.tmpl → ~/.codex/llama-cpp-model-catalog.json (defines the local llama.cpp router ids and context metadata)
• home/exact_bin/executable_,codex-llama-cpp → ~/bin/,codex-llama-cpp
• home/dot_config/opencode/readonly_opencode.jsonc — declares the llama-cpp provider
• home/exact_bin/executable_,opencode-llama-cpp → ~/bin/,opencode-llama-cpp
• home/dot_claude/settings.llama-cpp.json.tmpl → ~/.claude/settings.llama-cpp.json (base local autoCompactWindow and CLAUDE_CODE_ATTRIBUTION_HEADER=0)
• home/dot_claude/settings.llama-cpp.qwen3.6.json.tmpl → ~/.claude/settings.llama-cpp.qwen3.6.json (Qwen3.6 autoCompactWindow and CLAUDE_CODE_ATTRIBUTION_HEADER=0)
• home/exact_bin/executable_,claude-llama-cpp → ~/bin/,claude-llama-cpp