Models & Providers

There are two ways to get there:

• Status bar click - In Code mode, click the active model name at the bottom of the screen. This opens Settings and navigates directly to the Models section.
• Settings nav - Press Ctrl+, and click Models in the left nav.

The panel has three tabs at the top: Discover, My Models, and Providers. The search field (top-right of the tab bar) works across catalog results and the installed model list simultaneously.

A hardware tier badge (e.g., Large - 24 GB) appears in the panel header when GPU data is available. An air-gap notice appears in the footer when air-gap mode is active.

Bodega ships 36 pre-configured provider presets - 11 local (no API key required), 23 cloud (API key required), a Remote model server (LAN) slot, and a Custom OpenAI-Compatible slot.

To connect any provider:

1. Open Settings → Models → Providers.
2. Find the provider card and click its name to expand it.
3. Fill in the base URL (pre-filled for most presets) and your API key if required.
4. Click Test Connection - success shows the model count and latency (Connected - 5 model(s)).
5. Click Set as Active to make it your primary provider. This saves the key and URL and clears stale model names from your previous provider.

After a successful test, Use This Provider appears as a shortcut that does steps 4 and 5 in one click.

Auto-detect: Click Auto-Detect Providers at the top of the Providers tab. Bodega probes default ports for every local preset. Detected servers appear with a green badge (3 models - 45ms). Auto-detect only checks default ports - custom ports need manual URL entry.

Local server type: A Local server type dropdown appears above the provider list when a local preset is active. Leave it on Auto unless you hit tool-call or FIM issues - mismatching it will break those features.

Local (no API key)

Cloud (API key required)

All six show real token usage in Spending. xAI, DeepInfra, Novita and SambaNova price their curated models from each provider's own published rates (a gateway is billed at the gateway's price, never the upstream model's). Cerebras and Nebius publish no rate Bodega can read, so their calls show as unpriced - a guardrail estimate marked as such, never presented as a real charge.

Plus two open-ended slots:

• Remote model server (LAN) - a model served on another machine on your network over an OpenAI-compatible endpoint (llama-server, vLLM, Ollama, or LM Studio serving on 0.0.0.0). It exists as its own preset rather than a flag so the air-gap posture is legible: remote is explicitly non-loopback, and it is blocked under Air-Gap mode.
• Custom OpenAI-Compatible - any other endpoint.

What switching providers actually clears. It does not wipe every role. It clears four keys - llm.default_model, llm.chat_model, llm.code_model, and fim.model - because a model name from one provider won't resolve on another. Your Research / Debug / Advisor role assignments are pruned separately, only if they're incompatible with the new provider family. One carve-out: switching to llama.cpp keeps a model value that already looks like a GGUF id, so onboarding doesn't erase the model it just downloaded for you.

When Ollama is the active provider, the Discover tab shows a curated catalog and a download input.

Pull any model by name:

1. Type a model name in the Pull Any Model field at the top (e.g., llama3.3:70b, mistral:7b, phi4).
2. Press Enter or click Pull.
3. Progress shows inline: stage label (Queued → Connecting → Downloading → Validating → Finalizing), percent, and bytes/sec. Click Cancel to abort.

After download completes, the model appears in My Models automatically.

Browse the catalog:

• Filter chips: Fits My GPU toggle (hides models that exceed your VRAM) and category filters (All / Code / Chat / General / Reasoning).
• Recommended for Your GPU - up to 4 models that fit and have recommendation badges. Disappears when a filter is active.
• Featured Models - full catalog, sorted by VRAM fit then headroom.
• Cloud Alternatives - collapsed section; expand with the chevron.

The search bar (top-right) narrows by display name, model family, and description.

Air-gap mode disables the Pull input and shows a warning. The curated catalog still loads from local JSON.

Automatic catalog updates: The curated catalog ships bundled as local JSON, and Bodega can also refresh it from a single hardcoded Bodega URL so new models appear without waiting for an app update. The control lives at the top of the Discover tab (Automatic catalog updates). It fetches the model catalog JSON only - no telemetry, and nothing about your machine is sent. Use Refresh now to check on demand. When the toggle is off, a short line on the Discover screen reminds you that new models will not appear until you turn it back on or install the next app update. Under Air-Gap Mode the toggle and refresh are disabled entirely and Bodega never reaches out - it always uses the bundled or cached catalog.

When llama.cpp is the active provider, the Discover tab switches to a GGUF browser (the Ollama UI disappears).

Download from the curated catalog:

1. Set llama.cpp as active: Models → Providers → llama.cpp → Set as Active.
2. Go to Models → Discover.
3. Browse the curated entries (38 shipped in the bundled catalog as of this build; the list grows whenever the catalog refreshes, so trust the tab, not this number). Filter by category (All / Code / General / Reasoning) or toggle Only fitting to hide quants that exceed your VRAM. An Uncensored toggle (off by default) at the end of the filter row reveals community-modified models with reduced refusals - each such entry shows a provenance note stating who modified it, what was changed, and which claims are the author's rather than verified by Bodega. These entries don't get the graded reasoning-effort picker; they use the standard thinking on/off toggle.
4. Click a model card to expand it. Per-quant options (Q4/Q5/Q6/Q8) appear - greyed-out quants exceed VRAM. Click a quant button to select it. MoE models get a second chance: when a mixture-of-experts quant exceeds VRAM but your combined VRAM + system RAM can carry it, the card shows Fits with CPU offload instead of a red badge - expert layers run from system RAM while attention stays on the GPU. The card shows the exact llama-server flag that makes it true (--n-cpu-moe N, or --cpu-moe when the layer count isn't known), with Copy and Apply to this model buttons; Apply saves it into that model's per-model custom args so the next load picks it up. This verdict only appears when Bodega actually knows the model's expert/active-parameter figures (from the curated catalog, or an explicit 30B-A3B-style name); without that metadata the plain VRAM verdict stands - Bodega never guesses a hybrid fit.
5. Click Download <quant> to start. Progress shows stage, percent, and bytes/sec inline. Click Cancel to abort.

HuggingFace search:

1. Expand Browse all GGUFs on HuggingFace at the bottom of the tab.
2. Type a query and press Enter or click Search.
3. Results show download count and likes. HuggingFace results do not have VRAM scoring. To use one, download the GGUF manually from the HF page and side-load it via My Models → llama.cpp section.

Air-gap mode disables HF search. The curated catalog still works.

Side-loaded GGUFs are inspected, not guessed at. A GGUF you download yourself and drop into the llama-models folder is read when Bodega indexes it: its own chat template decides whether it is listed as tool-capable, and its architecture fills in the family. If the model is loaded without its own chat template in play, its row in My Models is marked plain template - hover it for the reason (no tool calling in the template, no template at all, or a header Bodega could not read). Bodega never assumes tool support it did not find, because assuming it produces quietly wrong prompts rather than an error. Existing entries are re-checked on every folder rescan, so a model added before this check existed fixes itself without being re-added.

Bodega routes different tasks to different models using role assignments - nine settings keys under the hood, but My Models now shows only the four that carry real weight: Default, Fast, Smart, and Agent. Model Roles is presented this way as of beta.39; the underlying settings keys and fallback behavior are unchanged, so nothing you had set before was lost.

Global:

• Default - fallback for any role that is left empty.

Auto-routing:

• Fast - quick responses; auto-routing picks this for simple questions.
• Smart - complex reasoning; auto-routing picks this for hard questions.
• Agent - the coding agent (Ctrl+L).

Advanced: per-panel overrides (closed by default - click to expand):

• Chat - used only when Fast is left empty; the fast-tier chain checks Fast first.
• Research - Research panel (Ctrl+Shift+R).
• Debug - Debug panel (Ctrl+Shift+E).
• Advisor - Advisor panel (Ctrl+Shift+A).

Each role uses a text input with typeahead. Type part of a model name to filter the dropdown. Leave any role empty to fall back to Default. After making changes, click Save Model Settings.

On a single-model provider (LM Studio, llama.cpp, KoboldCpp, GPT4All, MLX, TabbyAPI - anything that can only serve one loaded model at a time), the role pickers don't appear at all. Instead you see one card naming the model that is actually loaded and stating plainly that every role routes to it. Setting the other providers' role pickers on these providers was previously accepted and silently ignored - that's what this card replaces.

Note: FIM (Fill-in-the-Middle) autocomplete has its own dedicated panel below the role grid. It does not appear in the role grid and model cards do not show a FIM badge.

Embedding models are excluded from all role pickers - they cannot chat.

1. Open Settings → Models → My Models.
2. In the Model Roles card, assign Default, Fast, Smart, and Agent directly.
3. Need Chat, Research, Debug, or Advisor? Click Advanced: per-panel overrides to reveal those four cards.
4. Click the input for a role and type part of a model name - the dropdown narrows to matching models.
5. Select a model or type a full model ID (Featherless users can paste HuggingFace org/model IDs directly).
6. Leave roles you don't need empty - they fall back to Default (advanced overrides fall back through Agent to Default).
7. Click Save Model Settings.

A VRAM warning banner appears automatically below the role pickers if the DISTINCT models currently assigned across every visible role together would exceed your estimated VRAM - not just Chat and Agent. This does not block saving - it is informational. It only appears on multi-model providers (Ollama, cloud); a single-model provider shows the honest one-model card instead.

When an answer in the transcript was served by a role other than Default (Fast, Smart, Agent, or Chat), a small pill next to the model name in the response header names which role served it.

Below the Model Roles card, My Models shows all installed models grouped by category: Reasoning, Code, Chat, Multimodal, General, Embeddings, Other.

• Star icon on any model row marks it as a favorite. Favorites float to the top in a separate section.
• Click a model row to expand it: shows role badges (which roles that model is assigned to), per-model override controls (temperature, max tokens, context window, reasoning effort), an Advanced Sampling panel (top_p, top_k, min_p, repeat penalty, seed, stop sequences - greyed-out controls show which the active provider does not honor), and a Delete button.
• Delete prompts for confirmation, then permanently removes the model from Ollama. There is no undo.
• The search bar at the top also narrows the installed model list and the role picker dropdowns simultaneously.

If a model appears in Other, it does not have a recognized model profile yet - it still works, it just has no VRAM estimate or category metadata.

llama.cpp loop-prevention floor: if you run a local GGUF (llama.cpp) and leave top_p / top_k / min_p / repeat penalty untouched (no per-model override, no global default), Bodega applies a small built-in floor - repeat penalty 1.1, min_p 0.05, top_p 0.95, top_k 20 - to prevent degenerate repetition loops on sideloaded or fine-tuned models. This only applies to the local llama.cpp provider; Ollama and cloud providers are unaffected. Setting any of these values yourself (per-model or global) overrides the floor. Temperature is never touched by this floor.

At the bottom of My Models, the Task Performance card shows per-model QEL (Quality Evaluation Layer) pass rates from the last 100 tasks.

This data populates automatically after the first code-creation task. No setup needed.

"No QEL data yet" means the model has not completed any creation tasks (new files, functions, routes). Conversational queries and read-only tasks are not scored.

A 0% pass rate at a low average score (e.g., 45/100) is accurate - it means the model is consistently below the QEL threshold for creation tasks, not that something is broken.

FIM is the inline code completion that fires as you type in the editor. It is configured separately from the primary chat provider.

Three modes (select in Models → My Models → FIM (Inline Completion)):

• Primary - reuse the active chat provider for completions.
• Custom - dedicate a separate provider with its own URL and API key.
• Off - disable autocomplete.

Provider quality tiers:

• Native FIM (best quality, lowest latency): Ollama, vLLM, llama.cpp. These use the /v1/completions endpoint with proper FIM tokens.
• Prompt-injection fallback (slower, still works): LocalAI, LM Studio, KoboldCpp, GPT4All, MLX, Jan, Custom.
• Not recommended (charges per-keystroke): all cloud providers - OpenAI, Anthropic, Gemini, Groq, Together, OpenRouter, Mistral, Cohere, DeepSeek, Fireworks, Qwen, Kimi, Azure.

qwen3-coder does not have native FIM tokens and falls back to prompt-injection - slightly slower but functional.

In Custom mode, set the Base URL, optionally set an API key, click Test Connection, then pick a model from the dropdown (or leave on (auto-detect)).

Embeddings power the semantic codebase index used by Ask the Map and codebase search. Configure the embedding provider at Models → My Models → Codebase Embeddings.

Providers:

• Ollama (local, free) - recommended default. Set Base URL (default: http://localhost:11434) and pick a model. Recommended models: qwen3-embedding:4b, nomic-embed-text, mxbai-embed-large, all-minilm, snowflake-arctic-embed.
• llama.cpp (managed) - Bodega spawns the embedding server automatically (see below).
• OpenAI - cloud, metered. Models: text-embedding-3-small, text-embedding-3-large, text-embedding-ada-002.
• Off - disables codebase search.

Toggle Auto-index on project open to build the index 60 seconds after a project loads.

Changing the embedding model after an index exists requires a full rebuild. A banner will appear - click Rebuild now. The old index uses a different vector dimension and is incompatible with the new model.

With the managed option, Bodega spawns a dedicated llama-server instance for embeddings so you do not have to run it yourself.

Requirements: The llama.cpp binary must be installed (Settings → Models → llama.cpp section). At least one GGUF must be downloaded via the llama.cpp Discover tab.

Setup:

1. In Models → My Models → Codebase Embeddings, select llama.cpp.
2. Enable Let Bodega manage the embedding server.
3. Pick an installed GGUF from the dropdown, or type a full file path.
4. Set the port (default: 8081). This is separate from the chat server on port 8080.
5. Save settings. Bodega starts the server on next use.

GGUFs appear in the dropdown only after they have been downloaded via the llama.cpp Discover tab.

When you attach an image to a message, Bodega routes it based on your active provider:

• Cloud vision-capable models (Claude Sonnet, GPT-4o, Gemini) - the image is sent inline. No swap, no delay.
• llama.cpp with a text-only model - if configured, Bodega pauses the text model, loads a VLM (vision language model), answers the vision question, and resumes. The UI shows swap progress: swap_started → loading_weights → ready → querying → complete. Swap typically takes 10–60 seconds.
• Ollama with a multimodal model - Ollama handles the swap internally with no visible pause.

Configure vision in Settings → Models → Vision Binding:

• Vision engine - Auto (prefers Ollama to avoid the pause) / Ollama / llama.cpp.
• Allow llama.cpp model swap - disable this if you want vision questions declined rather than triggering a slow swap.
• Vision companion - pin a specific installed VLM as the default pair for your text model, or leave it on Automatic to let Bodega pick the smallest that fits. The picker lists each installed vision model with its engine (Ollama internal swap or llama.cpp process hot-swap) and a rough VRAM estimate. A per-session pick still overrides this default.

To install a vision model:

• llama.cpp: download a VLM-capable GGUF (e.g., LLaVA) via Models → Discover.
• Ollama: pull a multimodal model (llava:13b, etc.) via the Pull Any Model input.

If llama.cpp swap is disabled and there is no Ollama VLM, vision questions are declined with a message directing you to install a VLM.

Fast Mode skips the extended-thinking pre-pass on models that do it, so replies start sooner. The underlying model is the same - only the pre-pass is skipped.

Click the Fast toggle in the message composer, next to the reasoning control. It appears only when the active model supports extended thinking.

From that point on, those calls skip extended thinking unless you set a reasoning level on a specific message with the composer's reasoning control - the per-message control always wins.

The setting key is llm.claude_fast_mode (default off). It has no effect on models that don't do an extended-thinking pass.

When llama.cpp is your active preset, Bodega spawns llama-server itself, on port 8080, using the binary it installs for you. You do not start it, and you should not health-check it as if it were an external service - readiness is a state machine (idle → starting → ready → swapping), and every request waits on the manager confirming the requested model is loaded rather than on a port being open.

What that ownership actually buys you:

• Crash recovery, bounded: up to 3 restarts within any 60-second window, so a genuinely broken configuration surfaces as an error instead of a restart loop.
• Backoff on your own flags. If a spawn with Performance Mode or custom args crashes, the retry drops those extras and spawns bare - a typo'd flag costs you the flag, not the provider.
• Teardown on quit, wired into the backend's shutdown path. stop() reports whether the OS process was actually confirmed gone, not just that it was asked to stop.
• Orphan reaping at boot - but by executable path only, never by process name. A llama-server you started yourself is never killed.

The honest exception: if Bodega fails to bring its own server up, it falls back to talking to whatever is already listening at the configured base URL, on the assumption that you're managing one yourself. That path is real and supported - point any local preset at your own server (or llm.llamacpp_api_key at an auth-gated one) and it works. It just means "the app owns the process" describes the normal case, not an absolute.

Power-user controls for the managed llama-server process are in Models → My Models → Advanced Flags (visible only when llama.cpp is the active provider).

Available controls: GPU layers, context size, batch size, and a free-text field for extra flags (llamacpp.custom_args). On machines where Bodega detects two or more splittable GPUs (same-vendor cards llama.cpp can actually split across - an iGPU doesn't count), a Multi-GPU split editor also appears: per-GPU proportion fields for llamacpp.tensor_split plus a split-mode select (--split-mode layer or row). Single-GPU machines never see it, and the flags are dropped at launch if the device count doesn't support them. Per-model overrides keyed by GGUF filename live in llamacpp.model_custom_args and are applied after the global string, so the per-model value wins.

Bodega strips the flags it owns out of whatever you type - --host, --port, -m - and logs what it removed. Under Air-Gap mode --rpc is stripped too. Changes take effect on the next server start.

Both of those keys are global-only. They are deliberately excluded from what a project's .bodega/config.json may set, so cloning a repo can never change which flags llama-server starts with on your machine.

Bodega detects your GPU and assigns a hardware tier, shown in the Models panel header and used to score which catalog models fit.

Apple Silicon uses unified memory - all RAM counts as VRAM.

The tier is computed from FREE VRAM when free VRAM is readable, falling back to total only when it isn't. That is why the badge can drop a tier while a model is already resident, or while another application is holding the card. If your tier looks wrong, check what else is on the GPU before you doubt the detection.

The Fits My GPU filter on the Discover tab hides models that would exceed your VRAM. Note the two catalogs do not use the same arithmetic: the GGUF catalog reserves ~4 GB for the OS and then allows 5% over budget per quant - but only when VRAM detection confidence is high; when it isn't, the slack drops to zero and the budget is enforced exactly. The Ollama catalog allows 10% over unconditionally. A model can therefore show as fitting in one tab and not the other - neither is a bug, they're different estimates.

The VRAM warning in Model Roles uses the same hardware data to flag when the distinct models currently assigned across every visible role combined would exceed available VRAM - not just Chat and Agent.