Hardware & System Requirements

Minimum - the app runs, CPU-only inference with small models:

Recommended - comfortable agentic use with mid-size models:

If you have no discrete GPU, Bodega detects CPU-only mode and caps context based on system RAM. You'll see CPU mode - context capped for sustainable inference in the AI panels. 1–4B models are usable; anything larger will be slow.

On every launch, Bodega probes your GPU VRAM using the systeminformation library. The result feeds three things:

• Model recommendations - the Discover tab's Fits My GPU filter and first-run model cards
• Context window ceiling - the max tokens Bodega will send to a local model (written to hardware.local_context_ceiling)
• Cloud Boost escalation - whether your hardware tier triggers automatic escalation on reasoning-heavy tasks

The detection runs once at startup. VRAM capacity is cached for the session. Free VRAM refreshes every 3 seconds so that a model already loaded is correctly accounted for.

Apple Silicon: Bodega uses total system RAM minus 3 GB as the effective VRAM pool. This matches how unified memory actually works - your GPU and CPU share the same physical pool. The onboarding badge shows your chip name and memory figure.

No GPU: vramGB is reported as 0. Bodega sets isCpuOnly = true and switches to RAM-based context ceilings.

Multi-GPU: Bodega uses the largest single GPU's VRAM. Ollama loads models onto one GPU, not split across cards.

A live hardware readout appears at the top of the Hardware section in the Help panel.

1. Open the Help panel: gear icon (top right) → Help
2. Select Hardware in the left rail
3. The Your System card at the top shows: CPU model, total RAM, GPU model, VRAM, and a one-line capability summary

The summary uses these thresholds:

The card fetches from the backend on mount. If the backend isn't running when you open it, the card won't appear - no error is shown.

Bodega uses two separate hardware classification systems. They serve different purposes and have different thresholds - do not confuse them.

Discover tab tiers (used for model recommendations and the Fits My GPU filter):

Routing tiers (used by the agentic loop for model selection and Cloud Boost escalation decisions):

The routing tiers also expose two capability flags: canRunSmartTier (13B+ accessible, requires 10 GB) and canRunCodeTier (code-specialized 7B models, requires 6 GB).

For Q4_K_M quantization - the default in Bodega's managed llama.cpp mode - model weight size in VRAM is roughly:

VRAM (GB) ≈ parameters (B) × 0.55 + KV cache + ~1 GB overhead

Actual sizes from Bodega's catalog (Q4_K_M):

KV cache is the hidden cost. An 8B model's KV cache grows from about 1.2 GB at 8K tokens to ~20 GB at 128K tokens. Context length is where VRAM gets consumed faster than the model weights table suggests. Bodega's context ceilings are set conservatively to prevent thrashing during agentic runs.

After VRAM detection, Bodega computes a context ceiling and writes it to hardware.local_context_ceiling. This caps how many tokens the agentic loop sends to local models.

GPU path (VRAM-based):

CPU-only path (RAM-based, no discrete GPU):

The ceiling calculation uses max(free VRAM, total VRAM − 4 GB) rather than free VRAM alone. This prevents a model that's already loaded from causing the probe to misclassify your card. For example: a 32 GB GPU with a 26B model resident still probes as a 28 GB card for ceiling purposes, not a 4 GB card.

To raise the ceiling manually: Settings → (search) context_window_cap → set llm.context_window_cap to a positive integer. Set it to 0 to return to automatic. The budget bar in each AI panel reflects the active ceiling.

LLM inference is memory-bandwidth-limited, not compute-limited. VRAM capacity and bandwidth matter more than GPU core count.

AMD: The RX 7900 XTX has 24 GB VRAM but ROCm software support on Windows is inconsistent. On Linux, AMD gets the ROCm llama.cpp binary. On Windows, AMD gets the Vulkan build (HIP Radeon support is experimental).

NVIDIA on Linux: Bodega ships a Vulkan llama.cpp binary for Linux NVIDIA - not CUDA. CUDA binaries for Linux are not included in Bodega's managed installation.

On M-series Macs, the GPU and CPU share the same memory pool. Bodega detects Apple Silicon via process.platform === 'darwin' && process.arch === 'arm64' and uses total RAM − 3 GB as the effective VRAM figure.

Approximate memory bandwidth by chip (from Apple's published specs):

Higher bandwidth means faster token generation - bandwidth scales matter more than raw VRAM here.

When you install llama.cpp through Bodega on macOS, it installs the macos-arm64 binary, which uses Metal acceleration. No configuration required.

If you're using Ollama on Apple Silicon, Ollama handles MLX acceleration automatically. MLX runs roughly 20–30% faster than llama.cpp for inference on Apple Silicon.

Free memory note: macOS manages unified memory dynamically (compression, paging). The free memory reading Bodega sees can fluctuate. The probe uses os.freemem() with a 1 GB kernel reserve as a proxy.

When you install llama.cpp through Bodega (first-run onboarding or Settings → Providers → llama.cpp → Install), Bodega picks the right binary for your system automatically:

Bodega reads the CUDA version directly from nvidia-smi output (not inferred from driver version numbers). If nvidia-smi times out or is absent, it falls back to driver major version, then to Vulkan or CPU. The install is never blocked by a missing nvidia-smi.

Air-gap mode: the binary install endpoint returns 403. Use llamacpp.binary_path_override in Settings to point to a manually downloaded binary.

Cloud Boost lets you configure a cloud provider (OpenAI, Anthropic, and others) as a secondary fallback. On low-end hardware, it activates automatically for tasks that need it.

To configure:

1. Go to Settings → Cloud Boost
2. Enable the toggle
3. Choose a cloud provider
4. Enter your API key
5. Set optional daily and monthly budget limits

Auto-escalation triggers - both conditions must be true for trigger B:

• (A) The local model's QEL verification score falls below 50 for two consecutive iterations in the same session, OR
• (B) Hardware tier is minimal (< 6 GB VRAM) or budget (6–10 GB VRAM) AND the current task is classified as reasoning-heavy

Simple conversational messages on minimal hardware do not trigger auto-escalation. The task has to be a planning or reasoning task.

You can also toggle Cloud Boost manually in the chat input area.

Cost tracking: gear icon → Usage Dashboard

Air-gap mode: Cloud Boost is fully blocked when air-gap is enabled (Settings → Privacy & Safety → Air-Gap).