How to run Qwen3.8-27B locally in Bodega One Code

This is the hands-on version: why the model is everywhere this week, how good it is, what fits your hardware, how fast it runs, both ways to set it up in Bodega One Code, and the problems to know about before you hand it an agent task.

Why is everyone talking about Qwen3.8-27B this week?

Because a 27B model on a laptop did something people assumed needed a frontier model. On November 2, a post on “weight-space programming” claimed a frontier model could read and edit a small model's raw weights, and it became the top AI story on Hacker News (2,951 points). A rebuttal argued the headline result relied on running code, not just reading weights. Then Simon Willison tried a smaller version of the idea locally: he gave Qwen3.8-27B (the 4-bit build, about 17 GB, in LM Studio) the raw weights of small game-playing networks and asked which Atari game each one played. It named 3 of 4 and missed Freeway. The three timed runs took 46 minutes to just over an hour each. He calls it his favorite local model.

The week before, a project called Memdoor said a team of five Qwen3.8-27B instances beat Fable 5 on LiveCodeBench Hard, which reached 465 points on Hacker News. Read the details before quoting it. The team ran through a cloud API, not locally. It covered only the first 10 hard problems, five times each (9.8 out of 10 against 8.0 for a single Fable 5 call). The page says its edge over the original authors' own harness is within noise, and the Fable 5 numbers come from the authors' files rather than a fresh run.

What is Qwen3.8-27B?

Qwen3.8-27B is a 27.8-billion-parameter dense model that reads text, images and video and writes text, with a 262,144-token native context. It mixes two kinds of layers (Gated DeltaNet and gated attention, three to one), which keeps long contexts cheaper than in a standard transformer. It thinks before answering by default, and you can switch thinking off per request or set its depth with reasoning_effort (xhigh, the default, medium or low). Context stretches to about 1 million tokens with YaRN scaling, which Qwen warns can hurt short-text quality. The Apache 2.0 license allows commercial use.

How good is it at coding?

On Qwen's own numbers, it beats its predecessor on every coding benchmark below, and beats much larger models on SWE-bench Pro and LiveCodeBench. These rows are from Qwen's model card:

Two caveats. Qwen ran SWE-bench Pro through the Claude Code harness on a version of the benchmark it corrected itself, while the Opus score is the officially reported one, so the comparison is not like for like. And the card gives no SWE-bench Verified score, the benchmark most model lists use. Independent testing backs part of it: Quesma reproduced Qwen's GPQA and Terminal-Bench 2.1 numbers at full precision, and found the Q4_K_M build matched the full model. Artificial Analysis found it very wordy at its default effort, using 128 million output tokens on its test suite, with about two minutes before the first answer token through an API. Expect long thinking unless you turn effort down. For how to read these scores, see our local LLM comparison.

Will it fit on your GPU?

The 4-bit build is the one to start with: it fits a 24 GB card, and in Quesma's tests it scored as well as the full model. File sizes from Unsloth's builds on Hugging Face (decimal GB):

Context costs memory on top of the file, so a bigger file means a smaller window. Skip the 1-bit builds: testers found they loop and overthink. On a Mac, Bodega One Code does not count all your unified memory as GPU memory. It budgets about 70% of RAM (75% at 64 GB and up), so a 24 GB Mac has roughly 17 GB to work with, which is too tight for the 4-bit build plus context. 32 GB or more is the comfortable floor.

How fast is it?

On a 24 GB NVIDIA card, expect roughly 30 to 50 tokens per second, and much more with its built-in drafter switched on. Qwen trained a multi-token prediction head into the model, and llama.cpp can use it to draft tokens ahead. These are community measurements collected in one GitHub repo, single stream, on different llama.cpp builds and settings, so treat them as a rough guide:

Two more data points with different setups. NVIDIA measured 28.3 tokens per second with the 4-bit build and a DFlash drafter on RTX Spark hardware. A Mac Studio with an M3 Ultra ran Ollama's 4-bit build at about 14 tokens per second. Drafting helps one conversation at a time, so it fades when several requests run at once, and it did not help on an Apple M4 in the same repo. We have not measured Bodega One Code on any of these machines, so these are not our numbers.

How do you run Qwen3.8-27B in Bodega One Code with Ollama?

Ollama is the simplest path: one pull, one context setting, then pick the model in Code. Ollama added Qwen3.8-27B in version 0.32.12, so update first. The default tag is the 4-bit build, 18 GB:

$ollama pull qwen3.8:27b

Then raise Ollama's context window. Ollama sets it from your VRAM: 4K under 24 GB and 32K from 24 to 48 GB. Its own docs say coding tools need at least 64,000 tokens, and a coding agent's tool list and project rules fill a small window fast. The easy way is the context slider in the Ollama app's settings. From a terminal, quit the Ollama app, then start the server with a bigger window:

$OLLAMA_CONTEXT_LENGTH=65536 ollama serve

In PowerShell on Windows:

PS>$env:OLLAMA_CONTEXT_LENGTH=65536; ollama serve

Check it took with ollama ps: the CONTEXT column should read 65536, and PROCESSOR should say 100% GPU. If part of the model spills to the CPU, it gets much slower, so lower the context until it fits.

Now open Bodega One Code. On first launch the setup screen scans for local providers, and a running Ollama shows up under On this computer. If you are already set up, go to Settings > Models & Providers > Providers and make Ollama active. Pick qwen3.8:27b from the model chip in the composer. Expand the model in My models and set its context window to the same 65536, so Code plans its context around the size you gave Ollama. Our Ollama setup guide covers the rest of the provider settings.

How do you run it with llama.cpp inside Bodega One Code?

The other path is to let Bodega One Code run llama.cpp for you and download the model from its built-in catalog. Code starts and stops its own llama-server, so there is no server to manage:

1. Go to Settings > Models & Providers > Providers > llama.cpp and choose Set as primary. If you have never used llama.cpp in Code, the setup flow installs the engine first.
2. Open Models > Discover, filter to Code, and turn on Only fitting to hide builds too big for your GPU.
3. Pick Qwen3.8-27B and download it. The catalog needs llama.cpp build b7990 or newer and checks before it offers the download. If yours is older, update it from Settings > Models > My Models > llama.cpp engine. Nothing updates in the background.

What you get for the extra step: if your card cannot hold the model's full context, Code cuts the context to a size that loads and tells you so, instead of falling back to the CPU without saying anything. When you have not set sampling yourself, it adds a small anti-repetition floor that keeps local models from looping. And a model downloaded from the catalog gets the model's full set of reasoning levels in the effort control. The same file added by hand does not, on purpose: an edited copy of this model once looped for over four minutes after picking up those settings by matching the file name.

If you want the MTP speed-up from the table above, run llama-server yourself with --spec-type draft-mtp --spec-draft-n-max 2 --parallel 1 and point Code's llama.cpp preset at it. When Code cannot start its own server, it uses whatever is already listening at the preset's address. llama-server now defaults to port 9931, not 8080, so pass --port or change the preset's base URL to match.

What settings should you use?

Use Qwen's recommended sampling, and turn thinking off for quick edits. From the model card:

In Bodega One Code, expand the model under My models: temperature is one of its per-model overrides, and top_p, top_k and min_p are in its Advanced Sampling panel. Code's docs list no presence penalty control, so the thinking-off row cannot be matched exactly there. The composer's model chip has the thinking control. On a catalog download through llama.cpp you get the model's graded levels, and Code's global High setting maps to the model's highest, most expensive level. Through Ollama you get an on/off switch, and an open Ollama issue reports that Ollama runs this model at its medium depth, not Qwen's xhigh. At full effort this model thinks for a long time, so for renames, small fixes and boilerplate, switch thinking off and save it for planning and debugging.

What problems should you know about?

• Ollama tool-loop error: an open Ollama issue reports a 500 error, “no user query found in messages”, after several rounds of tool calls. That is exactly what a coding agent does. If you hit it, try the llama.cpp path.
• Overthinking: at the default effort, answers can take minutes. Lower the effort or turn thinking off.
• Old llama.cpp builds: builds that predate the model fail with “unknown model architecture: qwen35”. Update before you blame the model.
• Very small builds: 1-bit builds loop, and 2-bit loses some agent accuracy.
• Macs: an open Ollama issue reports its MTP builds running about half as fast as the plain ones on Apple Silicon, so use the plain or MLX tags.

Should you pair it with a cloud model?

For most everyday coding, the local model alone is enough. For the hardest jobs, splitting the work is cheaper than sending everything to the cloud. A custom agent in Bodega One Code can run on a different provider from the one you are chatting with, so a cloud model can plan a large change while Qwen3.8-27B does the edits for free on your machine. Cloud models cost what your provider charges; local runs cost nothing per token. See BYOLLM for the providers Code supports, and air-gap mode if your code must never leave the machine.

Sources

• Qwen3.8-27B model card (specs, benchmarks, sampling, thinking controls): huggingface.co/Qwen/Qwen3.8-27B
• Alibaba Cloud announcement: alibabacloud.com
• Simon Willison, Qwen can interpret model weights, November 3, 2026: simonwillison.net
• Weight-space programming post and rebuttal: rikiyat.github.io, honnibal.dev
• Memdoor team-solve results: github.com/guregodevo/memdoor
• Quesma quantization benchmarks: quesma.com
• Artificial Analysis model page: artificialanalysis.ai
• Unsloth GGUF builds and file sizes: huggingface.co/unsloth/Qwen3.8-27B-GGUF
• Community MTP speed results: github.com/sudoingX/qwen38-mtp
• NVIDIA RTX Spark speed table (Nemotron 3.5 Bolt page): research.nvidia.com
• Mac Studio M3 Ultra test: terminalbytes.com
• Ollama model tags and context-length docs: ollama.com/library/qwen3.8, docs.ollama.com/context-length
• Ollama issues: tool-loop error #17778, MTP speed on Apple Silicon #17776, thinking depth #18632
• llama-server options, including the default port: github.com/ggml-org/llama.cpp
• Bodega One Code docs: models and providers, reasoning, hardware