Local runtimes compared: Ollama, LM Studio, llama.cpp

A runtime is the program that loads a model into memory and does the math to generate a reply. It is a different thing from the model file itself. The model is the weights, most commonly shipped as a GGUF file. The runtime is what reads that file and turns your prompt into tokens, then tokens back into text. Three names come up constantly for running models locally: llama.cpp, Ollama, and LM Studio. They are not three competing versions of the same thing. They sit at different layers, and two of them are built on top of the third.

The question worth asking is not "which is best" but "how much do I want to type versus click, and how much control do I actually need." A CLI tool gets out of your way once you know the commands and scripts cleanly into anything else you build. A GUI gets you from zero to a working chat in minutes with nothing to remember. A raw engine gives up convenience for the fewest layers between you and the hardware. All three read the same kind of file and, as this guide gets to, all three can talk the same API, so the choice is lower-stakes than it first looks.

llama.cpp: the engine underneath

llama.cpp is a C/C++ inference engine for large language and vision-language models. It has no external dependencies and is built for minimal setup with strong performance across a wide range of hardware, CPUs included.¹ It defines the GGUF model format that most of the local-AI ecosystem now uses, and it ships both a command-line tool and a server binary with a built-in web UI and an OpenAI-compatible API.¹

You interact with llama.cpp by compiling it (or grabbing a prebuilt release) and running a binary from the command line: llama-cli for a chat session in your terminal, llama-server to expose an HTTP API. There is no installer, no app icon, and no model browser. You point it at a GGUF file, on disk or fetched straight from Hugging Face, and it runs.

This is the right layer for a narrow but real audience: people embedding inference directly into their own application, people running on unusual or brand-new hardware where a wrapper tool has not caught up yet, and people who simply want to know exactly what is happening between the prompt and the reply. It is overkill if you just want to chat with a model this afternoon.

Ollama: the CLI and daemon most developers reach for

Ollama is a CLI plus a background server for running open models on your own computer.² Its own documentation lists llama.cpp, the project founded by Georgi Gerganov, among its supported inference backends,³ so under the hood it is running the same kind of engine, wrapped in a Go server that adds a model library, a pull/run workflow, and a REST API on top.

Day to day, you talk to Ollama through the terminal: ollama pull to download a model, ollama run to chat with it, ollama ls to see what is installed. There is no required GUI. It also runs quietly in the background once installed and starts on login, so any app on your machine can reach it over its local API without you opening a window at all. The full walkthrough, install to first chat to editor integration, is in how to run a local model with Ollama.

This is the right layer if you are comfortable in a terminal and want a model running in one command today, without reading a build guide first. The trade-off is that you are one step removed from the raw engine: fewer flags to tune, less visibility into exactly which backend loaded, and you are trusting Ollama's defaults unless you go looking for the advanced options.

LM Studio: the GUI for people who do not want a terminal

LM Studio is a desktop application for discovering, downloading, and chatting with local models, on Windows, macOS, and Linux, with everything kept on your device.⁴ LM Studio supports running models on Mac, Windows, and Linux using llama.cpp, and on Apple Silicon Macs it also supports Apple's MLX,⁷ so it inherits the same engine work as Ollama rather than reinventing it.

You interact with LM Studio the way you would with any desktop app: search a model in the built-in catalog, click download, open a chat window. No terminal required for the core workflow. For people who do want programmatic access, LM Studio ships a CLI (lms) alongside JavaScript and Python SDKs, and, like the other two, a local server exposing an OpenAI-compatible API.⁵

This is the right layer if the idea of a terminal is itself a barrier, or if you would rather see model size, quantization, and hardware fit laid out visually before you commit to a download. The trade-off is the opposite of llama.cpp's: you get a friendlier surface, but a GUI is more to run in the background than a lightweight CLI daemon, and it is less natural to script into a headless server or a CI pipeline than Ollama or a bare llama-server process.

All three at a glance

Same files, different front doors

It is worth restating plainly: the runtime is not the model. GGUF is a packaging format, weights plus metadata in one file, and it is the format llama.cpp defined and all three tools read.¹ That shared format is what makes the choice of runtime reversible. A GGUF file downloaded through LM Studio's catalog will also load in Ollama or a bare llama-server, and a file you pulled with ollama pull lives on disk as a real file you could, if you wanted to, point another tool at directly.

That also means raw generation speed is usually decided more by which quantization level you picked and what hardware you are running on than by which of these three tools is doing the loading, since the same kind of computation is often happening at the same layer underneath. Do not expect switching runtimes alone to fix a model that is too slow or too large for your machine; that is a model-and-quantization problem, covered in which GGUF quantization level should you pick.

The one thing all three share: an OpenAI-compatible server

This is the detail that makes the choice lower-stakes than it looks. Every one of these three can expose a local HTTP server that speaks the same request and response shape as the OpenAI API, so any tool, editor plugin, or SDK that lets you set a custom base URL can point at your local model instead of a cloud one.

Picking one

• Pick llama.cpp if you want the smallest, most controllable footprint, you are building inference into your own application, or you need a hardware path (a new GPU, a new quantization scheme) before it lands anywhere else.
• Pick Ollama if you want a model running in one command, a library to pull from, and a local API you can point other tools at without babysitting a GUI.
• Pick LM Studio if you would rather click through a model catalog and chat in a window, and you still want a local server available for later if you start building.
• Pick more than one if that is honestly fine. They read the same GGUF files and expose the same kind of API, so nothing stops you from using LM Studio to browse and Ollama to run the thing your editor talks to.

Honest trade-offs

None of these three is strictly better. llama.cpp gives you the most control and the least abstraction, at the cost of doing more yourself: compiling or fetching the right binary, managing model files by hand, reading flags instead of clicking a button. Ollama trades some of that control for a workflow that gets out of your way, but you are one layer removed from the raw engine and its flags. LM Studio trades the terminal for a GUI, which is exactly right for a newcomer and mildly in the way once you are scripting everything anyway.

Setup cost and automation-friendliness tend to move together. The two CLI-first tools, llama.cpp and Ollama, run headless without any window at all, which is what you want on a home server, inside a Docker container, or on a machine you only ever reach over SSH. LM Studio is a desktop app first; it works well on the machine sitting in front of you, and less naturally as something you install on a headless box, even though its server and CLI exist for exactly that gap.

None of the three chooses your model for you, and none of them changes what quantization does to a model's memory use and quality, that trade-off lives in the GGUF file itself, not the runtime reading it.

Read next: how to run a local model with Ollama is the full install-to-editor walkthrough with one of these three, which GGUF quantization level should you pick covers the other half of the decision, the model file itself, and the free Running Local AI course builds up tokens, context windows, and local-versus-cloud AI from first principles if any of that is still new.