Why and when to run local

AI Foundations introduced the split between cloud AI and local AI: where a model runs decides where your data goes. This section turns that one idea into a practical decision, starting with the honest case for choosing local, one reason at a time.

Four reasons builders choose local

• Privacy: your prompt and any files you attach are processed on your own device. Nothing is sent to a company's servers for that request.
• No ongoing cost: once your hardware can run a model, using it costs nothing extra per prompt. There is no subscription and no per-token bill for that usage.
• Works offline: a local model runs without an internet connection. A flight, a slow connection, or a service outage does not stop you.
• Full control: you choose which model runs, when it updates, and how it is configured. No provider can change the model under you or shut off your access.

Why it adds up to more than one reason

These four reasons compound. A model you control, that keeps working with no network and no per-use bill, is one you can build a habit around without worrying it will change under you or run up a bill you did not plan for. That is the case for local: not that it beats the cloud at everything, but that it wins clearly on the things that matter for a lot of everyday and sensitive work.

It is easy to turn cloud vs local into a loyalty question. It is not one. Each has real strengths, and an honest comparison is what lets you choose correctly for a given task instead of defending a side.

When cloud wins

• The task needs the best quality available and the gap matters: hard reasoning, unfamiliar domains, the largest coding jobs.
• You need it working right now, with no setup and no hardware requirement.
• The workload is occasional enough that a subscription or per-use price beats buying hardware.

When local wins

• The data is sensitive: client files, health information, unpublished work, anything you would not want leaving your device.
• You use AI often enough that an ongoing subscription or per-token bill adds up.
• You need it to work without an internet connection, or without depending on a service staying up.

"Free" and "no ongoing bill" can make local sound like a pure win. It is not. You are trading a cloud provider's recurring price for a set of costs you pay yourself, some in money and some in effort. Here they are, without the sales pitch.

Four costs to plan for

The quality gap, honestly

This one deserves a straight answer. As of today, the biggest cloud models generally still lead on the hardest tasks: complex multi-step reasoning, unfamiliar or highly technical domains, and the largest coding problems. That gap has narrowed steadily, and for a large share of everyday work, drafting, summarizing, everyday coding help, chat, a well-chosen local model is genuinely good enough. Treat that as a rough pattern, not a guarantee for every model or every task.

You do not have to pick one lane forever. The useful question is smaller: for this task, right now, is local or cloud the better fit? Three questions get you most of the way there.

The same builder, different tasks

• Drafting personal notes or brainstorming: local is fine, and keeps it private.
• Debugging an unusual, hard problem where you want the strongest model available: cloud is worth it.
• Working through a client's confidential documents: local, by default.
• A quick one-off question with no sensitive data and no capable machine handy: cloud is simpler.

"Your data stays on your machine" is the headline claim for local AI. It is worth being precise about what that actually means, and where the edges of the claim sit.

What happens when you run a model locally

1. 1You type a prompt into a local app.
2. 2Your own device, CPU, GPU, or both, processes it. No request is sent anywhere.
3. 3The model generates a response using only your machine's hardware.
4. 4The response appears on your screen. At no point did the prompt or the response leave your device.

• No third-party server ever sees your prompt.
• No company can log it, review it, or use it to train a future model.
• It works exactly the same with the network cable unplugged.

AI Foundations covered why this distinction matters for what you should and should not paste into an AI tool, in Using AI well. The rest of this course is about setting up a local model so this payoff is real for you.