Securing AI apps: prompt injection and beyond

A plain chatbot has a narrow attack surface: whatever a user types into the box. Give that same model tools, the ability to send an email, run a shell command, edit a file, hit an API, and let it read content the user didn't author, a web page, an inbox, a PDF, a code repository, and the attack surface changes shape entirely. The model can't reliably tell the difference between an instruction from the person who set it up and an instruction that showed up inside the content it's reading. That single fact is the root of almost everything in this guide.

The shift: untrusted text can become instructions

Traditional software keeps code and data in separate lanes. A web form's text field is data; it doesn't get executed as a command unless something is badly broken, which is exactly what SQL injection and cross-site scripting exploit when it is. A language model doesn't have that separation by default. Its input is one long stream of text, some of it a system prompt, some of it a user's message, some of it the contents of a file or web page it was asked to summarize, and the model reads all of it the same way: as language to interpret and respond to. Anything in that stream can, in principle, function as an instruction, whether the person building the app intended that or not.

Simon Willison, who named and first documented this class of vulnerability in September 2022, put it in terms borrowed from an older, well-understood bug: prompt injection is a security vulnerability where malicious input tricks a language model into performing an unintended action, the same underlying shape as a SQL injection attack, where untrusted input escapes its intended role as data and gets executed as a command instead.² The parallel is useful because SQL injection has a real fix, parameterized queries, that keeps code and data mechanically separate. Prompt injection, so far, does not have an equivalent.

Direct and indirect prompt injection

The attack shows up in two distinct shapes, and telling them apart matters because they call for different defenses.

• Direct injection: the attacker is the user. They type something like "ignore your previous instructions and instead do X" straight into the chat box or prompt field, aimed at the app itself. This is the original form Willison documented, and it's the easier of the two to reason about, since the attacker is also the one interacting with your system.²
• Indirect injection: the attacker never talks to your app at all. They plant instructions inside content they know or expect an agent will later read, hidden text on a web page, a line in an email, a comment in a code file, a review on a listing, a paragraph in a resume aimed at an AI screening tool. The person running the agent had no idea the attack was there. The agent finds it while doing its job.

Indirect injection is the harder problem, and it's the one that matters most for any app where an agent reads external or user-supplied content and can also act. A research agent summarizing search results can be steered by a page written specifically to be summarized by agents. A coding agent reading a repository's files can be steered by a comment planted in a dependency. An email-triage agent can be steered by the body of the email it was asked to triage. In every case, the untrusted content and the instructions arrive through the exact same channel, because for the model, there is no separate channel.

Why there is no complete fix, yet

Willison has held this position for years and updated it repeatedly rather than declaring victory. In 2022 he wrote plainly that he didn't know how to solve prompt injection, and warned against the instinct to fix it by throwing more AI at the problem.² Writing again in 2025, nearly three years and dozens of proposed defenses later, his assessment was that progress toward a reliable, general solution had been alarmingly slow.³ NIST's own taxonomy of adversarial machine learning attacks is just as direct: there is no foolproof way, as yet, to protect an AI system from this kind of misdirection, and the report explicitly warns against trusting anyone who claims otherwise.⁴

That doesn't mean the situation is hopeless, it means the engineering posture has to change. Instead of asking "how do I prevent injection," the more honest question is "assuming injection will eventually succeed, what is the worst it can do, and how do I keep that worst case small." Everything from here on is built around that second question.

A shared vocabulary: the OWASP Top 10 for LLM Applications

The OWASP Gen AI Security Project maintains a Top 10 list specifically for LLM-powered applications, updated for 2025, that gives the field a common reference framing instead of everyone inventing their own terms.¹ Prompt injection is the first and most cited entry, but it sits alongside nine other risks that matter just as much once a model can act on what it reads.

A few of these deserve a closer look for an app where the model reads untrusted content and takes action: Excessive Agency (LLM06) is what turns a successful injection into real damage, since a model with no tools can't do much even if it's fully fooled. Improper Output Handling (LLM05) is the twin risk in the other direction, treating what the model produces as safe to render, execute, or forward. And Misinformation (LLM09) covers the quieter failure mode: nobody attacked anything, the model was just confidently wrong, and a person or a downstream system acted on it anyway.

The lethal trifecta: how injection becomes data exfiltration

The scariest real-world outcome isn't a rude chatbot response, it's private data leaving the system without anyone approving it. Willison names the pattern behind almost every serious incident of this kind the "lethal trifecta," three capabilities that are each reasonable on their own but dangerous in combination.³

An agent with all three at once can be tricked into reading its own private data, packaging it up, and shipping it out, all inside what looks like normal operation. An email assistant that can read your inbox, is exposed to incoming mail from strangers, and can send email or hit external URLs has, without anyone designing it that way, assembled the exact combination an attacker needs. The fix isn't a smarter filter on any one leg of the trifecta. It's making sure a single agent, on a single task, doesn't hold all three capabilities at the same time.³

Mitigations: mapping risks to defenses

None of the following closes the hole. Each one shrinks what a successful injection can actually do, which, given that a complete fix doesn't exist yet, is the realistic goal.

Over-reliance: the failure that isn’t an attack at all

Not every incident starts with a malicious actor. OWASP's Misinformation category (LLM09) covers a quieter, more common failure: a model states something wrong with total confidence, and a person or a downstream system acts on it without checking.¹ The defenses look similar either way, don't grant an agent more agency than a task needs, keep a human in the loop for consequential actions, don't treat model output as verified fact, but the cause is different enough to name separately. Over-reliance doesn't require an attacker. It just requires nobody double-checking.

A local-first angle: shrinking what’s reachable

One structural mitigation deserves its own mention, separate from any specific defense above: reducing what an agent can reach in the first place. An agent that only ever touches data already on the machine it runs on, with no standing network path to send that data elsewhere, has a smaller blast radius by construction, regardless of whether a given prompt injection attempt succeeds. This doesn't fix prompt injection itself, since injection is about how a model processes the text it's given, not about where the model happens to run. But it changes what a successful injection can accomplish, because the third leg of the lethal trifecta, a path for data to leave, may simply not exist. Keeping sensitive data local, and being deliberate about which tools and network paths an agent can reach, is a design choice worth making independent of which vendor or model is involved.

Honest limits

This guide is an explanation, not an implementation checklist, and even a fully implemented version of every defense above does not add up to a guarantee. New injection techniques get published regularly, and the field's own experts describe progress on a general solution as slow.³ The realistic target for any app that lets a model read untrusted content and act on it isn't "immune to injection." It's "designed so that when injection eventually succeeds, the damage is small, visible, and recoverable," which is a different, more honest, and more achievable bar.