Prompt injection is the vulnerability class that defines applications built on language models, and most of the systems we test have never been checked for it. This page is what we have learned finding it in production systems, kept up to date as the pattern changes.

It is not an introduction to what an LLM is. It assumes you are shipping one.

The one-sentence version

A language model cannot reliably distinguish the instructions you gave it from the data it is asked to process. Everything else follows from that. If any content the model reads can be influenced by somebody else, that person can issue instructions to your application.

This is not a bug in a particular model, and it is not fixed by a better system prompt. It is a property of putting instructions and untrusted data in the same channel — which is the same shape as SQL injection, where the query and the data shared a channel until parameterised statements separated them. We do not yet have the equivalent separation for prompts.

Direct and indirect, and why only one of them matters much

Direct injection is a user typing “ignore your instructions” into your chat box. It is real, it is what most people test, and it is the less interesting half — the blast radius is usually the attacker’s own session.

Indirect injection is where the damage is. The model reads something on the attacker’s behalf — a web page, a document, an email, a support ticket, a code comment, a package README — and that content carries instructions. The victim did nothing. They asked a normal question, and your agent followed somebody else’s orders while answering it.

In our engagements, indirect is the one that produces critical findings, because the content sources are almost never treated as untrusted input.

What we actually find, in order of how often we find it

1. The model can reach an API, and nobody bounded which one

The most common serious finding. A support assistant is wired to internal APIs so it can answer account questions. It composes calls from natural language. Ask it politely for a bit more and it composes a different call — one nobody intended it to be able to make.

The failure is not the model’s. It is that the model was given a capability instead of a menu. A model that can construct arbitrary requests will eventually construct a request you did not want.

Worked example, from a real engagement: The chatbot that called our admin API.

2. Authorization is enforced on the user, not on the agent

The application checks that you may see a document. The AI layer then reads documents on your behalf using a service identity that may see everything. The check passed; the wrong data came back anyway.

This is an old bug in new clothing — it is IDOR with an extra hop — which is why it slips past teams who have already hardened their direct endpoints.

Worked example: When the AI builds your API call.

3. The model can fetch a URL

Any feature where a model retrieves a link is a server-side request forgery primitive, because the fetch happens from your infrastructure with your network position. Add a model that can be talked into choosing the URL and you have SSRF with a natural-language front end.

Worked example: the SSRF we found in an AI document-analysis feature.

4. The context window is a disclosure surface

System prompts leak. So does anything else placed in context — retrieved documents from other tenants, prior conversation, tool definitions that name internal endpoints. Teams routinely put things in the prompt that they would never put in an API response.

5. The tool inventory nobody has

Agents acquire tools, plugins and MCP servers the way browsers acquired extensions. Most organisations we test cannot produce a list of what their agents can currently do, which means the attack surface is not merely unprotected — it is unknown.

What does not work

  • Telling the model to ignore injected instructions. You are asking the thing that cannot reliably tell instructions from data to tell instructions from data.
  • Filtering for “ignore previous instructions”. Injection is semantic. There are unbounded ways to express an instruction, including in other languages, encodings, and inside code.
  • Relying on the vendor’s guardrails. They are a floor, they are tuned for generic harm rather than your business logic, and they know nothing about which of your objects belong to whom.
  • Treating it as a content-moderation problem. The risk is not that the model says something rude. It is that it acts.

What does work

The controls that hold are the boring architectural ones, because they do not depend on the model behaving:

  • Give the model a menu, not a capability. Enumerated operations with typed parameters, validated server-side. If it cannot express the dangerous call, it cannot be talked into making it.
  • Run every action with the end user’s authority, never a service identity. If the user cannot read that document directly, the agent must not read it for them.
  • Validate on the server regardless of origin. A parameter that came from a model is untrusted input in exactly the way a parameter from a browser is.
  • Mark retrieved content as data, and keep the boundary explicit in how it is assembled into context.
  • Require human confirmation for anything outbound or irreversible.
  • Run the agent with least privilege, in isolation, with egress control. Assume injection succeeds and ask what it reaches.
  • Keep an inventory of tools, plugins and MCP servers, and review it.

How we test for it

Prompt injection testing is not a scanner run. What we do, in order:

  1. Map the content sources. Everything the model can read, and who can influence each one. This is usually the moment the team discovers a source they had forgotten.
  2. Map the capabilities. Every tool, function and endpoint the agent can reach, and the identity it uses to reach them.
  3. Cross the two. For each content source an attacker can influence, what is the most damaging capability reachable from it? That product is the actual risk, and it is usually much larger than the team expects.
  4. Test indirect first. Plant instructions in the sources rather than typing at the chat box.
  5. Test the authorization boundary specifically — the same object-reference and role work as any other application, applied to what the agent retrieves.

Step 3 is where the findings come from, and it is the step tools do not do, because it requires knowing what the application is for.

Related reading from our engagements

Maintained by AppSec Labs. We test AI and LLM features as part of application penetration testing — see AI & LLM penetration testing.