Retrieval testing examines what reaches the model’s context and who decided it should. In a retrieval-augmented system the context is assembled at request time from stores the user does not see, so two questions matter: whether the assembled context respects the requesting user’s entitlements, and whether anyone can influence what gets assembled.

What an attacker is trying to achieve

The attacker wants their content in someone else’s context, or someone else’s content in theirs. Poisoning the store is the durable version: one document planted once, acted on every time the system retrieves it, for every user whose query matches.

Test cases

6 test cases in this category. Each states what the test proves, how it is carried out, what has to be in place beforehand, what a positive result looks like, and whether the work is manual, automated or a mixture.

AI-RAG-01 Retrieval Authorization Boundary

Objective — Determine whether the documents assembled into a user’s context are restricted to those that user is entitled to read.

How it is tested — With two accounts in different ownership scopes, ask questions whose answers exist only in the other account’s documents. Vary the phrasing so the retrieval is driven by content rather than by an identifier. Establish whether filtering happens before retrieval, after retrieval but before the model, or not at all — a system that retrieves widely and asks the model to be discreet has no access control.

Prerequisites — Two accounts with distinct, recognisable documents; the ability to ask arbitrary questions; ideally the retrieval trace for a given answer.

Indicators — An answer containing another scope’s content; a citation to a document the user cannot open; the model declining but naming the source; filtering applied only by instruction to the model.

Coverage — LLM02LLM08CWE-285CWE-668

Depth — Manual Requires two populated scopes and questions written against the difference between them.

AI-RAG-02 Context Poisoning Through Indexed Sources

Objective — Determine whether content a user can place into an indexed source will be retrieved into other users’ contexts and acted upon.

How it is tested — Identify every source that is indexed and who may write to each — shared drives, wikis, ticket systems, public pages, customer-submitted content. Place a document that is both plausibly relevant to a common query and carries an instruction. Then, as a different user, ask the question it targets and observe whether the planted document is retrieved and whether its instruction is followed.

Prerequisites — Write access to an indexed source; a second account to run the query; a rebuildable index; the operator’s agreement that content may be planted.

Indicators — A planted document retrieved for another user; its instructions followed; content ranked highly because it was written to match the retrieval rather than the question.

Coverage — LLM04LLM08CWE-1427CWE-501

Depth — Manual The attack spans two users and an indexing cycle; nothing about it fits a request-response test.

AI-RAG-03 Embedding and Similarity Manipulation

Objective — Determine whether retrieval ranking can be influenced so that chosen content is returned for queries it should not match.

How it is tested — Craft documents designed to sit close to a target query in embedding space while reading as something else to a human — keyword stuffing, paraphrase padding, repeated question forms. Measure whether they displace the legitimate result for that query. Where the store supports metadata filtering, test whether metadata supplied at ingestion is trusted and whether it can be set to widen a document’s reach.

Prerequisites — The ability to add documents and observe what is retrieved; a set of target queries with known correct answers; a rebuildable index.

Indicators — A crafted document outranking the correct one; retrieval driven by padding rather than meaning; ingestion metadata trusted without validation.

Coverage — LLM08LLM09CWE-807

Depth — Manual Requires observing ranking, not just output, and a corpus with known right answers.

AI-RAG-04 Cross-Session and Cross-User Context Leakage

Objective — Determine whether context from one conversation or one user can appear in another’s.

How it is tested — Establish distinctive content in one session, then ask questions in another session and another account designed to surface it. Test concurrent sessions, sessions resumed after a gap, and the same user on two devices. Where the application caches responses or embeddings, establish whether the cache key includes identity, since a cache keyed on the question alone will serve one user’s answer to another.

Prerequisites — Two accounts and the ability to hold concurrent sessions; recognisable content planted in each; a non-production environment.

Indicators — Content from another session in an answer; an identical question returning another user’s cached answer; conversation history surviving sign-out.

Coverage — LLM02CWE-668CWE-524

Depth — Manual The finding only appears when two identities are exercised against the same system deliberately.

AI-RAG-05 Sensitive Data in Retrieved Context

Objective — Determine what the retrieval layer is willing to place in front of the model, irrespective of what the model then chooses to say.

How it is tested — Where the retrieval trace is available, inspect what is assembled for ordinary questions and classify it. Where it is not, infer it: ask questions whose answers require the model to have seen material it should not have. Establish whether documents are chunked in a way that separates content from its access-control metadata, which is how restricted material commonly reaches a context it should not.

Prerequisites — The operator’s classification of what is sensitive; retrieval traces where available; accounts at different entitlement levels.

Indicators — Restricted material in the assembled context; chunks that have lost their provenance; secrets or credentials indexed as ordinary documents; the model refusing to repeat something it demonstrably received.

Coverage — LLM02LLM08CWE-200CWE-1230

Depth — Manual The test is about what the model was given, not what it said – which requires either the trace or a carefully designed inference.

AI-RAG-06 Ingestion Pipeline Trust

Objective — Determine whether the pipeline that turns documents into retrievable content validates what it ingests, or trusts anything reachable.

How it is tested — Establish what the ingestion pipeline will fetch and parse: file types, remote URLs, archives, embedded objects. Submit documents that cause the parser to fetch a further resource, and observe whether a request arrives at a tester-controlled listener — the ingestion worker frequently runs with more network reach than the application. Test oversized and malformed documents for resource exhaustion.

Prerequisites — The ability to submit documents for ingestion; a listener the target can reach; a non-production environment and permission, since exhaustion tests degrade the pipeline.

Indicators — An out-of-band request from the ingestion worker; a remote reference resolved during parsing; unbounded resource use on a crafted document; file types accepted that the documentation does not mention.

Coverage — LLM03LLM10CWE-918CWE-400

Depth — Hybrid Out-of-band confirmation automates once a listener is standing; deciding what the pipeline will accept is manual.

What we need from you

Two accounts in different ownership scopes with genuinely different data; write access to at least one indexed source; visibility of what was retrieved for a given answer, or the operator’s help in obtaining it; and a non-production index that can be rebuilt after poisoning.

Standards this category maps to

Every identifier below is quoted from the published control list it belongs to. Where a test has no defensible mapping, none is given rather than an approximate one.

  • LLM02 — Sensitive Information Disclosure (OWASP Top 10 for LLM Applications 2025)
  • LLM08 — Vector and Embedding Weaknesses (OWASP Top 10 for LLM Applications 2025)
  • CWE-285 — Improper Authorization
  • CWE-668 — Exposure of Resource to Wrong Sphere
  • LLM04 — Data and Model Poisoning (OWASP Top 10 for LLM Applications 2025)
  • CWE-1427 — Improper Neutralization of Input Used for LLM Prompting
  • CWE-501 — Trust Boundary Violation
  • LLM09 — Misinformation (OWASP Top 10 for LLM Applications 2025)
  • CWE-807 — Reliance on Untrusted Inputs in a Security Decision
  • CWE-524 — Use of Cache Containing Sensitive Information
  • CWE-200 — Exposure of Sensitive Information to an Unauthorized Actor
  • CWE-1230 — Exposure of Sensitive Information Through Metadata
  • LLM03 — Supply Chain (OWASP Top 10 for LLM Applications 2025)
  • LLM10 — Unbounded Consumption (OWASP Top 10 for LLM Applications 2025)
  • CWE-918 — Server-Side Request Forgery (SSRF)
  • CWE-400 — Uncontrolled Resource Consumption