Applications that embed a language model have a new problem: part of the system now takes instructions from text, and some of that text comes from users. Traditional application security assumes code decides and data is inert. That assumption no longer holds.

The failures we see are rarely exotic. They are ordinary application security failures — excessive privilege, missing authorization, unvalidated outbound requests — with a model in the middle that can be talked into triggering them.

Who this is for

  • You have shipped an AI assistant, chatbot or copilot that can read customer data or take actions in your system.
  • Your model has tools, functions or agents wired to real operations — issuing refunds, changing records, calling internal services.
  • You use retrieval over internal documents and need to know whether one user can reach another’s content through the model.
  • Your customers or auditors have started asking how you secured the AI feature, and you need an answer better than “we use a reputable provider”.

What we actually test

Prompt injection, direct and indirect

Direct: can a user instruct the model to ignore its constraints, reveal its system prompt, or act outside its intended role? Indirect and more dangerous: can instructions be planted in content the model will later read — a document, a support ticket, a web page, an email, a filename — so the model executes them on behalf of a different user?

Excessive agency

The question that decides real impact: what can the model actually do, and with whose authority? Models given broad service credentials, tools without their own authorization checks, functions that trust model-supplied parameters, and agent loops that can be steered into destructive or expensive actions. We map every tool the model can reach and test each as an attack surface in its own right.

Authorization through the model

Retrieval and tool calls that run with the application’s privileges rather than the user’s — so asking the right question returns another customer’s data. This is broken object-level authorization wearing a new interface, and it is the finding we most often report.

Server-side request forgery and data exfiltration

Models induced into calling attacker-controlled URLs, or into embedding sensitive data in a link, image or tool call that leaves your environment (a real chain we have written up). Secrets, session tokens and internal endpoints placed in prompts, where they can be extracted.

Output handling

Model output rendered as HTML or Markdown without sanitisation, producing cross-site scripting; output passed to a shell, a database query or an eval; output trusted as a control decision by downstream code.

Abuse, cost and availability

Unbounded token consumption, prompt-driven cost amplification, and rate limits enforced at the interface but not at the model or tool layer.

How we approach it

We test the AI feature as part of the application, not as a curiosity. The model is one component; the security boundary is the whole system around it — what the model can reach, with whose identity, and what happens to its output. We use the same role matrix discipline as any other engagement, and we chain findings to demonstrate genuine impact rather than reporting an interesting prompt.

The methodology is developed from real engagements, and our founder speaks on this work — including at OWASP AppSec Israel on penetration testing AI-based applications.

Common questions

Is this the same as red-teaming a model?

No. Model red-teaming asks whether the model produces harmful or biased content – a model-provider concern. We test your application: what the model can reach, whose authority it acts with, and what an attacker can make it do to your data and your systems.

We use OpenAI or Anthropic – isn’t security their responsibility?

They secure the model. You remain responsible for what you connect it to. Nearly every serious finding we report is in the integration: the tools, the credentials, the retrieval scope and the output handling – none of which the provider controls.

Can prompt injection actually be fixed?

Not eliminated – treat it as a permanent property of language models. It is contained by architecture: never give the model authority it should not have, enforce authorization outside the model on every tool call, and never let model output reach a dangerous sink unvalidated. That is what we assess.

Do you test AI features inside an existing application, or standalone?

Both, but testing it as part of the surrounding application is far more valuable – impact almost always comes from what the model is connected to.

Do you follow the OWASP Top 10 for LLM Applications?

We use it as a checklist rather than a methodology. It is a good taxonomy; it does not tell you what your particular agent can be talked into doing, which is the actual question.

Talk to us about testing your AI-enabled application

Expert-led testing of complex authenticated applications – multi-tenant isolation, authorization/BOLA, business-logic and workflow abuse, complex role models – with developer-usable remediation.

Tell us what the application does and what worries you about it. We will come back with scoping questions, a clear proposal and a realistic timeline — and if a test is not what you need yet, we will say so. Get in touch.

See also: API penetration testing · web application testing · prompt injection to account takeover.