It Produced Harmful Output 80% of the Time Without Anyone Even Attacking It

An AI system broke Meta’s model in 85% of attempts. No code. No special attack tooling. Within 3 hours.

Wait – that is not the mad part yet.

The mad part is that even with no attempt to break in at all – no jailbreak, no sophisticated prompt injection, just asking politely – the model produced harmful content in 80% of cases. That is 80% success without even trying.

Let’s stop there for a second.

Everybody is shipping, nobody is testing

We are living through a period in which every second company is racing to push an AI agent into the product. Every startup adds “AI-powered” to the headline on the site. Every CTO tells the board they are already integrating artificial intelligence.

But who checked the attack surface? Who did red teaming on the model before it reached production? Who even knows what that model is capable of saying when somebody pushes it the right way?

The answer: almost nobody.

I have been working in application security for more than 15 years and I have seen the same story over and over. A new technology arrives → everybody rushes to adopt → nobody tests → somebody gets breached → suddenly everybody is an expert. It happened with web applications. It happened with mobile. It happened with cloud. Now it is happening with AI agents.

The one difference this time

The attack surface is natural language. You do not need to know how to write an exploit. You do not need Burp Suite. You do not need years of experience. You need to know how to phrase a sentence.

Which means anyone can attack. The intern, the customer, the competitor, the fourteen-year-old. Anyone who can type.

And here is what genuinely frightens me: companies are putting AI agents in place with access to sensitive information, to financial operations, to customer data. With no serious defensive layer. With no resilience testing. With no basic understanding of what can go wrong.

It is not a question of if. It is a question of when.

So what do you do?

Before integrating an AI agent into production, red team it the way you would pentest an application. Test prompt injection. Test data leakage. Test what happens when somebody tries to push the model past its boundaries. Do not rely on the vendor’s guardrails.

I first shared a version of this as a LinkedIn post on 2026-05-27. It is republished here, lightly edited, so it is easier to find and reference. — Erez Metula