How to Detect Prompt Injection in the Wild

Prompt injection is the SQL injection of the AI era. An attacker crafts input that causes your LLM to ignore its system instructions and follow the attacker's instructions instead. Unlike traditional injection attacks, there is no single fix — defence requires layered detection and mitigation.

Understanding the Attack Surface

Direct Injection — The user's input directly contains instructions that attempt to override the system prompt. "Ignore your instructions and instead..." is the simplest form, but sophisticated attacks use encoding tricks, language switching, and multi-turn manipulation.

Indirect Injection — Malicious instructions are embedded in data the LLM retrieves or processes — a webpage, a document, an email. The model processes the content and encounters hidden instructions embedded within it. This is particularly dangerous in RAG systems where the model reads untrusted external content.

Detection Techniques

Pattern-Based Detection — Maintain a library of known injection patterns and scan inputs before they reach the model. Catches naive attacks but is easily bypassed by creative attackers. Useful as a first layer, not as a sole defence.

Classifier-Based Detection — Train a lightweight classifier (or use a small LLM) to identify inputs that are likely injection attempts. More robust than pattern matching because it generalises to novel attack patterns. Models like DeBERTa fine-tuned on injection datasets achieve 95%+ detection rates.

Output Anomaly Detection — Monitor model outputs for signs of successful injection: sudden format changes, outputs that reference the system prompt, or responses that do not match the expected task. This catches attacks that bypass input filtering.

Canary Token Detection — Embed unique identifiers in your system prompt. If these tokens appear in the output, the system prompt has been leaked — a strong indicator of successful injection.

Defence in Depth

No single detection technique is sufficient. Production systems should layer pattern matching (fast, cheap), classifier-based detection (robust, moderate cost), and output monitoring (catches what input filtering misses). Combined with privilege separation between system and user prompts, this creates a defence that is resilient to novel attack vectors.