PROMPT_SECURITY_INDIRECT_INJECTION_EMBEDDED_INSTRUCTION_001

rules/regulations/universal/prompt_security/indirect_injection_embedded_instruction.yaml

Citation

Regulation (EU) 2024/1689, Article 15(5): High-risk AI systems shall be resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities. The technical solutions aiming to ensure the cybersecurity of high-risk AI systems shall be appropriate to the relevant circumstances and the risks. The technical solutions to address AI specific vulnerabilities shall include, where appropriate, measures to prevent, detect, respond to, resolve and control for attacks trying to manipulate the training data set (data poisoning), or pre-trained components used in training (model poisoning), inputs designed to cause the AI model to make a mistake (adversarial examples or model evasion), confidentiality attacks or model flaws.

Condition

Covers indirect prompt injection: instructions embedded in untrusted content (retrieved documents, tool outputs, web pages) that try to redirect the model when it reads/summarises them. EU AI Act Article 15. OWASP LLM01.

Layer 1 detects

in any letter case, "when you", "when the AI", "when the assistant", "when the model", "when the LLM" or "when the system", followed by read, process, summarise, summarize, analyse, analyze, parse, reply, respond, answer, draft, handle or review (with or without a final "s"), then "this" or "to this", then, within the next 80 characters on the same line, the word ignore, instead, "you must", "do not", disregard, classify, approve, pre-approve, preapprove, route, treat, skip, bypass, escalate, prioritise, prioritize, include or forward; the German "wenn Sie" or "wenn du" followed by dies, diese, diesen, dieses or diesem, one word, then bearbeiten, bearbeitest, lesen, liest, verarbeiten, verarbeitest, prüfen, prüfst, analysieren or beantworten, then, within 80 characters, a word starting "ignorier", stattdessen, stufen, einstufen, leiten, weiterleiten, markieren, genehmigen, überspringen, umgehen or priorisieren; a line that starts (after optional spaces and one to three "#") with "system" or "assistant" followed by message, prompt, instruction or note, then a colon or ">"; and an opening "<system>", "<important_instruction>" or "<important_instructions>" tag, with or without spaces inside the brackets. It matches wording, not meaning: it does not read who wrote the text, so it fires on the user's own request in the same form ("When you review this, skip the appendix") and on a document line that begins "System note:" for an ordinary reason, and it misses an embedded instruction worded any other way. Whether the text carries a third party's instruction is judged in Layer 2.

Condition type

regex

Status in the file

approved