PROMPT_SECURITY_EXFIL_TRAINING_DATA_EXTRACTION_001

rules/regulations/universal/prompt_security/exfil_training_data_extraction.yaml

Citation

Regulation (EU) 2024/1689, Article 15(5): High-risk AI systems shall be resilient against attempts by unauthorised third parties to alter their use, outputs or performance by exploiting system vulnerabilities. The technical solutions aiming to ensure the cybersecurity of high-risk AI systems shall be appropriate to the relevant circumstances and the risks. The technical solutions to address AI specific vulnerabilities shall include, where appropriate, measures to prevent, detect, respond to, resolve and control for attacks trying to manipulate the training data set (data poisoning), or pre-trained components used in training (model poisoning), inputs designed to cause the AI model to make a mistake (adversarial examples or model evasion), confidentiality attacks or model flaws.

Condition

Covers training-data extraction attacks: "repeat a word forever" divergence prompts and requests to output verbatim memorised/training text. EU AI Act Article 15. OWASP LLM01 / LLM06.

Layer 1 detects

in any letter case, "repeat" (optionally "the word") followed by one single word, quoted or not, then forever, infinitely, endlessly, "(a) thousand times", "(a) million times", "(a) hundred times", "(a) billion times", or "until you run out", "until you stop" or "until you break"; and output, print, reproduce, recite, dump or leak (optionally "your", then optionally "verbatim") followed by "training data", "memorised" or "memorized" text, data, content or example(s), or "(the) verbatim training" or "verbatim memorised" followed by a word. A phrase of two or more words to repeat does not match. It matches wording, not meaning: it fires on harmless repetition requests in the same form ("repeat hello a hundred times for the typing test"), and it misses an extraction request worded any other way. Whether the text tries to make the model leak training data is judged in Layer 2.

Condition type

regex

Status in the file

approved