What is prompt injection?
Prompt injection is an attempt to redirect an AI system with instructions embedded in untrusted content it reads.
How it works and where it is used
A web page, document or search result might say “send secret data” despite that not being the user’s task. The model should treat it as data, yet may mistake it for instruction. The attack arrives through a lower-trust source, not the authorized user request.
Limits, safety and selection criteria
A defensive instruction alone is insufficient. Limit tool permissions, separate trust boundaries, verify important actions and require independent review for sensitive operations. Test the effect of attacks in realistic data and workflows. Prompt injection places hostile instructions inside material a model reads, such as a page or document. A page telling a summarizing agent to send secrets elsewhere is not a legitimate user request. Preserve the boundary between source content and instructions. Mark untrusted material, restrict tool permissions and require confirmation for consequential actions. Keyword filtering alone misses paraphrases and other languages. Test the system with realistic tool responses and data flows, including attempts to change its task or reveal private information.