Prompt injection is an attack that attempts to change an AI system's behaviour through instructions planted in external material (p.19).
When it arrives through content the AI reads, such as webpages, documents, or comments, OWASP classifies it as “indirect prompt injection” (p.30).
If AI can call connected services rather than merely summarize material, a misinterpreted instruction can lead to a real action (p.30).
Simply telling it to ignore suspicious instructions is therefore insufficient.
Source material must be distinguished from user instructions, and permissions to read, analyze, approve, publish, and sign must be separated according to need.
Important actions must be checked outside the AI. Human approvers must be able to see the original content independently of the AI's summary (p.19, p.31, p.33).
This article is based on three contributions in KCBCA Signals Vol.1 Issue 1 (p.19, p.30–33).
Data is not an instruction
“Data is not an instruction. Analysis is not approval. A record is not the truth.” — KCBCA Signals Vol.1 p.30
These principles emerged from the development of a content analysis service.
Video subtitles, comments, and imported documents are material to analyze, rather than instructions from the user.
AI can nevertheless mistake sentences within them for instructions it should follow.
OWASP also includes risks transmitted through media such as images within indirect prompt injection (p.30).
This development team treated it as an architectural issue rather than a wording issue (p.30).
Case 1. A content analysis service: between analysis and publication
If even one boundary fails, an unreleased campaign could leak, an approver could be misidentified, or material could be published beyond the permitted scope.
OWASP calls excessive permissions and autonomy “excessive agency” (p.30).
The analyst therefore reads material and writes findings but is not permitted to publish its own output.
The publisher releases only results approved by a reviewer.
A fluent AI response is not a record of approval (p.31).
Separation of permissions (adapted from Figure 5, p.30)
| Action | Analyst | Authorized reviewer | Publisher |
|---|---|---|---|
| Read and analyze material | Allowed | Review | — |
| Approve results | Not allowed | Approve | — |
| Release approved results | Not allowed | — | Publish |
Case 2. A wallet-connected AI assistant: approving from a summary alone
Consider a hypothetical scenario.
An AI assistant reads documents, explains a community activity, and prepares a transaction (p.19).
If it accepts instructions on a malicious page as trustworthy, it may prepare a transaction the user did not intend.
A user who approves solely on the basis of the AI's summary may miss the underlying problem (p.19).
Explaining an activity does not require control of a wallet.
Authority delegated for one task must not quietly expand into broader authority.
Users must be able to review the proposed action and delegated permissions, and revoke them where the account architecture allows (p.19).
Case 3. A transfer approval agent: signatures can occur even with protected keys
Consider another hypothetical scenario.
An AI agent reviewing a virtual asset transfer request reads an attachment explaining the transfer's purpose.
If an attacker plants instructions in that document, the agent may interpret them as directions to bypass approval conditions.
This is an example of indirect prompt injection (p.32).
Whether it causes a loss depends on the boundary between approval and signing.
If the signing service treats the agent's approval as sufficient, it may sign a transaction the company did not intend even while the private key remains secure.
Checking key protection alone misses this weakness (p.32).
The response is to place a separate check immediately before signing that the agent cannot modify.
Comparing the recipient, asset, amount, and contract operation with approved authority can reject a request even if the model has been persuaded (p.33).
The common response across the three cases
To address prompt injection, software must limit what AI can execute even when it misreads source material.
| Measure | What it does | Source |
|---|---|---|
| Separate permissions | Assign analysis, approval, publication, and signing to different roles | p.31 |
| Independent check before signing | Compare the recipient, asset, amount, and operation outside the model | p.33 |
| Minimal, short-lived access | Grant only necessary access, for a short period | p.19 |
| Human review using independent information | Verify independently of the AI's explanation and approve only that transaction | p.33 |
| Test refusal and revocation paths | Test revoked access, substituted files, interrupted tasks, and deceptive inputs | p.31, p.33 |
The UK's National Cyber Security Centre (NCSC) also recommends limits on actions outside the model, minimal and short-lived access, and meaningful oversight (p.19).
Beyond successful demonstrations, test whether the system also blocks revoked access, substituted files, interrupted tasks, and deliberately deceptive inputs (p.31, p.33).
Frequently asked questions
Q1. Is telling the model to ignore suspicious instructions enough?
It helps, but cannot be the only line of defence protecting personal information or publishing permissions (p.30).
Limits on actions must be enforced by the software around the AI, rather than by instructions alone (p.19).
Q2. How does prompt injection differ from phishing?
Phishing persuades people, while prompt injection targets AI systems.
Both can influence decisions that lead to action (p.19).
Q3. Is it safe if a human approves at the end?
Reviewing only the AI's summary can miss the problem (p.19).
Reviewers must see independently verified transaction details and approve only the specific transaction submitted (p.33).
Key points
- External material read by AI is not a user instruction
- Separate permissions for analysis, approval, publication, and signing
- Check transaction details outside the model before signing
- Test whether refusal and permission revocation actually work
The aim of responding to prompt injection is not to slow useful automation.
It is to prevent the ability to read, reason, and act from quietly becoming permission to do so (p.31).
The same material is also presented in a 40-second video on Instagram @kcbca_korea.
Read KCBCA Signals Vol.1 Issue 1 (p.30)
References: KCBCA Signals Vol.1 Issue 1. Jun Ahn (MemeCore CEO), p.19; Ted Kim (Shorti CEO), p.30-31; Marc Krisjanous (Zanarc founder and principal consultant), p.32-33
