AI agent security, or agentic AI security, involves more than answer accuracy. Chatbots produced answers while people decided what to do, but agents read and write business systems, send messages, and call tools to perform parts of the work directly (p.28).
Mistakes therefore change subsequent actions as well as answers. Organizations must now decide what agents can execute and how much to delegate without human review.
The same principle connects all 14 contributions in the first issue of KCBCA Signals. The ability to produce analysis, read API responses, and write code must not be confused with authority to execute financial, administrative, or deployment decisions (p.5). This article brings permission separation, independent checks, and gradual delegation from the October series into a 10-point AI agent security checklist.
This article concludes themes from the October series, following the articles on prompt injection and supply chain attacks.
AI with execution authority needs oversight
Traditional risk management assumed that software followed predefined behaviour and authenticated people initiated important transactions. Agents with execution authority challenge both assumptions.
Connecting them to repositories and deployment pipelines without runtime boundaries amounts to installing a privileged insider operating at scale without an audit trail (p.16). This is why boards need to oversee them.
A successful call can still reflect poor judgment
Even when authentication, access, and API calls are all permitted and successful, an action may be inappropriate to its context. An agent may find and accurately quote the right policy yet reach the wrong conclusion by missing an exception (p.28).
AI agent security therefore asks both about access permissions and how much to delegate without human review. Product recommendations and refunds carry different acceptable risks, while financial approvals and permission changes require much lower residual risk (p.28).
Separate analysis, approval, and publishing permissions
“Data is not instruction. Analysis is not approval. A record is not a fact.” (p.30) Comments and attachments are material to analyse, rather than instructions.
Indirect prompt injection occurs when AI follows commands hidden inside them (p.30). Enforce boundaries outside the model: analysis workers should lack permission to publish their own output, and public material should pass an explicit approval process. A polished model response is no approval record (p.31).
Use limits and independent checks for actions involving money
First define the transactions an agent may approve. Narrow the scope, for example to transfers between designated corporate wallets, and impose per-transaction and cumulative limits for each period. Limit increases require separate, higher-level approval (p.32-33).
Immediately before signing, a check beyond the agent's control compares recipient, asset, amount, and contract operation with the approved scope (p.33). Even with a secure private key, a signing service that trusts only the agent's approval can sign an unintended transaction (p.32). Human reviewers should consult information independent of the agent's explanation and approve only that specific transaction (p.33).
Four stages of gradual delegation and four operating metrics
In AI agent security, delegation is expanded according to operating evidence rather than fixed all at once (p.29).
| Stage | Activity |
|---|---|
| 1. Observe | Operate alongside existing work; humans remain responsible for final actions |
| 2. Compare | Compare judgments with actual outcomes and collect failures |
| 3. Narrow execution | Only low-impact actions for a limited group of users |
| 4. Expand | After sufficient operating evidence has accumulated |
Decisions should rest on operating metrics rather than overall model accuracy (p.28).
| Operating metric | What to check |
|---|---|
| Automated processing rate | How much is processed automatically |
| Human intervention frequency | How often people intervene |
| Serious failures among allowed cases | How many serious failures remain among permitted cases |
| Unnecessary blocking | How often legitimate cases are stopped unnecessarily |
The organization responsible for the work determines acceptable risk, rather than the supplier (p.29).
Pre-launch testing: refusal, revocation, and interruption paths
Test refusal paths such as revoked access, substituted files, and interrupted tasks, alongside successful demonstrations (p.31). Before entrusting real funds, use deliberately deceptive inputs to check whether prohibited actions and limit breaches are blocked, and practise suspending authority and revoking permissions that remain after deactivation (p.33).
Chain reactions can occur even when rules are followed
One agent sees an anomaly and changes a limit; another system reads that as a new risk signal, prompting a third agent to respond. Even if each follows its rules, the organization may reach an outcome nobody intended (p.29).
Processing by another system does not establish trust. Practise stopping and tracing action chains, revoking access, and isolating work (p.31).
A 10-point AI agent security checklist
| # | What to check | Basis |
|---|---|---|
| 1 | Have reading, writing, and execution scopes been defined for each task? | p.28 |
| 2 | Do analysis workers lack approval and publishing permissions? | p.30-31 |
| 3 | Are instructions in documents, comments, and attachments blocked outside the model? | p.30, p.32 |
| 4 | Are credentials verified per service, with least privilege? | p.31 |
| 5 | Are per-transaction and cumulative limits and limit-increase approval procedures in place? | p.32-33 |
| 6 | Are transactions checked outside the agent before signing? | p.33 |
| 7 | Does the reviewer use independent information and approve only that transaction? | p.33 |
| 8 | Are the four delegation stages and four metrics followed? | p.28-29 |
| 9 | Can permission, appropriateness, and reasons be reconstructed? | p.29 |
| 10 | Have revocation, substitution, interruption, and deceptive inputs been tested? | p.31, p.33 |
Frequently asked questions
Q1. How does AI agent security differ from conventional AI security?
Conventional AI produced answers and people acted. Agents act directly, so the scope of execution authority and its verification must be examined together (p.16, p.28).
Q2. Is an instruction to ignore suspicious commands enough?
It helps, but cannot be the only defence. Place controls outside the model that still work if the model is persuaded (p.30-31, p.33).
Q3. Does final human approval make it safe?
A reviewer who sees the same manipulated material as the agent may approve the attacker's intended outcome. Use separately verified information and approve only that transaction (p.33).
At a glance
- Capability is not authority (p.5)
- Separate analysis, approval, publishing, and signing permissions
- Apply transaction limits and independent checks before signing
- Expand delegation gradually in line with operating evidence
AI agent security aims to keep the ability to read, reason, and act from quietly turning into authority, while allowing useful automation (p.31). Complete the 10-point checklist before adoption.
Read the full KCBCA Signals Vol.1 Issue 1
References: KCBCA Signals Vol.1 Issue 1; JJ Park (Head of Publishing, KCBCA GLOBAL) p.5; Jack Wong (CEO, Scryer; Director, KCBCA GLOBAL) p.16; Minseung Kang (Founder and CEO, Tnaps) p.28-29; Ted Kim (CEO, Shorti) p.30-31; Marc Krisjanous (Founder and Principal Consultant, Zanarc) p.32-33
