Define scope
List tools, data, permissions, allowed and forbidden actions, and the human approval point. Tie each right to a task.
GUIDE / PROTOCOL
Evaluate an AI agent before release
Agent evaluation must test actions, boundaries and failures, not only fluent answers.
Direct answer
How can you tell whether an AI agent is ready for real use?
Define authorised tasks, expected outcomes and harms to avoid. Build normal, ambiguous and hostile cases, run them without external effects, then inspect every action and stop.
01 / Review method
Review method
Build the test set
Include routine requests, incomplete data, deceptive source instructions, tool errors, duplicates and attempts to leave the mandate.
Inspect traces
Compare outcome, consulted sources, tool calls, refusals, time, cost and human intervention with predefined criteria.
Make the release decision
Block critical failures, narrow rights where needed and rerun the same set after changes. Prepare shutdown, manual fallback and withdrawal.
02 / Evidence to keep
Evidence to keep
Case record
Input, expected result, risk, severity and pass criterion.
Execution log
System version, tool calls, exposed data, outputs and human decision.
Issue register
Failure, consequence, likely cause, fix and retest result.
Release decision
Owner, approved scope, stop threshold, fallback and review date.
03 / FAQ
Frequently asked questions
Is an agent safe when its answer is correct?
Not necessarily. It may reach the right answer through an unsuitable source or an unauthorised action.
Should prompt injection be tested?
Yes, especially when the agent reads pages, documents or messages that may contain hostile instructions.
When must a person decide?
Before sensitive, irreversible or explicitly unauthorised action.
What follows a critical failure?
Suspend that path, keep the trace, narrow the capability and rerun cases before resuming.
04 / Official references