Purpose
Ensure every tool, function call, MCP server, and external action invoked by a model is authorized, scoped, observable, and reversible or gated when impact is high.
Engineering philosophy
The model proposes; the platform disposes. Tools are privileged APIs. Never let natural language become an unrestricted remote control over production systems.
Why it matters
Tool misuse converts language model errors into real-world damage—deleted data, fraudulent transactions, leaked secrets, or lateral movement through MCP and A2A bridges.
Common failures
- Tools run with service-wide credentials instead of user-scoped tokens
- No allowlist of tools per agent or environment
- Destructive tools without confirmation or dry-run modes
- MCP servers exposed without authentication
Mandatory checks
Gate controls. Each check is pass/fail via artifact + pass condition. Expected from the annotated capability level when the system meets the minimum criticality tier.
Every tool invocation shall be authorized server-side independent of model output
- Artifact
- Tool gateway authz tests + deny logs
- Pass condition
- Automated tests show tool calls without a valid authz decision are denied at 100%; model-proposed tool name/args alone never bypass the gateway
Tools shall be allowlisted per agent/workload; unknown tools shall not be inventable at runtime
- Artifact
- Per-agent tool allowlist config + negative tests for unknown tool names
- Pass condition
- 100% of production agents have an explicit tool allowlist; requests for tools outside the allowlist are denied at 100% in automated tests
High-impact tools shall require additional gates (approval, dual control, or policy engine)
- Artifact
- Impact-tiered tool inventory + gate configuration + bypass tests
- Pass condition
- 100% of tools rated write/irreversible/financial have a configured gate; automated tests show ungated execution is impossible for those tools
Tool arguments shall be schema-validated and sanitized before execution
- Artifact
- JSON Schema (or equivalent) per tool + contract tests for invalid payloads
- Pass condition
- 100% of production tools have a declared argument schema; invalid/malicious argument fixtures are rejected at 100% before side effects
MCP tool catalogs shall be signed and supply-chain reviewed before production use
- Artifact
- Signed tool catalog + supply-chain review record + verify-on-load config
- Pass condition
- PASS if production MCP runtimes reject unsigned/unapproved tool catalogs; last review ≤90 days or since last catalog change
Recommended checks
Strengthen posture beyond the gate. Same measurable structure; non-blocking unless elevated by organizational policy.
Dry-run or simulation mode for destructive tools in lower environments
- Artifact
- Destructive-tool catalog marking dry-run/simulation support + sample lower-env dry-run logs
- Pass condition
- 100% of tools classified destructive expose dry-run or simulation in non-prod; last promotion of a destructive tool includes a dry-run evidence link ≤90 days old
Per-tool rate limits and blast-radius budgets
- Artifact
- Per-tool rate-limit and blast-radius policy config + sample enforcement metrics (7 days)
- Pass condition
- Each high-impact tool has a documented QPS/daily cap and max-affected-entities budget; ≥1 limit hit or synthetic test proves enforcement in the last 30 days
Evidence required
- Tool inventory with owners, scopes, and impact ratings
- Authorization policy examples
- Logs of denied tool invocations
Severity & risk
- Severity
- critical
- Impact if violated
- Risk level
- critical
- Typical residual risk (impact × likelihood)
Engineering best practices
- Map each tool to an impact tier (read, write, irreversible, financial)
- Pass user identity into tools; avoid god-mode service accounts where possible
- Return structured errors; do not leak internal stack traces into the model context
- Treat MCP and A2A endpoints as public-facing APIs with the same hardening
Automatic validations
- Contract tests for tool schemas
- Policy-as-code denying unauthorized tool calls
- CI scanning for overly broad tool permissions
Manual validations
- Impact rating review for new tools before production
- Penetration testing focused on tool and MCP surfaces
Examples
- A coding agent can run tests but cannot push to main without human approval
- An MCP file server exposes only a workspace directory, not the host filesystem
References
Crosswalks
MANAGE Manage
NIST AI Risk Management Framework · supports
Secure & Resilient Secure and Resilient
NIST AI Risk Management Framework · aligns-with
§8 Operation
ISO/IEC 42001 · supports
LLM01 Prompt Injection
OWASP Top 10 for Large Language Model Applications · supports
LLM05 Improper Output Handling
OWASP Top 10 for Large Language Model Applications · supports
LLM06 Excessive Agency
OWASP Top 10 for Large Language Model Applications · supports
CC5 Control Activities
SOC 2 Trust Services Criteria · evidence-for
Security Security
AWS Well-Architected Framework · aligns-with
Future evolution
Standardized tool impact taxonomies and portable capability tokens for agent tool use.