Agent frameworks
Put a guarantee on every action your agent takes
Before your agent sends a refund, holds an account, or approves a claim, CLI checks the evidence against a bar you set. If it clearly clears the bar, the action goes through. If it doesn't, a person looks at it first — nothing runs on a guess.
1. Your agent proposes an action
A tool call — issue_refund, hold_account, approve_claim — arrives with the context behind it: the case, the amount, the policy.
2. CLI checks the evidence
Not a confidence score. A calibrated guarantee decides whether the evidence clears the specific bar you set for that action, on your own labelled history.
3. The right thing happens
The tool runs, a person reviews it, or it's blocked outright — routed straight into the review mechanism your framework already has.
Why it helps
What you get that a confidence threshold can't give you
No blind spots
Every guarded call is allowed, escalated, or blocked. Never silently skipped, and never left to a threshold no one wrote down.
Fails closed
A missing, undersized, or stale calibration profile never approves anything by default. It escalates to a person instead.
Drops into what you have
Middleware for LangChain, a guard node for LangGraph, a callback for Google ADK, or function middleware for Microsoft Agent Framework — a few lines, not a rewrite.
Any model, including your own
OpenAI, Azure OpenAI, Anthropic Claude, Gemini, or open weights on vLLM or SGLang. Local mode calibrates and scores in your own process, and keys never leave it.
A number someone can defend
Every decision carries a guarantee card: the target, the method, the calibration size, and the date it was last audited — ready for a review, not just a demo.
Catches silent drift
An e-value monitor raises an alarm the moment a model alias moves or a server is re-quantized and the guarantee no longer holds — before it costs you.
Integrations
Every tool call an agent proposes gets a calibrated decision
An agent's tool call is a decision. CLI decides each guarded call as allow, escalate, or block, from an answer with a stated guarantee, and maps the three onto the human-in-the-loop mechanism your framework already has.
| Framework | Where the guard runs | Human review | Examples |
|---|---|---|---|
| LangChain | Middleware on create_agent: scores each proposed call once, enforces the stored decision | HumanInTheLoopMiddleware: approve, edit, or reject | Refunds; patient-portal triage |
| LangGraph | A guard node between the model node and ToolNode | interrupt() and Command(resume=...), on any checkpointer | Insurance claim payments; discharge-summary claim checks |
| Google ADK | before_tool_callback on one agent, or a plugin for every agent a runner drives | Tool confirmation, resumed on the same session | Anti-money-laundering account holds; clinical-trial pre-screening |
| Microsoft Agent Framework | Function middleware on the agent | Approval requests bound to the session, so a forged approval never runs a tool | Credit risk tiers; drug-safety reports |
- Allow
- The calibrated answer clears the rule: a Gate auto-approves, a Belief pair sits entirely above the threshold, or a Set is exactly the proposed label. The tool runs.
- Escalate
- The calibration cannot settle it: a straddling pair, a set with more than one label, an interval across a boundary. The framework pauses for a person.
- Block
- The answer is clearly on the wrong side of the rule. The tool does not run, and the agent is told why.
- Fail closed
- A missing, undersized, or stale calibration profile, or an error while scoring, never allows. It escalates.
Python, LangChain
from cli_sdk import Gate, LocalCLIClient
from cli_sdk.evidence import vllm_backend # or OpenAI, Azure OpenAI, Claude, Gemini
from cli_sdk.integrations import GuardRule, ToolGuard
from cli_sdk.integrations.langchain import cli_middleware
client = LocalCLIClient(vllm_backend("google/gemma-4-12B-it"))
refund = Gate(instructions="Is issuing this refund correct under the policy?",
calibration_profile="refund-approvals-v1", guarantee="risk", target=0.05)
client.calibrate(refund, labelled_refunds) # your own labelled history
guard = ToolGuard(client, [GuardRule(tool="issue_refund", query=refund,
context=refund_context)])
agent = create_agent(model, tools=[lookup_order, issue_refund],
middleware=cli_middleware(guard), checkpointer=InMemorySaver())