The interface

Seven primitives, every one guarantee-bearing

Mix any of them in a single request against the same context. Every answer carries a guarantee card: type, method, target, calibration profile, sample size, and last audit.

The seven query primitives, what each returns, and its guarantee
Primitive	Returns	Guarantee
Belief	A Venn-Abers probability interval for one statement	Calibrated interval [p₀, p₁]
Set	A set of options that contains the correct one	Coverage ≥ 1 − α
Interval	A conformalized ordinal or continuous interval	Coverage ≥ 1 − α
Gate	`auto_approve`, `escalate`, or `abstain`	Expected risk ≤ α; risk ≤ α with probability ≥ 1 − δ; or batch FDR ≤ q
Claim	Long-form output filtered to its supported claims	P(every retained claim is supported) ≥ 1 − α
Judge	An LLM-as-judge verdict, with an escalation cascade	Human agreement ≥ 1 − α on verdicts that are not escalated
Route	Which backend in a model cascade serves the request	P(cost per request ≤ budget) ≥ 1 − α, or an accuracy bound

Every guarantee is marginal, or group-conditional with Mondrian profiles, over data exchangeable with the named calibration profile. None is a promise about one decision in isolation.

The SDK

One call. Typed answers. A guarantee on each.

Queries are plain Python objects. Answers come back typed, each with the guarantee card that says exactly what was proven and on which calibration.

Python

from cli_sdk import CLIClient, Set, Gate

backend = {"provider": "openai", "model": "gpt-4.1-2025-04-14"}
ticket = {"ticket": "Our bank payout has been pending since Monday."}

with CLIClient() as client:
    department = client.evaluate(context=ticket, backend=backend, queries={
        "department": Set(
            instructions="Which support queue should own this ticket?",
            options={"billing": None, "technical": None, "sales": None},
            calibration_profile="support-routing-v3",
            alpha=0.10,
        ),
    }).answers["department"]

    # The gate judges the exact action: this ticket, this proposed queue.
    gate = client.evaluate(context={**ticket, "proposed_queue": department.top},
                           backend=backend, queries={
        "route": Gate(
            instructions="Is proposed_queue the correct team for this ticket?",
            calibration_profile="support-auto-route-v3",
            guarantee="fdr",
            target=0.05,
            delta=0.05,
        ),
    }).answers["route"]

print(department.guarantee.describe())
# Contains the correct answer at least 90% of the time
# on profile 'support-routing-v3' (n=1204).

if gate.approved:
    assign_queue(department.top)
else:
    # gate.decision is "escalate" or "abstain"
    send_to_human_triage(department.set, reason=gate.decision)

result.answers["department"]SetAnswer

set: ["billing", "technical"]

guarantee

type: coverage
method: APS
alpha: 0.10
calibration_profile: support-routing-v3
calibration_n: 1204
coverage_ci: [0.886, 0.914]
last_audited: 2026-09-18T00:00:00Z

result.answers["route"]GateAnswer

decision: escalate
approved: False

guarantee

type: fdr
method: LTT
target: 0.05
delta: 0.05
calibration_profile: support-auto-route-v3
calibration_n: 1204
last_audited: 2026-09-18T00:00:00Z