Use cases
Where a guarantee changes the decision
Seven patterns teams ship with CLI. Each replaces a hand-picked threshold with a calibrated procedure and a statement someone can defend in review.
Support routing with an FDR gate
Auto-route tickets to billing, technical, or sales only while the error rate of the automated batch stays inside budget. Tickets the gate escalates reach a person with the calibrated set of plausible teams attached.
GuaranteeWith 95% confidence, at most 5% of auto-routed tickets go to the wrong queue. Nightly batches get the same 5% bound in expectation across the batch.
Gate referenceRAG claim filtering
Split a generated answer into atomic claims, score each one against the retrieved documents, and ship only the subset that clears a calibrated threshold. Dropped claims come back with a reason.
GuaranteeEvery retained claim is supported in at least 95% of answers, over questions exchangeable with the calibration set.
Claim referenceHigh-stakes approvals
Give balance checks and wire transfers different bars. Each action type gets its own calibration profile, its own target, and its own audit history, so a reviewer evaluates a stated number rather than a threshold picked in a spreadsheet.
GuaranteeWith probability at least 95% over the calibration draw, the error rate on auto-approved transfers is at most 1%.
Guarantee referenceLLM-judge cascades
A low-cost judge decides first. Verdicts its calibration cannot support escalate to a stronger judge, then to a human queue, so the expensive step runs only on the genuinely ambiguous comparisons.
GuaranteeAgreement with human raters of at least 90% on verdicts that are not escalated.
Judge referenceCost routing
Serve most requests from a self-hosted open-weight model and pay for a frontier API call only when the calibrated cascade says the inexpensive answer is not good enough.
GuaranteeCost per request stays under the target, such as 0.4 cents, with probability at least 90%.
Route referenceSilent model update detection
Label a small sample of live traffic. A betting-martingale e-process tracks realized coverage and raises an alarm the first time the evidence says a guarantee no longer holds, for example after a model alias moves or a server is re-quantized.
GuaranteeFalse-alarm rate at most 5%, no matter when or how often the monitor is checked.
Monitoring guideEntity matching with Venn-Abers intervals
Ask whether two records describe the same customer. A narrow interval near 0 or 1 merges or separates automatically; a wide interval, or one that straddles your decision threshold, goes to review.
GuaranteeA calibrated probability interval [p₀, p₁] for every pair, with its width reporting how well the calibration data supports it.
Belief reference
Risk-sensitive domains
Built for decisions someone has to defend
Eight runnable examples guard agent actions in healthcare, finance, insurance, and financial-crime compliance. Each runs offline with a mock model, and with your own keys when you are ready.
Healthcare
Patient-portal triage, discharge summaries, clinical-trial pre-screening, and drug-safety reporting. Routine, low-acuity actions are automated only when the calibration settles them; everything urgent or ambiguous goes to a clinician.
GuaranteeTriage replies are automated only when the 95% prediction set is a single low-acuity level. In at least 90% of published discharge summaries, every claim kept is supported by the chart.
Finance and lending
Refunds and credit risk tiers. Each action has its own calibration profile and target, and tier coverage is calibrated separately for each application channel, so a strong overall number cannot hide a weaker channel.
GuaranteeAt most 5% of refund requests are auto-approved and wrong, in expectation. Tier sets cover the correct tier at least 90% of the time within each channel.
Insurance
Claim payments approved by an agent that reads the policy and the claim file. Borderline claims pause for an adjuster, who can approve, edit the amount, or reject.
GuaranteeWith 90% confidence over the calibration draw, the rate of payments auto-approved and wrong is at most 5%.
Financial-crime compliance
Account holds on anti-money-laundering alerts, one at a time or as a nightly queue decided together. Every hold the gate does not approve goes to an analyst.
GuaranteeWith 90% confidence, at most 10% of automatic holds are unwarranted. For a nightly queue, conformal selection bounds the expected unwarranted fraction across the batch.
The examples use synthetic data and are not medical, financial, or legal advice. A guarantee bounds error rates on data like the calibration set; it does not make any single decision safe, and qualified people stay in the loop for everything the guard escalates. Read the risk-sensitive domains playbook.