Conformal Logit Inference
GitHubDocsSign in

How it works

Three engines, one assumption

Every guarantee rests on exchangeability between your calibration data and your production traffic, never on the model being accurate. A weaker model does not break a guarantee; it makes sets larger, intervals wider, and escalations more frequent.

  1. Engine 01

    Conformal prediction

    Wraps any nonconformity score — an option probability, a sample frequency, a claim-support score — in a threshold calibrated on your own labelled examples.

    The output is a set of options or an interval that contains the correct answer at least 1 − α of the time, or a decision rule whose error rate is bounded.

    Guarantees
    Coverage ≥ 1 − α. Expected risk ≤ α. Risk ≤ α with probability ≥ 1 − δ. FDR ≤ q across a batch.
    Methods
    LAC, APS, RAPS, CQR, conformal risk control, RCPS, Learn-then-Test, Mondrian group-conditional calibration, conformal selection.
    Powers
    Set, Interval, Gate, Claim, Judge, Route.

  2. Engine 02

    Venn-Abers calibration

    Replaces a single opaque confidence number with a probability interval [p₀, p₁], without retraining the model. Under exchangeability, Venn-Abers predictors are well calibrated.

    The width of the interval shows how much the calibration data supports the estimate. Narrow and near 0 or 1 means well supported; wide means it is not, and you see that before any threshold is applied.

    Guarantees
    A calibrated probability interval [p₀, p₁].
    Methods
    Inductive Venn-Abers (IVAP), cross Venn-Abers (CVAP).
    Powers
    Belief, and the probability annotations on Set and Judge answers.

  3. Engine 03

    E-values

    Anytime-valid evidence that stays honest under continuous monitoring and optional stopping. No sample size committed in advance, and no penalty for checking early or often.

    The same machinery controls false discoveries across many simultaneous decisions, even when they depend on each other, and makes a small human-labelled sample go further.

    Guarantees
    False-alarm rate ≤ δ at any stopping time. FDR ≤ q under arbitrary dependence.
    Methods
    Betting-martingale e-processes, e-BH, prediction-powered inference, post-hoc α.
    Powers
    Drift and coverage monitors, label-efficient calibration, batch FDR control.