What is cost per verified success?

Cost per token and cost per API call are the numbers vendors publish, because they're the numbers vendors control. Neither one tells you what a business actually pays for: a correct result. A cheap model that gets the wrong answer costs infinitely more than its token price, because someone still has to catch the error, and a document processed incorrectly can cost far more than the API call that produced it.

Cost per verified success closes that gap by putting a real, checked answer in the denominator instead of a raw output count.

For a given task type and model: sum the full cost of every attempt, including retries and escalations, then divide by the count of attempts whose output matched computed ground truth under the task's scoring rule. An attempt that returns a wrong-but-well-formed answer is not a success, even though it completed without error and cost money like every other call.

cost per verified success = (sum of all attempt costs) ÷ (count of ground-truth-verified correct outputs)

It is specific to the task type and corpus it was measured on. A cost-per-verified-success figure for invoice line matching says nothing about eligibility determination or PII redaction; the number does not generalize across task shapes. It also depends on the ground-truth scorer being independently validated — a scorer that hasn't been tested with deliberately wrong answers can produce a cost-per-verified-success figure that looks precise and is not trustworthy. And it is a point-in-time figure: model pricing changes, sometimes sharply, so a figure from an earlier benchmark round should carry its date and be treated as stale until re-measured.