What is guess rate on ambiguous input?

Accuracy percentages describe how a system performs on cases with a clear right answer. They say nothing about what happens when the input is genuinely incomplete or contradictory — and real documents are incomplete or contradictory often enough that this is not an edge case. A system that guesses plausibly instead of flagging uncertainty produces output that looks identical to a correct answer, which makes it the failure mode least likely to be caught downstream.

A model that says "I don't have enough information" is more useful, and more trustworthy, than one that always answers — even when the confident answer happens to be wrong only some of the time.

A subset of the test corpus is constructed with missing or conflicting facts, such that no correct determination exists — the verified correct output is "insufficient information," known at generation time by construction. Guess rate is the fraction of those cases where the system instead returns a specific determination.

guess rate = (ambiguous cases answered with a specific determination) ÷ (total ambiguous cases)

It does not measure accuracy on the cases that do have a correct answer — a low guess rate and a high accuracy rate are separate properties, and a system can be strong on one and weak on the other. It also depends on how "ambiguous" was constructed for a given corpus; a task shape's ambiguous cases should reflect real gaps a client's documents actually contain, not artificial ones, or the number won't predict real-world behavior. And like every figure here, it is scoped to the task type and sample size it was measured on.