# OpenAI web_search — retrieval accuracy figures, recomputed and corrected (2026-08-25)

The published report stated two different things that don't reconcile: a table showing "Top-1 26% |
Top-3 25.5% | Top-10 26%" and a narrative claim that "top-1, top-3, and top-10 all collapse to the same
26% retrieval figure." Neither the table (26% appears twice, 25.5% once) nor the narrative (a single
26% figure) is correct. Recomputed directly from `OPENAI-scored-run-export.csv`, 47 answerable rows
(Q01–Q47, excluding the 5 unanswerable Q48–Q52):

| Metric | Hits | n | % |
|---|---|---|---|
| Top-1 | 11 | 47 | **23.4%** |
| Top-3 | 12 | 47 | **25.5%** |
| Top-10 | 12 | 47 | **25.5%** |

**Top-3 and Top-10 are exactly equal (both 12/47), but Top-1 is not equal to either** — the "all three
collapse to one number" claim is false as stated. There is exactly one counterexample:

**Q23 · ambiguity — `Apollo pricing`.** Quoted verbatim from `OPENAI-per-query-output.md`:

> **openai/answer** — **CORRECT** · top3 PASS · **hit rank 2** · $0.01 · 6007.2ms
>
> If you mean **Apollo.io**, its current listed pricing is: [pricing table follows]

`supporting_url_rank: 2` in the scored CSV confirms it directly. OpenAI's answer disambiguated the
query ("If you mean Apollo.io...") and cited the matching `apollo.io/pricing` URL as its **second**
citation, not its first — contradicting the structural assumption that a cited URL is always at
position 1. This is the one query separating the 23.4% top-1 figure from the 25.5% top-3/top-10
figure.

**Corrected framing:** Top-3 and Top-10 collapse to the same number (25.5%) because OpenAI's answer
mode returns citations rather than a ranked list, and once a supporting citation appears at all it
almost always appears first — Q23 is the sole exception, where a second citation used to disambiguate
the query pushed the correct one to rank 2. Top-1 (23.4%) is measurably lower than Top-3/Top-10 (25.5%)
for exactly this reason, and should be reported as a distinct figure, not folded into "all three are
26%."