Failure Modes
Every verifier failure is counted under exactly one mode — 1,394 of them across this drop.
- Failure stage
- 4 of them — WHERE in the work a trial broke, at the earliest point that broke.
- Failure mode
- 5 of them — WHAT went wrong, named, inside one stage. Every mode belongs to exactly one stage.
Most trials break down at Delivering the result
The run gathered material and never turned it into the comparison, plan, document or answer the task asked for. It is tagged in 200 of the 399 labelled runs — more than any other stage, and every model on the board is in that number.
The most common single failure is Unfinished output or synthesis
It appears in 200 of the 399 labelled runs, across 89 of the 100 tasks that carry labels — so it is a property of the work rather than of one task.
Claude Opus 5.5 carries the smallest share of this drop's failures
111 verifier failures out of the 941 charged to any model, over 100 trials — 12% of them. Verifier failures labelled grader error are left out of that denominator, here as everywhere else on the site. What follows is what it does differently from the rest of the board, not its biggest numbers, which mostly repeat the field's.
It handles Unfinished output or synthesis better than the field
Tagged at Delivering the result in 22% of its 100 trials, against 59% for the median of the other 3 models. The same measurement as above, read the other way — a failure reading that only ever accuses is not a reading.
GPT-6 sol carries the 2nd largest share of this drop's failures
279 verifier failures out of the 941 charged to any model, over 100 trials — 30% of them. Verifier failures labelled grader error are left out of that denominator, here as everywhere else on the site. What follows is what it does differently from the rest of the board, not its biggest numbers, which mostly repeat the field's.
Its characteristic failure is Unfinished output or synthesis
Tagged at Delivering the result in 69% of its 100 trials, against 51% for the median of the other 3 models — the widest gap it opens anywhere in the taxonomy.
It handles Missing required details better than the field
Tagged at Standing up the evidence in 38% of its 100 trials, against 44% for the median of the other 3 models. The same measurement as above, read the other way — a failure reading that only ever accuses is not a reading.
GPT-5.6 sol carries the 3rd largest share of this drop's failures
243 verifier failures out of the 941 charged to any model, over 99 trials — 26% of them. Verifier failures labelled grader error are left out of that denominator, here as everywhere else on the site. What follows is what it does differently from the rest of the board, not its biggest numbers, which mostly repeat the field's.
Its characteristic failure is Unfinished output or synthesis
Tagged at Delivering the result in 59% of its 99 trials, against 51% for the median of the other 3 models — the widest gap it opens anywhere in the taxonomy.
Claude Opus 5 carries the largest share of this drop's failures
308 verifier failures out of the 941 charged to any model, over 100 trials — 33% of them. Verifier failures labelled grader error are left out of that denominator, here as everywhere else on the site. What follows is what it does differently from the rest of the board, not its biggest numbers, which mostly repeat the field's.
Its characteristic failure is Navigation or access
Tagged at Reaching the page in 20% of its 100 trials, against 9% for the median of the other 3 models — the widest gap it opens anywhere in the taxonomy.
It handles Unfinished output or synthesis better than the field
Tagged at Delivering the result in 51% of its 100 trials, against 59% for the median of the other 3 models. The same measurement as above, read the other way — a failure reading that only ever accuses is not a reading.
Failure modes
Derived by this site, not published by Apollo. Every rubric the judge failed is read for the words its reasoning uses, and placed in the first of four signals that matches, in the order a run breaks: navigation or access, then weak source verification, then missing details, then an unfinished result. One mode per failed rubric, so every count here is a count of failed rubrics. A failure matching no signal is counted under no recognised signal and never ranked. A rubric the judge could not grade is not a failure and carries no mode. The bar under each stage is its share of every classified failure — how much of the drop it accounts for. The dots are how often each model’s own trials were tagged with it, on one scale across every card. Two different denominators, so a large stage is not the same claim as a model that hits one often.
- Claude Opus 5
- Claude Opus 5.5
- GPT-5.6 sol
- GPT-6 sol
Delivering the result41.0%
The run gathered material and never turned it into the comparison, plan, document or answer the task asked for.
1 mode- Unfinished output or synthesis89 tasks · 572 failures · 41.0%The judge says the run never produced, created, synthesised or completed the requested result.Seen on0a42045c-25ce-577e-b217-3c07f78c61460ae0492d-7f6c-5169-b55e-59b801a5a69e12e288df-c90b-5c29-8417-acd1135922fc15b8b4dc-307f-5fb6-ad11-e4087ecc79911639f872-cf17-5e9c-9f49-f029cc2c59271785cae6-ff06-580e-99c7-1fa226e3507fand 83 more
Trials tagged, per model47 pts spread
0%25%50%75%
Claude Opus 551 / 100 trialsClaude Opus 5.522 / 100 trialsGPT-5.6 sol58 / 99 trialsGPT-6 sol69 / 100 trialsStanding up the evidence21.4%
The run reached the web but the result it built was not supported, or not complete.
2 modes- Missing required details77 tasks · 255 failures · 18.3%The judge finds a result, but with required fields or coverage missing, omitted or only partly done.Seen on0ae0492d-7f6c-5169-b55e-59b801a5a69e12e288df-c90b-5c29-8417-acd1135922fc15b8b4dc-307f-5fb6-ad11-e4087ecc79911785cae6-ff06-580e-99c7-1fa226e3507f17bb688a-bfc2-5885-a862-00cec97b5a8b1d4edadd-d004-515b-9cd9-8747ec9ea9cdand 71 more
- Weak source verification17 tasks · 43 failures · 3.1%The judge says a claim was not verified, confirmed or cited, or rested on an unofficial source.Seen on0ae0492d-7f6c-5169-b55e-59b801a5a69e19c17bee-65df-5462-8c57-c53dc27726ce2bb9ba28-a74d-5fe6-a9ec-8e5a7862bd834a4c1b56-fb73-59d0-ba34-f6d9290628014ad27177-4106-5e84-991d-537b92f7eb9066dd1acd-fb73-55ea-aae7-2e2e55d02c63and 11 more
Trials tagged, per model21 pts spread
0%25%50%75%
Claude Opus 554 / 100 trialsClaude Opus 5.533 / 100 trialsGPT-5.6 sol48 / 99 trialsGPT-6 sol40 / 100 trialsReaching the page5.1%
The run could not get to what it needed: a page would not load, or a login, paywall or bot check stood in the way.
1 mode- Navigation or access39 tasks · 71 failures · 5.1%The judge records a load, login, paywall, captcha or blocking problem.Seen on0a42045c-25ce-577e-b217-3c07f78c61460ae0492d-7f6c-5169-b55e-59b801a5a69e12e288df-c90b-5c29-8417-acd1135922fc15b8b4dc-307f-5fb6-ad11-e4087ecc79911785cae6-ff06-580e-99c7-1fa226e3507f17bb688a-bfc2-5885-a862-00cec97b5a8band 33 more
Trials tagged, per model11 pts spread
0%25%50%75%
Claude Opus 520 / 100 trialsClaude Opus 5.59 / 100 trialsGPT-5.6 sol13 / 99 trialsGPT-6 sol9 / 100 trialsNo signal in the judge’s wordsthe grader32.5%
The judge failed the rubric in words none of the signals above recognise. Counted, so every failure is accounted for, and never ranked.
1 mode- No recognised signal89 tasks · 453 failures · 32.5%None of the four signals matches the judge’s reasoning.Seen on0a42045c-25ce-577e-b217-3c07f78c61460ae0492d-7f6c-5169-b55e-59b801a5a69e12e288df-c90b-5c29-8417-acd1135922fc15b8b4dc-307f-5fb6-ad11-e4087ecc79911785cae6-ff06-580e-99c7-1fa226e3507f17bb688a-bfc2-5885-a862-00cec97b5a8band 83 more
Trials tagged, per model30 pts spread
0%25%50%75%
Claude Opus 559 / 100 trialsClaude Opus 5.529 / 100 trialsGPT-5.6 sol52 / 99 trialsGPT-6 sol58 / 100 trials
Failure profile
How often each model broke down at each stage of the work, as a share of its trials — a different measurement from the counts below, and the one that compares models. The chart opens on the stages the field does worst at and on the models furthest apart there; pin a model in the picker and it re-picks to that model’s own worst. Every stage and every model stays one chip away.
Point at a shape — or a model above — to read its values.
Incidence — model × failure stageTrials tagged with each plotted stage, as a share of the trials that model made. The figure beside a stage is its spread — the gap between the model that hits it most and the one that hits it least.
1Delivering the result47 pts
2Standing up the evidence21 pts
3Reaching the page11 pts