Compile a review-ready comparison of the main officialNews & Media Publishers
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
100%
Claude Opus 5.5
83%
Claude Opus 5
17%
GPT-5.6 sol
0%
GPT-6 sol
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
20m
Claude Opus 5
7m
GPT-5.6 sol
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
48
Claude Opus 5
23
GPT-6 sol
19
Claude Opus 5.5
14
GPT-5.6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (6) | ||||
| 100% | 0% | 0% | 100% | |
How to passNavigate to https://www.granma.cu/ and https://en.granma.cu/. Record the publication focus as the official voice of the Communist Party any language options coverage areas source URLs and confirmation status. | ||||
| 100% | 0% | 0% | 100% | |
How to passNavigate to https://www.cubadebate.cu/. Record its publication/content focus, available language options, digital or multimedia access, any print or other platform availability shown on the official site, source URL(s), and confirmation status. | ||||
| 100% | 0% | 0% | 100% | |
How to passNavigate to https://www.prensa-latina.cu/ and https://www.acn.cu/. For each agency, record its publication role and coverage focus, available language options, any print, digital, broadcast, or other platform availability shown on the official site, source URL(s), and confirmation status. | ||||
| 100% | 0% | 100% | 0% | |
How to passNavigate to https://www.juventudrebelde.cu/ and https://www.trabajadores.cu/. For each publisher, record its publication focus and coverage areas, available language options, any print, digital, multimedia, or other platform availability shown on the official site, source URL(s), and confirmation status. | ||||
| 100% | 0% | 0% | 100% | |
How to passReturn to the main official sites to reconfirm the currently visible homepage structure, language availability, update indicators, and print or multi-platform availability. Identify the publishers most relevant to the task's goal of staying informed on current events and local developments, and record the final set with cross-references to the original official source URLs. | ||||
| 100% | 0% | 0% | 100% | |
How to passFor every selected publisher, produce a structured entry containing the publisher name, publication focus, available languages, print or multi-platform availability, update indicators if shown, full official source URL(s), and confirmation that the information was verified on the live public page at research time. Assemble all entries into one coherent review-ready comparison for final decision-making. | ||||
Failure modes on this task
12 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Delivering the result1 mode | — | 4 | 3 | — |
| Reaching the page1 mode | — | 1 | — | — |
| Standing up the evidence1 mode | — | — | 1 | — |
| No signal in the judge’s words1 mode · the grader | — | 1 | 1 | 1 |