I want to plan a future trip aroundSports
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
100%
Claude Opus 5.5
67%
GPT-6 sol
67%
Claude Opus 5
17%
GPT-5.6 sol
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
14m
GPT-5.6 sol
14m
Claude Opus 5
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
47
Claude Opus 5
30
Claude Opus 5.5
21
GPT-6 sol
12
GPT-5.6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (6) | ||||
| 100% | 100% | 100% | 0% | |
How to passResearch professional football matches taking place around the target period. Verify the match, competition, venue, participating teams, and official event information using reliable sources. Identify several realistic options rather than selecting the first match found. | ||||
| 100% | 100% | 0% | 100% | |
How to passFor the strongest match options, investigate the stadium layout, seating areas, entry requirements, prohibited items, accessibility information, transportation options, and official matchday guidance. Use the club or stadium's official information wherever possible. | ||||
| 100% | 100% | 0% | 100% | |
How to passResearch restaurants, cafés, landmarks, and activities near the selected stadium. Group them geographically and determine which could realistically be visited before or after the match without creating unnecessary travel. | ||||
| 100% | 100% | 0% | 100% | |
How to passResearch public transportation and walking routes between the selected accommodation area, stadium, restaurants, and other activities. Account for realistic travel time, stadium entry requirements, and additional time around the match. | ||||
| 100% | 0% | 0% | 0% | |
How to passCompare the strongest options based on overall match experience, stadium accessibility, surrounding activities, transportation convenience, and practicality. Explain the trade-offs instead of simply choosing the most famous team. | ||||
| 100% | 0% | 0% | 100% | |
How to passCreate a practical itinerary from arrival through departure, including where to go, approximate sequence of activities, transportation between locations, what to verify before attending, and Use the official football competition or club fixture page, the selected stadium’s official matchday and attendance pages, and official/local transport and nearby-activity sources. needed to confirm event details later. | ||||
Failure modes on this task
9 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Delivering the result1 mode | — | 1 | 2 | — |
| Standing up the evidence1 mode | — | — | — | 2 |
| No signal in the judge’s words1 mode · the grader | — | 1 | 3 | — |