I want to plan a practical one-week food-focusedFood and Drink
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
100%
Claude Opus 5
29%
GPT-6 sol
14%
Claude Opus 5.5
14%
GPT-5.6 sol
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
39m
Claude Opus 5
5m
GPT-5.6 sol
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
62
Claude Opus 5
12
GPT-6 sol
6
GPT-5.6 sol
0
Claude Opus 5.5
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (7) | ||||
| 0% | 0% | 0% | 100% | |
How to passSearch official Seoul tourism sources, restaurant websites, reputable food publications, and map/review sources to identify at least 12 food experiences across Seoul. Include a mix of Korean barbecue, traditional Korean food, noodles or soups, street food, markets, cafés, and one or two distinctive regional or seasonal specialties. Record the location, cuisine or dish, and source for each option. | ||||
| 0% | 100% | 100% | 100% | |
How to passFor each shortlisted option, check recent map or review information and the restaurant's own website when available. Compare review themes, typical meal cost, location, and any practical considerations such as reservations, queueing, limited operating days, or cash-only requirements. Exclude options that cannot be reasonably verified from public sources. | ||||
| 0% | 0% | 0% | 100% | |
How to passUse Google Maps and other reliable map sources to group the selected food stops by neighborhood. Minimize unnecessary cross-city travel and identify convenient combinations of food stops with nearby attractions or markets. Keep each day's activities geographically realistic rather than simply selecting the highest-rated places. | ||||
| 0% | 100% | 0% | 100% | |
How to passFor every selected restaurant or market, verify the address, usual opening pattern, reservation requirements, ordering method, and any important visitor information from official or reliable sources. Where information differs between sources, record the discrepancy rather than assuming one source is correct. | ||||
| 100% | 0% | 0% | 100% | |
How to passCreate a seven-day Seoul itinerary with breakfast, lunch, dinner, and optional snack or café stops where appropriate. Include no more than two major food destinations per meal period and leave reasonable time for travel and sightseeing. Make sure the itinerary includes different types of Korean food rather than repeating the same cuisine throughout the trip. | ||||
| 0% | 0% | 0% | 100% | |
How to passEstimate the expected food spending for one traveler using publicly listed menu prices or clearly identified recent price information. Compare the selected experiences on cost, uniqueness, convenience, and review sentiment. Identify at least three lower-cost alternatives that could replace more expensive meals without substantially changing the character of the itinerary. | ||||
| 0% | 0% | 0% | 100% | |
How to passCreate a concise seven-day food itinerary containing the day, neighborhood, meal, selected restaurant or market, recommended dish or experience, estimated cost, practical access information, and source link. Include a summary of the total estimated food budget, the main trade-offs, backup options, and any details that should be rechecked before traveling. Do not make reservations or purchases. Country: South Korea Subjects: Food and Drink > Restaurants and Delivery, Travel and Tourism > Tourist Attractions, Travel and Tourism > Ground Transportation Deliverable: A written plan / document | ||||
Failure modes on this task
17 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Standing up the evidence1 mode | 2 | 3 | 2 | — |
| Delivering the result1 mode | — | 1 | 3 | — |
| No signal in the judge’s words1 mode · the grader | 4 | 1 | 1 | — |