As a resident already living in Yerevan ArmeniaHealth
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
100%
Claude Opus 5.5
80%
GPT-6 sol
80%
GPT-5.6 sol
80%
Claude Opus 5
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
38m
Claude Opus 5
14m
GPT-5.6 sol
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
80
Claude Opus 5
49
Claude Opus 5.5
38
GPT-6 sol
33
GPT-5.6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (5) | ||||
| 100% | 100% | 100% | 0% | |
How to passNavigate to https://visityerevan.am/ and any food, restaurant, or dining sections. Record restaurants highlighted for traditional Armenian cuisine, key dishes or specialties mentioned, and location information in Yerevan. Capture source URLs and confirmation status. | ||||
| 100% | 100% | 100% | 100% | |
How to passNavigate to https://dolmama.am/ and https://www.yeremyanprojects.com/en/restaurants/ (covering Lavash, Tavern Yerevan, and related brands). Record menu focus on Armenian specialties, atmosphere or dining style notes, and any practical visitor information. Capture source URLs and confirmation status. | ||||
| 100% | 100% | 100% | 100% | |
How to passIdentify and visit public pages for other restaurants emphasizing classic or regional Armenian preparations (using Visit Yerevan listings or linked sites). Record signature dishes, atmosphere notes, and neighborhood details. Capture source URLs and confirmation status. | ||||
| 100% | 100% | 100% | 100% | |
How to passReturn to the main sources to reconfirm focus on Armenian-specific cuisine. Select strong options suitable for a relative dining window beginning at least 30 days from task start. Record the final shortlist with cross-references to the original source URLs. | ||||
| 100% | 0% | 0% | 100% | |
How to passFor every selected restaurant produce a structured entry containing the relative dining window label, restaurant name, key Armenian dishes or cuisine focus, atmosphere or location notes, full source URL, and confirmation that the data was verified on the live public page at research time. Include a short summary of the strongest traditional options. Assemble all entries into one coherent review-ready document format. | ||||
Failure modes on this task
3 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Delivering the result1 mode | — | 1 | 1 | — |
| No signal in the judge’s words1 mode · the grader | — | — | — | 1 |