Compile a review-ready comparison of the best physicalGames
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
83%
GPT-6 sol
83%
GPT-5.6 sol
83%
Claude Opus 5
67%
Claude Opus 5.5
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
56m
Claude Opus 5
17m
GPT-5.6 sol
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
113
Claude Opus 5
57
GPT-6 sol
53
Claude Opus 5.5
44
GPT-5.6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (6) | ||||
| 100% | 100% | 100% | 100% | |
How to passNavigate to https://www.superpotato.com/ and any Akihabara specific pages such as those linked for the Akihabara store. Record store locations in Tokyo specialties in retro consoles and games any listed opening hours source URLs and confirmation status. | ||||
| 0% | 100% | 100% | 0% | |
How to passNavigate to https://www.suruga-ya.jp/ and search for Akihabara or Tokyo branch pages. Record locations carrying retro games specialties price focus if noted source URLs and availability status for the relative window. | ||||
| 100% | 100% | 100% | 100% | |
How to passNavigate to https://www.bookoff.co.jp/ and https://www.mandarake.co.jp/. Identify Tokyo and nearby city branches known for video games. Record addresses specialties in second hand or vintage titles source URLs and confirmation status. | ||||
| 0% | 0% | 100% | 100% | |
How to passNavigate to https://www.akihabara-beep.com/ https://www.e-trader.jp/ and https://www.retrogamecamp.com/. Record locations in the Tokyo area inventory focus on classic systems any listed hours source URLs and confirmation status. | ||||
| 100% | 100% | 100% | 100% | |
How to passReturn to the major retailer sites and expand search to Yokohama or other nearby cities within easy reach of Tokyo. Record any additional strong options for retro games that can be visited in the relative shopping window. Note source URLs and status. | ||||
| 100% | 100% | 0% | 100% | |
How to passFor every selected store produce a structured entry containing the relative shopping window label store name location city specialty in vintage or retro titles any current hours or stock notes full source URL and confirmation that the data was verified on the live public page at research time. Include a summary ranking or grouping of the strongest options in Tokyo versus nearby cities. Assemble all entries into one coherent review-ready document format. | ||||
Failure modes on this task
5 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Reaching the page1 mode | 1 | — | — | 1 |
| Standing up the evidence1 mode | 1 | — | — | — |
| Delivering the result1 mode | — | — | 1 | — |
| No signal in the judge’s words1 mode · the grader | — | 1 | — | — |