I am playing Stardew Valley and wantGames
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
100%
Claude Opus 5.5
100%
GPT-6 sol
100%
GPT-5.6 sol
75%
Claude Opus 5
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
100m
Claude Opus 5
49m
GPT-5.6 sol
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
119
Claude Opus 5
67
GPT-6 sol
48
GPT-5.6 sol
35
Claude Opus 5.5
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (8) | ||||
| 100% | 100% | 100% | 100% | |
How to passUse the current Stardew Valley Wiki as the primary reference and identify the six Community Centre rooms and all bundles associated with them. Record the exact room and bundle names and exclude the Joja Community Development Form. | ||||
| 100% | 100% | 100% | 0% | |
How to passFor every bundle, record every required item and the quantity needed. Do not omit items that can only be obtained during particular seasons or under specific conditions. | ||||
| 100% | 100% | 100% | 0% | |
How to passFor every required bundle item, use the current Stardew Valley Wiki to determine the main acquisition method, relevant season where applicable, relevant location where applicable, and important conditions such as weather, crop requirements, fishing requirements, animal requirements, or other prerequisites. If the Wiki does not explicitly state a requested detail, record it as “not stated” rather than inferring it. | ||||
| 100% | 100% | 100% | 100% | |
How to passCreate a spreadsheet with columns for: 1) Room 2) Bundle 3) Item 4) Quantity Required 5) Acquisition Method 6) Season 7) Location / Condition 8) Status 9) Notes Use consistent status values: Not Started | Collected | Submitted | ||||
| 100% | 100% | 100% | 100% | |
How to passMake the spreadsheet easy to use as a live tracker. Add filters to the main table so the player can filter by: - Community Centre room - Bundle - Season - Acquisition method - Status Keep the original room and bundle structure intact. | ||||
| 100% | 100% | 100% | 100% | |
How to passCreate a summary section showing: - Total required item entries - Number marked Collected - Number marked Submitted - Number still Not Started - Overall completion percentage Use formulas so these values update automatically when the status fields are changed. | ||||
| 100% | 100% | 100% | 100% | |
How to passInclude a small notes or instructions section explaining how to use the tracker and how the completion percentage is calculated. Make clear that the tracker covers the traditional Community Centre route and excludes Joja Community Development Form purchases. | ||||
| 100% | 100% | 100% | 100% | |
How to passCheck the completed spreadsheet against the current Stardew Valley Wiki to ensure that every Community Centre room is represented, every traditional Community Centre bundle is included, every required item and quantity is recorded, Joja Community Development Form items are excluded, and the acquisition and seasonal information is supported by the source. | ||||
Failure modes on this task
2 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Standing up the evidence1 mode | — | — | — | 1 |
| No signal in the judge’s words1 mode · the grader | — | — | — | 1 |