I am setting up a strength training cornerHealth
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
58%
GPT-5.6 sol
50%
GPT-6 sol
8%
Claude Opus 5.5
0%
Claude Opus 5
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
23m
Claude Opus 5
16m
GPT-5.6 sol
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
114
Claude Opus 5
100
Claude Opus 5.5
89
GPT-6 sol
42
GPT-5.6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (12) | ||||
| 0% | 0% | 100% | 0% | |
How to passSearch fitness retailers for adjustable dumbbell pairs that reach at least 80 lbs per dumbbell using a dial, pin, or selector mechanism, and build a shortlist of at least 6 to 8 candidates across at least 2 different retailers, watching for the very common 5 to 50 lb dial models surfacing in results and excluding any whose maximum weight per dumbbell falls short of 80 lbs. | ||||
| 0% | 100% | 100% | 0% | |
How to passOpen each shortlisted dumbbell pair's own live product page and confirm in the specification section that the adjustment mechanism matches the definition in the request and that the maximum weight per dumbbell is at least 80 lbs, not inferred from a product name like "Elite" or "Pro" alone. If the page sells the item as a single dumbbell rather than a pair, note that explicitly and calculate the pair price by doubling the single price, flagging that this was done. Record retailer, product name, price, weight range, adjustment mechanism, noting explicitly if price or shipping does not render on a plain page fetch. Continue until exactly 4 pairs pass every check, drawn from at least 2 different retailers. | ||||
| 100% | 100% | 0% | 0% | |
How to passSearch fitness retailers for adjustable weight benches offering flat, incline, and decline positions, and build a shortlist of at least 6 to 8 candidates across at least 2 retailers, watching for incline-only benches without a true decline position surfacing in results despite the search terms and flagging them for exclusion rather than trusting search relevance to have filtered them out. | ||||
| 0% | 0% | 100% | 0% | |
How to passOpen each shortlisted bench's own live product page and confirm the spec section states a decline position of at least 10 degrees in addition to flat and incline of at least 60 degrees, not just a seat that tilts slightly. Confirm the stated weight capacity is 800 lbs or more and that the bench is described as folding for storage rather than merely portable. Record retailer, product name, price, incline and decline angles, weight capacity, and shipping cost and delivery window, converting any measurement given in kilograms to pounds using a consistent rounding rule, and noting explicitly anything that fails to load. Continue until exactly 4 benches pass every check, drawn from at least 2 different retailers. | ||||
| 0% | 0% | 0% | 0% | |
How to passSearch fitness retailers for four-post freestanding power racks with integrated safety spotter bars, and build a shortlist of at least 6 to 8 candidates across at least 2 retailers, watching for two-post open squat stands and wall-mounted folding racks surfacing in results despite the freestanding four-post search terms. | ||||
| 0% | 100% | 100% | 0% | |
How to passOpen each shortlisted rack's own live product page and verify every required rack criterion against the product specifications. Continue searching until 4 distinct racks satisfying all criteria are verified, using at least 2 retailers. If fewer than 4 qualifying racks can be verified after a reasonable search, document the shortfall rather than including products that fail a requirement. | ||||
| 0% | 100% | 100% | 0% | |
How to passRecheck every candidate across all three categories against the spec sheet rather than the product title or marketing copy alone, since terms like adjustable, heavy duty, and commercial grade are used loosely in titles, and drop or flag any candidate whose actual spec section does not back up the claim the title implies. | ||||
| 0% | 100% | 100% | 0% | |
How to passReview every recorded field across all three categories and convert measurements to the units required by the task where necessary, preserving the original source values when possible. | ||||
| 0% | 0% | 0% | 0% | |
How to passSet up a spreadsheet with one tab per category, dumbbells, benches, and racks, with columns for retailer, product name, price, the applicable category-specific specifications, product page URL, and a notes field for verification issues or failed criteria. | ||||
| 0% | 0% | 0% | 0% | |
How to passEnter all 4 dumbbell pairs, 4 benches, and 4 racks into their respective tabs with every field filled in from what was recorded in the earlier steps, leaving no field blank without an explicit note explaining why. | ||||
| 0% | 0% | 0% | 0% | |
How to passCalculate the total setup price for every combination of 1 dumbbell pair, 1 bench, and 1 rack, showing the arithmetic for each combination in a dedicated verification tab, and identify every combination that comes in at 2400 dollars or less. | ||||
| 0% | 100% | 100% | 0% | |
How to passAdd a summary tab that names the recommended dumbbell pair, bench, and rack combination, shows the line-item prices and the total against the 2400 dollar cap with the arithmetic spelled out, and briefly states why this combination was chosen over the other qualifying combinations if more than one met the budget. | ||||
Failure modes on this task
34 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Standing up the evidence2 modes | 3 | 4 | 3 | 3 |
| Delivering the result1 mode | 6 | 1 | 1 | 5 |
| Reaching the page1 mode | 1 | — | — | — |
| No signal in the judge’s words1 mode · the grader | 1 | 1 | 1 | 4 |