I want to purchase a home blood-pressure monitorHealth
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
83%
Claude Opus 5.5
67%
Claude Opus 5
33%
GPT-6 sol
33%
GPT-5.6 sol
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
28m
GPT-5.6 sol
17m
Claude Opus 5
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
48
Claude Opus 5
38
GPT-5.6 sol
21
GPT-6 sol
8
Claude Opus 5.5
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (6) | ||||
| 100% | 100% | 100% | 100% | |
How to passResearch authoritative guidance on home blood-pressure measurement and identify the characteristics that matter when selecting a device, particularly device validation, cuff sizing, measurement technique, consistency, and appropriate device type. | ||||
| 100% | 0% | 0% | 0% | |
How to passIdentify at least six upper-arm monitors from multiple manufacturers. Verify the exact model numbers, cuff sizes, measurement features, memory, connectivity, power options, warranty, and manufacturer-stated specifications. Treat these specifications separately from independent validation evidence. Do not treat an entire product brand as validated; verify the specific model where possible. | ||||
| 100% | 100% | 100% | 100% | |
How to passCheck recognized validation databases and published validation evidence for each exact model. Distinguish between independently validated devices and products that merely use marketing language such as “clinically accurate.” Record the source and status of the validation evidence. | ||||
| 0% | 0% | 0% | 100% | |
How to passResearch independent reviews and user experiences for cuff comfort, setup, display readability, measurement consistency, app reliability, battery behavior, storage, portability, and long-term durability. Look for recurring issues and distinguish them from isolated complaints. | ||||
| 100% | 0% | 0% | 0% | |
How to passSelect three finalists and compare validation evidence, cuff options, usability, data storage, connectivity, warranty, reliability, and total cost. Give the greatest weight to evidence supporting measurement reliability rather than smart features. | ||||
| 100% | 0% | 0% | 100% | |
How to passRecommend the strongest device for general home monitoring and explain why it has the strongest evidence-to-usability balance. Include a checklist for verifying the exact model, cuff size, validation status, warranty, and return policy before purchase. Clearly state that home measurements are for monitoring and do not by themselves establish a diagnosis. | ||||
Failure modes on this task
11 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Reaching the page1 mode | — | — | 2 | 2 |
| Standing up the evidence1 mode | — | 3 | 1 | — |
| Delivering the result1 mode | — | 1 | — | — |
| No signal in the judge’s words1 mode · the grader | 1 | — | 1 | — |