Design an indoor air-quality and humidity-control systemHome and Garden
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
20%
Claude Opus 5.5
20%
GPT-5.6 sol
0%
GPT-6 sol
0%
Claude Opus 5
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
74m
Claude Opus 5
7m
GPT-5.6 sol
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
114
Claude Opus 5
100
Claude Opus 5.5
93
GPT-6 sol
25
GPT-5.6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (5) | ||||
| 100% | 0% | 100% | 0% | |
How to passCalculate room volume and use accessible UK public-health, housing or ventilation guidance plus DEFRA UK-AIR to define evidence-based criteria for ventilation, particulate control and relative humidity. Distinguish outdoor-air context from indoor measurements, define the three test scenarios and create pass/fail requirements for clean-air delivery, moisture extraction, noise and monitoring. Avoid medical claims and state when damp, mould or persistent symptoms require landlord or professional action. | ||||
| 0% | 0% | 0% | 0% | |
How to passUse official product, specification, filter, warranty and support pages plus accessible UK retailer listings. Record standard price, dimensions, weight, power, noise, controls, warranty and returns for every device. For purifiers record verified CADR, filter type and replacement price; for dehumidifiers record extraction conditions, tank, humidistat and low-temperature suitability; for monitors record sensors, accuracy where published, display or app requirements and data export. Verify each candidate through its manufacturer and one UK seller, cross-checking one candidate per category with a second seller. | ||||
| 0% | 0% | 0% | 0% | |
How to passCalculate each purifier’s air changes per hour from published CADR and the room volume, without treating nominal room coverage as equivalent. Compare dehumidifier capacity using the manufacturer’s stated test conditions and mark incomparable claims. Model first-year ownership cost from standard delivered price, current filter schedule and electricity using one stated daily-use assumption per scenario and the current Ofgem unit rate. Separate capital, energy and consumable costs, and mark missing specifications as unknown. | ||||
| 0% | 0% | 0% | 0% | |
How to passAssemble Essential, Balanced and Premium systems with exactly one purifier, one dehumidifier and one monitor each. Show itemised standard and delivered cost, first-year cost, footprint, estimated clean-air performance, moisture-control suitability, night noise, sensor coverage, filter availability and major limitations. Do not use sale prices to meet a cap or double-count a combination device’s capability. If a complete safe system misses a target, show the smallest transparent overrun and a compliant phased alternative. | ||||
| 0% | 0% | 0% | 0% | |
How to passCompare the systems using verified room performance, humidity and ventilation control, first-year value, sleep and study usability, and warranty, returns and consumable availability. Deliver a compact nine-device evidence table, calculations, three itemised systems, one recommendation, phased purchase order and a scenario-based operating checklist explaining when to ventilate, purify, dehumidify or do nothing. Include live links, access dates, assumptions, substitutions and a final measurement check before ordering. | ||||
Failure modes on this task
18 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Delivering the result1 mode | 1 | 3 | 3 | 3 |
| Standing up the evidence1 mode | 1 | 2 | 1 | — |
| Reaching the page1 mode | 2 | — | — | 1 |
| No signal in the judge’s words1 mode · the grader | — | — | — | 1 |