I want to plan an information hub butScience and Education
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
100%
Claude Opus 5.5
93%
GPT-5.6 sol
67%
GPT-6 sol
20%
Claude Opus 5
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
57m
GPT-5.6 sol
47m
Claude Opus 5
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
119
Claude Opus 5
100
Claude Opus 5.5
76
GPT-6 sol
49
GPT-5.6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (15) | ||||
| 100% | 100% | 100% | 100% | |
How to passOpen https://cryptpad.fr/. Create one spreadsheet for the page audit and one document for the plan. In the document, add sections for standards, repeated patterns, component scoring, site map, templates, prototype tests, recommendation and handoff. | ||||
| 100% | 100% | 100% | 0% | |
How to passStart at https://science.nasa.gov/. Choose exactly four current English science-topic landing pages, four mission profile pages and four science articles that each have at least four body section headings. In the CryptPad spreadsheet, add one row per page with its exact title, plain URL, page type and access date. | ||||
| 100% | 0% | 100% | 0% | |
How to passOpen every selected page in a signed-out browser. Confirm that there are 12 different working pages and exactly four of each type. Replace any blocked, duplicate, translated or wrongly classified page with another page of the same type, and record the replacement in the spreadsheet. | ||||
| 100% | 100% | 100% | 100% | |
How to passRead https://www.w3.org/WAI/WCAG22/quickref/, https://www.w3.org/WAI/tutorials/images/complex/ and https://designsystem.digital.gov/components/. Add spreadsheet columns for navigation and breadcrumbs, title and summary, hero media, long-page aids, image alternative text and credits, data displays, related links, mobile layout, useful pattern and problem found. | ||||
| 100% | 0% | 100% | 0% | |
How to passReview the four topic landing pages one at a time. Complete every audit column for each row and add a short page-based reason for every yes, no or partial result. Record the exact section or element that supports each result. | ||||
| 100% | 100% | 100% | 100% | |
How to passReview the four mission profile pages with the same columns and evidence rules. Keep the wording and rating method the same as the topic-page audit so the rows can be compared. | ||||
| 100% | 100% | 100% | 0% | |
How to passReview the four long science articles with the same columns and evidence rules. Complete every cell and do not score a feature from the page title or search snippet alone. | ||||
| 100% | 100% | 100% | 0% | |
How to passIn the spreadsheet, count how many of the 12 pages use each audited pattern. In the repeated-patterns section of the CryptPad document, record the counts, the three strongest repeated patterns and the three most common problems, with links to example pages. | ||||
| 100% | 100% | 100% | 0% | |
How to passUse the audit results and the U.S. Web Design System page at https://designsystem.digital.gov/components/ to create a pool of exactly 10 reusable components in the planning document. For each component, record its purpose, the NASA page evidence and the matching USWDS guidance if one exists. | ||||
| 100% | 100% | 100% | 0% | |
How to passIn the component-scoring section of the CryptPad document, make a table with one row per candidate. Use these weights: accessibility 25, content clarity 20, reuse across page types 20, mobile use 15, build effort 10 and visual consistency 10. Give source-based reasons, calculate totals and choose exactly six components. If totals are equal, place the higher accessibility score first. | ||||
| 100% | 100% | 100% | 0% | |
How to passIn the site-map section of CryptPad, plan exactly eight pages: one home page, three topic pages, two mission pages and two article pages. Give every page a working title, purpose, main audience need and parent page. Show the navigation order and explain one planning decision. | ||||
| 100% | 0% | 0% | 0% | |
How to passIn the templates section of CryptPad, make one table for the topic, mission and article templates. Show the section order, which of the six chosen components each template uses, the mobile change and the text alternative for important visual content. | ||||
| 100% | 100% | 100% | 0% | |
How to passOpen https://jsfiddle.net/ or another free public HTML, CSS and JavaScript editor. Build one responsive prototype with three template views and a control for switching between them. Use all six chosen components, short mock text and image placeholders. Record the editor and preview URLs in CryptPad. | ||||
| 100% | 0% | 100% | 0% | |
How to passIn the prototype-tests section of CryptPad, make a table with columns for test, expected result, actual result and fix made. Test the three view changes, all links and controls, keyboard use, visible focus, meaning without colour, 360, 768 and 1280 pixel widths and browser-console errors. Fix every problem and complete every row. | ||||
| 100% | 0% | 100% | 0% | |
How to passOpen all four URLs in a signed-out browser and verify that each resolves to the intended CryptPad spreadsheet, CryptPad document, code editor, or prototype preview, is accessible without an account/login, and remains usable as a durable review link. Record the verification result for each URL. | ||||
Failure modes on this task
18 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Delivering the result1 mode | — | 1 | 1 | 5 |
| Standing up the evidence1 mode | — | 2 | — | 1 |
| Reaching the page1 mode | — | 2 | — | — |
| No signal in the judge’s words1 mode · the grader | — | — | — | 6 |