I’m interested in understanding how people can researchReference Materials
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
100%
Claude Opus 5.5
67%
GPT-5.6 sol
50%
GPT-6 sol
33%
Claude Opus 5
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
29m
Claude Opus 5
21m
GPT-5.6 sol
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
119
Claude Opus 5
100
Claude Opus 5.5
57
GPT-5.6 sol
38
GPT-6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (6) | ||||
| 100% | 0% | 100% | 0% | |
How to passEstablish objective criteria for evaluating historical webpage evidence: original URL, archived URL, capture date, source archive, page version, visible content, downloadable files, images or embedded resources, redirects, completeness, metadata, current-page comparison, and evidence limitations. Define the difference between a current webpage, an archived capture, a search-engine result, a cached reference, and a secondary description of historical content. Establish when a claim should be labelled "verified," "partially verified," or "not verified." Do not treat a search-result snippet or a current page as proof of what a webpage previously contained. | ||||
| 100% | 100% | 100% | 100% | |
How to passFind at least six publicly accessible authoritative sources explaining web archiving, digital preservation, archived webpage access, or preservation limitations. Prefer recognised web archives, national libraries, university preservation programmes, government digital-preservation projects, archival institutions, and established research organisations. Open the actual guidance or documentation and record what they explicitly state about capture processes, timestamps, missing resources, dynamic content, robots exclusions, preservation scope, and limitations. Clearly distinguish technical documentation from assumptions about how a particular archive captured a page. | ||||
| 100% | 100% | 0% | 0% | |
How to passIdentify at least six public webpages or online documents with meaningful historical changes that can be investigated through accessible archival evidence. Select different types of sources, such as government information, institutional pages, public-service information, product or policy pages, or other durable public resources. For each case, record the original URL, current status, available historical captures, relevant capture dates, and the reason the case is useful for testing historical webpage reconstruction. Do not select cases where the historical content cannot reasonably be investigated. | ||||
| 100% | 0% | 0% | 0% | |
How to passFor at least four selected webpages or online documents, compare two or more accessible historical captures or versions and, where the current page is still available, compare it with the current version as well. Record the capture date, archived URL, original URL, source/archive used, and exactly what changed, such as wording, dates, links, downloadable files, images, navigation, embedded content, or page structure. Distinguish changes directly visible in the archived evidence from changes that can only be inferred. Check whether images, documents, scripts, or other embedded resources were actually preserved and note broken, missing, redirected, or dynamically generated content. Do not treat a later version as proof of what an earlier version contained, and do not assume that an archived capture represents the complete original webpage. Record “not verified” whenever the accessible evidence cannot establish the earlier content or the reason for a change. | ||||
| 100% | 100% | 100% | 100% | |
How to passIdentify at least three cases where historical webpage reconstruction is difficult because of missing captures, broken assets, dynamic content, changed URLs, redirects, or conflicting archival records. Research the documented limitations of the relevant archive or preservation method and explain what can and cannot safely be concluded. Do not infer the reason a capture is missing unless an authoritative source or the archive itself documents it. Preserve uncertainty rather than filling gaps from later copies or memory. | ||||
| 100% | 0% | 100% | 0% | |
How to passResearch publicly accessible accounts from researchers, journalists, archivists, digital-preservation practitioners, and ordinary web users describing practical difficulties with recovering historical web content, including missing captures, broken assets, dynamic pages, changing URLs, redirects, robots exclusions, incomplete archives, and locating older documents or images. Clearly separate these practical experiences from authoritative archival evidence and do not use anecdotes to establish that a historical webpage contained particular information. Use the findings, together with the authoritative guidance and reconstructed webpage cases from earlier steps, to create a practical reusable web-history verification guide covering: identifying the original URL, distinguishing current pages from archived captures, checking capture dates and completeness, comparing versions, verifying embedded resources, handling redirects and missing captures, recording uncertainty, and documenting the evidence trail. Include the exact public URLs for sources used and mark unsupported historical details as “not verified.” If a selected source cannot be accessed publicly, replace it with another qualifying source and record the replacement rather than relying on an inaccessible source or search-result snippet. | ||||
Failure modes on this task
9 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Standing up the evidence2 modes | — | 2 | 2 | 1 |
| Reaching the page1 mode | — | — | — | 2 |
| Delivering the result1 mode | — | 1 | — | 1 |