I’m interested in understanding how museums design exhibi…Arts & Entertainment
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
100%
Claude Opus 5
71%
GPT-6 sol
71%
GPT-5.6 sol
57%
Claude Opus 5.5
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
50m
GPT-5.6 sol
20m
Claude Opus 5
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
101
Claude Opus 5
100
Claude Opus 5.5
75
GPT-5.6 sol
46
GPT-6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (7) | ||||
| 100% | 100% | 100% | 100% | |
How to passFind at least six publicly accessible authoritative sources explaining museum accessibility, universal design, inclusive interpretation, accessible exhibitions, visitor services, and communication access. Prioritize government cultural authorities, official museum guidance, recognised museum and accessibility organisations, universities, and established cultural institutions. Define the accessibility dimensions that the evaluation will examine, including physical access and routes, entrances, lifts and stairs, toilets, seating and rest areas, tactile experiences, sensory considerations, audio description, captions, transcripts, sign-language resources, accessible interpretation, assistive equipment, staff support, accessible digital information, advance-planning information, and exhibition-specific provisions. Establish that the framework evaluates documented accessibility provisions and visitor-planning usefulness, not whether a museum can be declared universally accessible. Separate permanent museum-wide provisions from gallery-, exhibition-, programme-, or event-specific provisions. Record “not stated” whenever a source does not explicitly address a requested feature. | ||||
| 0% | 100% | 100% | 100% | |
How to passFind at least six publicly accessible authoritative sources explaining established approaches to museum accessibility, inclusive interpretation, universal design, accessible communication, sensory access, and visitor planning. For each source, record the organisation, title, publication or update date where available, URL, access date, and the specific principle supported. Distinguish formal accessibility standards or institutional requirements from recommended good practice. Do not convert general accessibility guidance into a claim that a particular museum provides a particular feature. | ||||
| 0% | 0% | 0% | 100% | |
How to passSelect at least six museums from different countries or regions and, where possible, different types of institution, such as art, history, science, natural history, cultural heritage, or historic-house museums. For each museum, open and review the actual official accessibility, visitor-information, exhibition, programme, and advance-planning pages rather than relying on search-result snippets. Record evidence for physical access, routes, lifts, seating, toilets, tactile or multisensory resources, audio description, captions, transcripts, sign-language resources, accessible interpretation, assistive equipment, communication support, staff assistance, sensory information, accessible digital information, and advance-booking or planning requirements. Clearly identify whether each feature is museum-wide, gallery-specific, exhibition-specific, programme-specific, available only at certain times, available by request, or not stated. Do not infer accessibility from the museum's size, reputation, disability statement, or provision of a single access feature. | ||||
| 100% | 100% | 100% | 100% | |
How to passFor at least four selected museums, compare their general accessibility information with specific gallery, exhibition, event, or visitor-service pages. Determine whether the information is consistent and whether visitors can realistically discover the information needed before arriving. Check for details such as accessible entrances, step-free routes, lift access, seating, exhibition-specific restrictions, captioning, audio description, tactile resources, sensory information, booking requirements, assistance availability, and temporary changes to access. Where the website does not provide enough detail to establish whether a feature exists, record “not stated” rather than assuming it is available. Distinguish between a museum saying that it provides accessibility support generally and an individual exhibition explicitly documenting that support. | ||||
| 100% | 100% | 100% | 100% | |
How to passIdentify at least three museums or heritage institutions operating in historic or architecturally constrained buildings and investigate their documented accessibility challenges and responses. Examine official access guides, visitor information, heritage documentation, accessibility policies, and exhibition information for evidence about steps, narrow routes, lifts, uneven floors, protected structures, alternative entrances, ramps, platform lifts, accessible routes, digital or tactile alternatives, seating, staff assistance, and other adaptations. Clearly distinguish documented physical constraints from assumptions about historic buildings. Evaluate whether the institution provides an alternative way to access information or experience an exhibition where a particular physical route or feature cannot be made fully accessible. Do not treat historic status as automatically preventing accessibility improvements. | ||||
| 100% | 100% | 100% | 100% | |
How to passSearch publicly readable discussions from disabled visitors, accessibility advocates, museum professionals, disability organisations, travel communities, and cultural-accessibility discussions for practical experiences involving inaccessible routes, unclear website information, staff assistance, tactile resources, sensory barriers, captions, audio description, communication support, and differences between advertised and experienced access. Treat these sources as reported experiences rather than verified institutional facts. Record the original URL and distinguish individual experiences from recurring patterns supported by multiple independent accounts. Do not use community reports to prove that a museum officially provides or does not provide a feature unless the museum's own documentation supports that conclusion. | ||||
| 0% | 0% | 0% | 100% | |
How to passCreate a practical evaluation guide covering at least six museums and three historic-building examples. Organise the guide around the visitor's planning needs rather than a single accessibility score. For each museum, record documented provisions for physical access, navigation, seating, tactile and sensory experiences, audio description, captions, sign-language resources, accessible interpretation, assistive equipment, staff support, digital accessibility information, advance-planning requirements, and exhibition-specific limitations. For every feature, identify the evidence source, date checked, scope of availability, booking or request requirements, and whether the information is verified, unclear, unavailable, or “not stated.” Include a separate section for disabled-visitor/community experiences and clearly label those as experiential evidence. Provide a practical pre-visit checklist that helps a visitor identify what they need, find the relevant official information, recognise gaps, contact the museum where necessary, and avoid assuming that a general accessibility statement applies to every exhibition. Do not assign a universal “fully accessible” or “inaccessible” label based on a small number of features. | ||||
Failure modes on this task
7 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Standing up the evidence1 mode | 1 | 1 | 1 | — |
| Delivering the result1 mode | 1 | 1 | — | — |
| No signal in the judge’s words1 mode · the grader | 1 | — | 1 | — |