I want to get into classic literature butArts & Entertainment
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one run to measure.
Core metrics
Check Pass %
Mean share of the task's checks passing, averaged over runs. N = 1 run per model.
86%
Claude Opus 5.5
86%
Claude Opus 5
71%
GPT-6 sol
71%
GPT-5.6 sol
Extended metrics
Median Time
Wall-clock minutes, median across runs. N = 1 run per model.
22m
Claude Opus 5
2m
GPT-5.6 sol
Tool Calls
Tool invocations, median across runs — counted from the trajectories. N = 1 run per model.
74
Claude Opus 5
20
GPT-6 sol
15
Claude Opus 5.5
10
GPT-5.6 sol
Verifier × model pass rate
How often each verifier passed, per model. Green is passing, red is failing — a red band across a row is a verifier nothing clears.
| Verifier | Claude Opus 5.5n=1 | GPT-6 soln=1 | GPT-5.6 soln=1 | Claude Opus 5n=1 |
|---|---|---|---|---|
| checks (7) | ||||
| 100% | 100% | 100% | 100% | |
How to passOpen Project Gutenberg and navigate to its Classics of Literature collection. Use this collection as the primary source for selecting the books. Project Gutenberg provides free online access to thousands of older works whose U.S. copyright has expired. | ||||
| 0% | 100% | 100% | 0% | |
How to passBrowse the classics collection and identify at least eight potential books that could reasonably work for someone reading classic literature for the first time. Consider books from different authors and genres rather than selecting several books by the same author. | ||||
| 100% | 100% | 100% | 100% | |
How to passOpen the individual Project Gutenberg pages for the selected candidates and confirm that each full work can be read online. Record the title, author, publication year (where provided), genre or subject classification, and the available online-reading option. | ||||
| 100% | 100% | 100% | 100% | |
How to passChoose the five books that appear most approachable for a beginner. Consider factors such as the apparent length of the work, narrative complexity, writing style, subject matter, and accessibility of the story. | ||||
| 100% | 0% | 0% | 100% | |
How to passRank the five selected books from 1 to 5, with #1 being the strongest starting point for a newcomer to classic literature. Explain the reasoning behind each position rather than relying solely on Project Gutenberg's popularity or download figures. | ||||
| 100% | 0% | 0% | 100% | |
How to passProvide a ranked list titled "Classic Literature Starter List". For each book, include its title, author, publication year, genre, a brief spoiler-free description, and the reason for its position in the ranking. Clearly identify the best book to start with and provide its Project Gutenberg reading link. | ||||
| 100% | 100% | 100% | 100% | |
How to passReopen the five selected Project Gutenberg pages and check that the titles, authors, publication information, and online-reading availability in the final answer match the source. Do not purchase, download, or sign into any service. | ||||
Failure modes on this task
6 tagged verifier failures, by the stage they were assigned to. Higher is worse.
| Failure stage | Claude Opus 5.5 | GPT-6 sol | GPT-5.6 sol | Claude Opus 5 |
|---|---|---|---|---|
| Standing up the evidence1 mode | 1 | — | 1 | 1 |
| Delivering the result1 mode | — | 2 | 1 | — |