Each model ran each task once, in a fresh browser session, acting through the mouse and keyboard, and an LLM judge graded every run rubric by rubric. The headline is the task pass rate: the share of tasks where the run met every rubric — a task with no run counts as a miss. Beside it, the rubric score gives partial credit: the share of a task's rubrics the judge found satisfied in the run's recorded actions and screenshots, averaged over tasks. A rubric the judge could not grade is left out of the score rather than counted against the agent. Ranked across4 models.
Last updated 2026-09-28
Total Tasks
100
each attempted 1× by 4 models
Best Pass Rate
41%
Claude Opus 5.5
Model Scorecard · open a row for its model cardtask pass rateRubric scoreAvg Steps
1Claude Opus 5.541%100 runs82.6%56
Model card · evaluated Sep 2026
100 runs
Task pass rate
41%
Rubric score
82.6%
Failure Modes
Standing up the evidence50%Standing up the evidence55 of 111 rubric failures
Delivering the result41%Delivering the result45 of 111 rubric failures
Reaching the page10%Reaching the page11 of 111 rubric failures
Every rubric failure was filed under one failure mode by a stated rule, not by a reviewer, grouped into the stages above. The shares split this model's own 111 rubric failures, worst first — grader error is excluded, so a model is never charged with its grader's mistake.
Pass rate by taxonomy subdomain
News & Media Publishers100%News & Media Publishers100% of 5 runs passed
Games60%Games60% of 5 runs passed
Health50%Health50% of 6 runs passed
Heavy Industry and Engineering50%Heavy Industry and Engineering50% of 2 runs passed
Sports50%Sports50% of 6 runs passed
Food and Drink42.9%Food and Drink42.9% of 7 runs passed
Jobs and Career42.9%Jobs and Career42.9% of 7 runs passed
Reference Materials41.7%Reference Materials41.7% of 12 runs passed
Science and Education41.2%Science and Education41.2% of 17 runs passed
Business and Consumer Services40%Business and Consumer Services40% of 10 runs passed
Law and Government40%Law and Government40% of 15 runs passed
Travel and Tourism40%Travel and Tourism40% of 10 runs passed
Finance37.5%Finance37.5% of 8 runs passed
Home and Garden37.5%Home and Garden37.5% of 8 runs passed
Computers Electronics and Technology36.4%Computers Electronics and Technology36.4% of 11 runs passed
Ecommerce & Shopping35.7%Ecommerce & Shopping35.7% of 14 runs passed
Vehicles33.3%Vehicles33.3% of 6 runs passed
Community and Society20%Community and Society20% of 5 runs passed
Hobbies and Leisure16.7%Hobbies and Leisure16.7% of 6 runs passed
Arts & Entertainment12.5%Arts & Entertainment12.5% of 8 runs passed
Lifestyle0%Lifestyle0% of 2 runs passed
The share of each subdomain’s rubrics this model’s runs cleared, its tasks’ runs added together.
2GPT-6 sol20%100 runs58.4%42
Model card · evaluated Sep 2026
100 runs
Task pass rate
20%
Rubric score
58.4%
Failure Modes
Delivering the result71%Delivering the result197 of 279 rubric failures
Standing up the evidence24%Standing up the evidence66 of 279 rubric failures
Reaching the page6%Reaching the page16 of 279 rubric failures
Every rubric failure was filed under one failure mode by a stated rule, not by a reviewer, grouped into the stages above. The shares split this model's own 279 rubric failures, worst first — grader error is excluded, so a model is never charged with its grader's mistake.
Pass rate by taxonomy subdomain
News & Media Publishers60%News & Media Publishers60% of 5 runs passed
Heavy Industry and Engineering50%Heavy Industry and Engineering50% of 2 runs passed
Hobbies and Leisure33.3%Hobbies and Leisure33.3% of 6 runs passed
Sports33.3%Sports33.3% of 6 runs passed
Food and Drink28.6%Food and Drink28.6% of 7 runs passed
Home and Garden25%Home and Garden25% of 8 runs passed
Community and Society20%Community and Society20% of 5 runs passed
Games20%Games20% of 5 runs passed
Jobs and Career14.3%Jobs and Career14.3% of 7 runs passed
Arts & Entertainment12.5%Arts & Entertainment12.5% of 8 runs passed
Finance12.5%Finance12.5% of 8 runs passed
Business and Consumer Services10%Business and Consumer Services10% of 10 runs passed
Travel and Tourism10%Travel and Tourism10% of 10 runs passed
Computers Electronics and Technology9.1%Computers Electronics and Technology9.1% of 11 runs passed
Reference Materials8.3%Reference Materials8.3% of 12 runs passed
Ecommerce & Shopping7.1%Ecommerce & Shopping7.1% of 14 runs passed
Law and Government6.7%Law and Government6.7% of 15 runs passed
Science and Education5.9%Science and Education5.9% of 17 runs passed
Health0%Health0% of 6 runs passed
Lifestyle0%Lifestyle0% of 2 runs passed
Vehicles0%Vehicles0% of 6 runs passed
The share of each subdomain’s rubrics this model’s runs cleared, its tasks’ runs added together.
3GPT-5.6 sol18%99 runs57.5%33
Model card · evaluated Sep 2026
99 runs
Task pass rate
18%
Rubric score
57.5%
Failure Modes
Delivering the result63%Delivering the result152 of 243 rubric failures
Standing up the evidence29%Standing up the evidence71 of 243 rubric failures
Reaching the page8%Reaching the page20 of 243 rubric failures
Every rubric failure was filed under one failure mode by a stated rule, not by a reviewer, grouped into the stages above. The shares split this model's own 243 rubric failures, worst first — grader error is excluded, so a model is never charged with its grader's mistake.
Pass rate by taxonomy subdomain
News & Media Publishers60%News & Media Publishers60% of 5 runs passed
Food and Drink28.6%Food and Drink28.6% of 7 runs passed
Law and Government28.6%Law and Government28.6% of 14 runs passed
Finance25%Finance25% of 8 runs passed
Home and Garden25%Home and Garden25% of 8 runs passed
Community and Society20%Community and Society20% of 5 runs passed
Games20%Games20% of 5 runs passed
Computers Electronics and Technology18.2%Computers Electronics and Technology18.2% of 11 runs passed
Hobbies and Leisure16.7%Hobbies and Leisure16.7% of 6 runs passed
Reference Materials16.7%Reference Materials16.7% of 12 runs passed
Sports16.7%Sports16.7% of 6 runs passed
Vehicles16.7%Vehicles16.7% of 6 runs passed
Ecommerce & Shopping14.3%Ecommerce & Shopping14.3% of 14 runs passed
Travel and Tourism10%Travel and Tourism10% of 10 runs passed
Science and Education6.3%Science and Education6.3% of 16 runs passed
Arts & Entertainment0%Arts & Entertainment0% of 8 runs passed
Business and Consumer Services0%Business and Consumer Services0% of 10 runs passed
Health0%Health0% of 6 runs passed
Heavy Industry and Engineering0%Heavy Industry and Engineering0% of 2 runs passed
Jobs and Career0%Jobs and Career0% of 6 runs passed
Lifestyle0%Lifestyle0% of 2 runs passed
The share of each subdomain’s rubrics this model’s runs cleared, its tasks’ runs added together.
4Claude Opus 515%100 runs55.4%101
Model card · evaluated Sep 2026
100 runs
Task pass rate
15%
Rubric score
55.4%
Failure Modes
Delivering the result58%Delivering the result178 of 308 rubric failures
Standing up the evidence34%Standing up the evidence106 of 308 rubric failures
Reaching the page8%Reaching the page24 of 308 rubric failures
Every rubric failure was filed under one failure mode by a stated rule, not by a reviewer, grouped into the stages above. The shares split this model's own 308 rubric failures, worst first — grader error is excluded, so a model is never charged with its grader's mistake.
Pass rate by taxonomy subdomain
Travel and Tourism30%Travel and Tourism30% of 10 runs passed
Ecommerce & Shopping28.6%Ecommerce & Shopping28.6% of 14 runs passed
Jobs and Career28.6%Jobs and Career28.6% of 7 runs passed
Finance25%Finance25% of 8 runs passed
Reference Materials25%Reference Materials25% of 12 runs passed
Business and Consumer Services20%Business and Consumer Services20% of 10 runs passed
Community and Society20%Community and Society20% of 5 runs passed
Games20%Games20% of 5 runs passed
Law and Government20%Law and Government20% of 15 runs passed
Science and Education17.6%Science and Education17.6% of 17 runs passed
Health16.7%Health16.7% of 6 runs passed
Sports16.7%Sports16.7% of 6 runs passed
Vehicles16.7%Vehicles16.7% of 6 runs passed
Food and Drink14.3%Food and Drink14.3% of 7 runs passed
Arts & Entertainment12.5%Arts & Entertainment12.5% of 8 runs passed
Home and Garden12.5%Home and Garden12.5% of 8 runs passed
Computers Electronics and Technology0%Computers Electronics and Technology0% of 11 runs passed
Heavy Industry and Engineering0%Heavy Industry and Engineering0% of 2 runs passed
Hobbies and Leisure0%Hobbies and Leisure0% of 6 runs passed
Lifestyle0%Lifestyle0% of 2 runs passed
News & Media Publishers0%News & Media Publishers0% of 5 runs passed
The share of each subdomain’s rubrics this model’s runs cleared, its tasks’ runs added together.
Benchmark Scope
What the tasks cover. Each task is counted once, under the category of the first subject its author filed it under; the Taxonomy tab counts every subject a task declares.
Task Category Coverage
How the task set distributes across Apollo's subject categories
Computers Electronics and Technology
7 of 100 tasks
Business and Consumer Services
6 of 100 tasks
Science and Education
6 of 100 tasks
Travel and Tourism
6 of 100 tasks
Arts & Entertainment
5 of 100 tasks
Community and Society
5 of 100 tasks
Ecommerce & Shopping
5 of 100 tasks
Food and Drink
5 of 100 tasks
Games
5 of 100 tasks
Health
5 of 100 tasks
Hobbies and Leisure
5 of 100 tasks
Home and Garden
5 of 100 tasks
Jobs and Career
5 of 100 tasks
Law and Government
5 of 100 tasks
News & Media Publishers
5 of 100 tasks
Reference Materials
5 of 100 tasks
Sports
5 of 100 tasks
Vehicles
5 of 100 tasks
Finance
4 of 100 tasks
Heavy Industry and Engineering
1 of 100 tasks
Aggregate Metrics
One bar per model, over every task. The task pass rate counts a task only where the run met every rubric; the rubric score gives partial credit, so the distance between them is the work runs left partly done.
Task Pass Rate
Share of tasks where the run met every rubric — a task with no run counts as a miss
100%0%
41%
Claude Opus 5.5
20%
GPT-6 sol
18%
GPT-5.6 sol
15%
Claude Opus 5
Rubric Score
Share of a task's rubrics the judge found satisfied, averaged across tasks — a task with no run counts as zero
100%0%
82.6%
Claude Opus 5.5
58.4%
GPT-6 sol
57.5%
GPT-5.6 sol
55.4%
Claude Opus 5
Category Difficulty
The rubric score per category. Here a task counts under every category it declares, as on the Taxonomy tab, and a category holds a handful of tasks — read a cell as where a model struggled, not as a rate to compare to the point.
Rubric Score — Category × Model
Rubric score per task category, over the runs on every task filed under it. Colour bands the value; every cell also prints it.
Claude Opus 5.5
GPT-6 sol
GPT-5.6 sol
Claude Opus 5
Health
68.6%
41.4%
47%
52.9%
Community and Society
69.4%
41.7%
69.1%
33.5%
Travel and Tourism
62.6%
44.3%
45.8%
62.5%
Computers Electronics and Technology
81.4%
50.1%
62.2%
33.4%
Lifestyle
67.9%
60.7%
45.2%
54.8%
Home and Garden
81.5%
41.9%
61.2%
59%
Business and Consumer Services
84.2%
58.6%
45.3%
59.6%
Science and Education
82.5%
49.1%
53.3%
64.4%
Arts & Entertainment
81.3%
64.3%
51.8%
54.5%
Sports
83.6%
63.3%
44.4%
66.1%
Food and Drink
81.3%
68.2%
61.4%
46.8%
Vehicles
84.5%
67.1%
56.9%
51.6%
Finance
80%
65.2%
59.2%
58.8%
Ecommerce & Shopping
86.6%
49.5%
57.8%
72.7%
Hobbies and Leisure
82.2%
59.8%
57.7%
67.8%
Law and Government
87.6%
61.8%
63.1%
55.7%
Reference Materials
89.8%
60.3%
60.8%
66.1%
Games
90.8%
61.9%
60.6%
68.5%
Jobs and Career
87.1%
59.3%
56.6%
80.4%
Heavy Industry and Engineering
93.8%
75%
75%
43.8%
News & Media Publishers
100%
75%
81.3%
37.7%
Domains at or above the mean
21/21
6/21
5/21
7/21
0%100%
Run Effort
What a run spent. Every step is one screenshot and at most one mouse or keyboard action, and a run stops when it reaches its step budget, whatever it had left to do.
Avg Steps
Steps a run took, averaged across runs
1010
101
Claude Opus 5
56
Claude Opus 5.5
42
GPT-6 sol
33
GPT-5.6 sol
Avg Actions
Mouse and keyboard actions a run took, averaged across runs
