EKWbench

Enterprise Knowledge Work

EKWbench evaluates agents on multi-hour operational knowledge work — the sourcing reviews, cost roll-ups, assortment resets and referral triage that keep an enterprise running between its systems of record. Every task hands over the working files the job actually runs on — bills of material, price masters, certification snapshots, signed handoff notes, policy excerpts — and asks for the two artifacts the role really produces: an auditable workbook and a decision memo, at exact paths, inside a four-hour budget. Nothing is graded on style; every verifier is named and graded against an expert value. The full corpus carries 191 expert-built tasks across nine enterprise domains, 103 of them finance and accounting and 187 built around Excel workbooks. The board below measures 69 of them, and published results cover that measured subset rather than the full corpus.

  • Multi-hour tasks
  • Real enterprise workflows
  • Live filesystem
  • Per-verifier grading
  • Claude Code · Codex · Gemini CLI
191
Tasks in licensable corpus
69
Tasks measured
6
Models
4
Trials per task

Task pass rate by model

Full leaderboard

The share of each model's trials in which every core verifier passed.

  • Claude Opus 5 · claude-code · max20.7%
  • GPT-5.6 Sol · codex · max19.6%
  • Claude Sonnet 5 · claude-code · max8.8%
  • GPT-5.6 Terra · codex · max8.3%
  • Gemini 3.6 Flash · gemini-cli · high4.4%
  • Gemini 3.1 Pro · gemini-cli · high0.7%

Measured on 69 tasks · 1,631 trials · 6 models

Grounded in named enterprise occupations

Full taxonomy

Each task represents a specific occupation from the domain-and-subdomain reference and uses the original files associated with that job. The chart groups tasks by domain. Domains that do not yet have any tasks are left out.

Each cell shows a model's performance across all trials for tasks in that domain. The rate is calculated from those individual trials, rather than by averaging the pass rate of each task.

Claude Opus 5
GPT-5.6 Sol
GPT-5.6 Terra
Claude Sonnet 5
Gemini 3.6 Flash
Gemini 3.1 Pro
Finance and insurance23 tasks
73%
71.6%
66.6%
63.2%
49.5%
35.9%
Retail trade13 tasks
71.9%
72.9%
66.5%
64.6%
52.9%
31.3%
Manufacturing9 tasks
72.1%
71.1%
69.6%
63.1%
47.9%
36.6%
Health care and social assistance8 tasks
86.1%
79.8%
76.8%
79.2%
60.7%
42.4%
Real estate and rental and leasing6 tasks
83.9%
78.6%
76.6%
83.9%
62.4%
50.1%
Information5 tasks
78.4%
78.3%
71.4%
68.1%
52.7%
43%
Professional, scientific, and technical services4 tasks
82%
87.3%
86.1%
84.3%
81.9%
57.9%
Government1 task
93.1%
77.8%
70.8%
76.4%
59.7%
41.7%

Covers 8 domains · 57 subdomains

Measured on 69 tasks · 1,631 trials · 6 models

Reward

How reward works

The board ranks models by reward rather than full task passes. Each task has multiple checks, with more important checks carrying more weight. Models receive partial credit for every check they pass, even when they don’t pass the whole task. This means a model can rank highly while fully passing only a small number of tasks.

Claude Opus 5leads the board

Reward
74.4%share of each task’s verifier weight
Verifiers passed
76.4%checks cleared, unweighted

Measured on 69 tasks · 1,631 trials · 6 models

The Verdict

How often a trial clears every core verifier — the strictest reading amongst all. A trial counts here only where every core verifier passed.

Claude Opus 5

20.7%

best model's pass rate

Across 69 tasks attempted 4× each, Claude Opus 5 cleared every core verifier on 20.7% of trials. Individual verifiers mostly pass; it is holding all of them at once — through a four-hour job, across a workbook and the memo that has to justify it — that these tasks make hard.

Failure Modes

Explore failure modes

Every failed verifier in this drop was reviewed and filed under exactly one failure mode, at the earliest point in the pipeline that broke. What follows is the reading those labels support.

  1. Passing most verifiers is not the same as passing a task

    Claude Opus 5 leads on individual verifiers with 76% of them passed, and clears every core verifier on 21% of its trials. The board's headline figure and its pass rate are answering two different questions, and the second is the one a buyer is asking.

  2. Most failures land at Reasoning and Computation

    The inputs are available, but extraction, rule application, selection, calculation, or reconciliation produces the wrong substantive result. It accounts for 4,429 of the drop's 11,391 verifier failures — more than any other stage, and every model on the board is in that number.

  3. The most common single failure is Required Evidence Fields or Rows Omitted

    It accounts for 2,398 of the drop's 11,391 verifier failures, and shows up on 3 of the 3 tasks this drop publishes — so it is a property of the work rather than of one task. From the runs: A reservation table contained client-slot pairs but omitted start time and reserved-minutes fields required to verify the mappings.

  4. How hard an agent works does not predict how well it does

    GPT-5.6 Sol and Gemini 3.6 Flash make almost the same number of tool calls — 47 and 54, at $4.75 and $1.31 — and score 72.5% against 53.7%, 18.8 pts apart. What separates them is what they do with what they read, not how much of it they touch.

The Evidence

Explore all tasks

Each card is one task the whole field attempted, headed by the id the packet carries and opened by the task's own brief. Under it is every model, ranked by the share of that task's verifiers its ten trials cleared — the number that separates this field, since almost no trial clears all of them. Open a card for the full brief, the packet the agent had to read, and how it was graded.