EKWbench
Switch benchEKWbench
Request dataset samplesby

Re-score every trial under a formula of your own

This bench publishes one weighting. Below, the same 72 trials across 6 models are scored again under any other — move the weight between tiers, set a gate on missing a core verifier, and watch the board re-rank. Nothing here changes what this bench publishes, and nothing is simulated: every figure is a re-scoring of trials that actually ran.

0.835–0.980 across tasks
0.020–0.165 across tasks
what a trial keeps after missing a core verifier

As scored. These are the numbers this bench publishes.

Weights are normalised to 100%, so only their ratio matters. Everything below is these 72 trials re-scored — nothing is simulated, and the intervals are recomputed from the same trials.

What does this formula look like?

reward  =  W_core      × (core passed, by weight)
        +  W_secondary × (secondary passed, by weight)

           W is each task's own weights summed by tier — 0.835–0.980 for core.

What does one trial score?

The weights above set how a trial is scored; the boxes below set what is being scored. Both start where this bench actually stands.

Trial outcomes — what the agent did

core12 of 130.835 of the task

secondary4 of 40.165 of the task

Contributions → gate → reward

All-core pass
0%
Verifiers passed
94.1%
Reward
97.5%
  1. 81%core
  2. 16.5%secondary
  3. 97.5%reward

Missed a core verifier. No gate is set, so nothing is taken off.

0.835 · 0.970 + 0.165 · 1.000

Who wins under this formula?

Every model's 12 trials, re-scored. ▲▼ is the move against the ranking this bench publishes; a row set in grey has a gap its own confidence interval cannot resolve.

  • core
  • secondary
  • lost to gate
  • as scored
  1. 1Claude Opus 582.3%±7.4 pts=
  2. 2GPT-5.6 Sol78.8%±8 pts=
  3. 3GPT-5.6 Terra77%±8.2 pts=
  4. 4Claude Sonnet 570.5%±10.5 pts=
  5. 5Gemini 3.6 Flash52.3%±10.4 pts=
  6. 6Gemini 3.1 Pro27.4%±7 pts=
How to read this
What is a tier?
A verifier's importance, set per task by the team that wrote it. This bench uses 2: core and secondary. Every verifier carries a published weight, and a tier's weight is its verifiers' summed — so the sliders move weight between groups of verifiers rather than between counts of them.
Why do the sliders show a range?
Because the weights are per task, not per bench. core spans 0.835–0.980 across this drop's tasks, so no single slider position reproduces the published score — which is why the published setting is a preset ("each task's own weights") rather than a place the sliders can be dragged to. Move one and you have left it; the badge says so, and reset returns you.
What does the gate do?
It is what a trial keeps after missing a core verifier. At 1.00 nothing is taken away, which is where this bench sits — it applies no gate, and its all-core rate is published separately instead. Drag it toward 0 and a trial that missed one core verifier starts losing the rest of what it earned; at exactly 0 the reward becomes the all-core pass rate.
What is the line under the bars?
The arithmetic, written out: each tier's share of the task multiplied by the fraction of that tier the trial cleared, summed, then scaled by the gate. It is the same expression as the formula above, with this trial's numbers in it, so the waterfall and the total can be checked against each other by hand.
Why did it open on this trial?
It is the best-scoring trial that missed a core verifier. The gate does nothing to a trial that cleared every core verifier, so opening on one would leave this page's central control looking broken. Every box starts where that trial actually left it, which is what makes "reset to actual" restore a measurement rather than a default.
Does any of this change the published numbers?
No. The board on the Leaderboard tab is this bench as it was scored, and every figure here is those same trials re-scored under a setting you chose. Nothing is written anywhere, nothing is shared, and reloading the page returns it to the published weighting.