Transfer balance NSF chainPayments & Transfers
Start with all trials for auto-written insights, or select a single trial to drill into its verifier results, trajectory, and workspace changes.
GPT-5.5
Mean reward
50%All trials
Across 10 trials on this task, reward ran from 0% to 100% — 5 of 10 cleared every check.
- 0.5 / 1avg checks passed
- 13avg turns
- 13avg tool calls
- 1mavg time
Summary · GPT-5.5
GPT-5.5 passed 5 of 10 trials. Failures cluster on gold_account_funding_self_healing_retry_failed_again (5/10 trials).
Common pattern
All 5 unsuccessful trials broke on exactly the same verifiers — gold_account_funding_self_healing_retry_failed_again failed in every one, and nothing else did. That is one reproducible defect, not variance.
Why they broke
Most of GPT-5.5's failures on this task were classified as Wrong answer — 5 verifiers across 5 of 10 trials.
Per-trial notes
- Trial 01Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 02Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 03Passed every verifier it was graded on.
- Trial 04Passed every verifier it was graded on.
- Trial 05Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 06Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 07Passed every verifier it was graded on.
- Trial 08Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 09Passed every verifier it was graded on.
- Trial 10Passed every verifier it was graded on.
Claude Opus 4.8
Mean reward
20%All trials
Across 10 trials on this task, reward ran from 0% to 100% — 2 of 10 cleared every check.
- 0.2 / 1avg checks passed
- 13avg turns
- 14avg tool calls
- 1mavg time
Summary · Claude Opus 4.8
Claude Opus 4.8 passed 2 of 10 trials. Failures cluster on gold_account_funding_self_healing_retry_failed_again (8/10 trials).
Common pattern
All 8 unsuccessful trials broke on exactly the same verifiers — gold_account_funding_self_healing_retry_failed_again failed in every one, and nothing else did. That is one reproducible defect, not variance.
Why they broke
Most of Claude Opus 4.8's failures on this task were classified as Wrong answer — 8 verifiers across 8 of 10 trials.
Per-trial notes
- Trial 01Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 02Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 03Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 04Passed every verifier it was graded on.
- Trial 05Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 06Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 07Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 08Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 09Passed every verifier it was graded on.
- Trial 10Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
Gemini 3.1 Pro
Mean reward
50%All trials
Across 10 trials on this task, reward ran from 0% to 100% — 5 of 10 cleared every check.
- 0.5 / 1avg checks passed
- 28avg turns
- 26avg tool calls
- 2mavg time
Summary · Gemini 3.1 Pro
Gemini 3.1 Pro passed 5 of 10 trials. Failures cluster on gold_account_funding_self_healing_retry_failed_again (5/10 trials).
Common pattern
All 5 unsuccessful trials broke on exactly the same verifiers — gold_account_funding_self_healing_retry_failed_again failed in every one, and nothing else did. That is one reproducible defect, not variance.
Why they broke
Most of Gemini 3.1 Pro's failures on this task were classified as Near miss — 4 verifiers across 4 of 10 trials.
Per-trial notes
- Trial 01Passed every verifier it was graded on.
- Trial 02Passed every verifier it was graded on.
- Trial 03Passed every verifier it was graded on.
- Trial 04Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 05Passed every verifier it was graded on.
- Trial 06Passed every verifier it was graded on.
- Trial 07Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 08Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 09Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 10Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
Gemini 3.5 Flash
Mean reward
0%All trials
Across 10 trials on this task, reward ran from 0% to 0% — 0 of 10 cleared every check.
- 0.0 / 1avg checks passed
- 68avg turns
- 66avg tool calls
- 40mavg time
Summary · Gemini 3.5 Flash
Gemini 3.5 Flash passed none of its 10 trials. Failures cluster on gold_account_funding_self_healing_retry_failed_again (10/10 trials).
Common pattern
All 10 unsuccessful trials broke on exactly the same verifiers — gold_account_funding_self_healing_retry_failed_again failed in every one, and nothing else did. That is one reproducible defect, not variance.
Why they broke
Most of Gemini 3.5 Flash's failures on this task were classified as Wrong answer — 6 verifiers across 6 of 10 trials.
Per-trial notes
- Trial 01Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 02Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 03Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 04Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 05Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 06Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 07Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 08Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 09Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.
- Trial 10Missed 1 of 1 check — gold_account_funding_self_healing_retry_failed_again.