Post-closure force-capture exposureCards & Interchange
Start with all trials for auto-written insights, or select a single trial to drill into its verifier results, trajectory, and workspace changes.
GPT-5.5
Mean reward
100%All trials
Across 10 trials on this task, reward ran from 100% to 100% — 10 of 10 cleared every check.
- 7.0 / 7avg checks passed
- 13avg turns
- 16avg tool calls
- 2mavg time
Summary · GPT-5.5
GPT-5.5 cleared every check in all 10 trials.
Common pattern
Every trial cleared the core tier, so there is no failure pattern to read.
Per-trial notes
- Trial 01Passed every verifier it was graded on.
- Trial 02Passed every verifier it was graded on.
- Trial 03Passed every verifier it was graded on.
- Trial 04Passed every verifier it was graded on.
- Trial 05Passed every verifier it was graded on.
- Trial 06Passed every verifier it was graded on.
- Trial 07Passed every verifier it was graded on.
- Trial 08Passed every verifier it was graded on.
- Trial 09Passed every verifier it was graded on.
- Trial 10Passed every verifier it was graded on.
Claude Opus 4.8
Mean reward
88.6%All trials
Across 10 trials on this task, reward ran from 42.9% to 100% — 8 of 10 cleared every check.
- 6.2 / 7avg checks passed
- 9avg turns
- 13avg tool calls
- 1mavg time
Summary · Claude Opus 4.8
Claude Opus 4.8 passed 8 of 10 trials. Failures cluster on cp2_post_closure_capture_usd (2/10 trials).
Common pattern
All 2 unsuccessful trials broke on exactly the same verifiers — cp2_post_closure_capture_usd, cp3_post_closure_capture_count and cp5_accounts_affected and 1 more failed in every one, and nothing else did. That is one reproducible defect, not variance.
Why they broke
Most of Claude Opus 4.8's failures on this task were classified as Wrong answer — 2 verifiers across 2 of 10 trials.
Per-trial notes
- Trial 01Passed every verifier it was graded on.
- Trial 02Missed 4 of 7 checks — cp2_post_closure_capture_usd, cp3_post_closure_capture_count and cp5_accounts_affected and 1 more.
- Trial 03Passed every verifier it was graded on.
- Trial 04Passed every verifier it was graded on.
- Trial 05Passed every verifier it was graded on.
- Trial 06Passed every verifier it was graded on.
- Trial 07Passed every verifier it was graded on.
- Trial 08Passed every verifier it was graded on.
- Trial 09Missed 4 of 7 checks — cp2_post_closure_capture_usd, cp3_post_closure_capture_count and cp5_accounts_affected and 1 more.
- Trial 10Passed every verifier it was graded on.
Gemini 3.1 Pro
Mean reward
87.1%All trials
Across 10 trials on this task, reward ran from 28.6% to 100% — 8 of 10 cleared every check.
- 6.1 / 7avg checks passed
- 41avg turns
- 41avg tool calls
- 21mavg time
Summary · Gemini 3.1 Pro
Gemini 3.1 Pro passed 8 of 10 trials. Failures cluster on cp2_post_closure_capture_usd (2/10 trials).
Common pattern
cp2_post_closure_capture_usd, cp3_post_closure_capture_count and cp5_accounts_affected failed in every unsuccessful trial; what else broke varies across the other 3 verifiers. One reproducible defect, with noise around it.
Why they broke
Most of Gemini 3.1 Pro's failures on this task were classified as Missing field — 2 verifiers across 2 of 10 trials.
Per-trial notes
- Trial 01Passed every verifier it was graded on.
- Trial 02Passed every verifier it was graded on.
- Trial 03Passed every verifier it was graded on.
- Trial 04Missed 5 of 7 checks — cp1_card_processor, cp2_post_closure_capture_usd and cp3_post_closure_capture_count and 2 more.
- Trial 05Passed every verifier it was graded on.
- Trial 06Passed every verifier it was graded on.
- Trial 07Passed every verifier it was graded on.
- Trial 08Passed every verifier it was graded on.
- Trial 09Passed every verifier it was graded on.
- Trial 10Missed 4 of 7 checks — cp2_post_closure_capture_usd, cp3_post_closure_capture_count and cp5_accounts_affected and 1 more.
Gemini 3.5 Flash
Mean reward
40%All trials
Across 10 trials on this task, reward ran from 14.3% to 42.9% — 0 of 10 cleared every check.
- 2.8 / 7avg checks passed
- 46avg turns
- 44avg tool calls
- 25mavg time
Summary · Gemini 3.5 Flash
Gemini 3.5 Flash passed none of its 10 trials. Failures cluster on cp2_post_closure_capture_usd (10/10 trials).
Common pattern
cp2_post_closure_capture_usd, cp3_post_closure_capture_count and cp5_accounts_affected and 1 more failed in every unsuccessful trial; what else broke varies across the other 2 verifiers. One reproducible defect, with noise around it.
Why they broke
Most of Gemini 3.5 Flash's failures on this task were classified as Missing field — 9 verifiers across 9 of 10 trials.
Per-trial notes
- Trial 01Missed 4 of 7 checks — cp2_post_closure_capture_usd, cp3_post_closure_capture_count and cp5_accounts_affected and 1 more.
- Trial 02Missed 4 of 7 checks — cp2_post_closure_capture_usd, cp3_post_closure_capture_count and cp5_accounts_affected and 1 more.
- Trial 03Missed 6 of 7 checks — cp1_card_processor, cp2_post_closure_capture_usd and cp3_post_closure_capture_count and 3 more.
- Trial 04Missed 4 of 7 checks — cp2_post_closure_capture_usd, cp3_post_closure_capture_count and cp5_accounts_affected and 1 more.
- Trial 05Missed 4 of 7 checks — cp2_post_closure_capture_usd, cp3_post_closure_capture_count and cp5_accounts_affected and 1 more.
- Trial 06Missed 4 of 7 checks — cp2_post_closure_capture_usd, cp3_post_closure_capture_count and cp5_accounts_affected and 1 more.
- Trial 07Missed 4 of 7 checks — cp2_post_closure_capture_usd, cp3_post_closure_capture_count and cp5_accounts_affected and 1 more.
- Trial 08Missed 4 of 7 checks — cp2_post_closure_capture_usd, cp3_post_closure_capture_count and cp5_accounts_affected and 1 more.
- Trial 09Missed 4 of 7 checks — cp2_post_closure_capture_usd, cp3_post_closure_capture_count and cp5_accounts_affected and 1 more.
- Trial 10Missed 4 of 7 checks — cp2_post_closure_capture_usd, cp3_post_closure_capture_count and cp5_accounts_affected and 1 more.