Firmware Patch BackdoorSecurity & Forensics
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one trial to measure.
Core metrics
Avg Cost
Mean API spend across trials; the line under a value is its observed range. N = 8 trials per model.
$0.73$0.62–$0.87
Opus-4.8
$0.27$0.17–$0.44
GPT-5.4
$0.03$0.02–$0.07
Gemini-3
Extended metrics
Median Time
Wall-clock minutes, median across trials. N = 8 trials per model.
4m
Opus-4.8
1m
GPT-5.4
0m
Gemini-3
What the reward cost on this task
Reward vs Cost
Mean reward against mean spend per trial — top-left is more reward for less.
- Opus-4.8
- Gemini-3
- GPT-5.4
$0.03Cost / trial$0.73