Kafka Transactional IdDistributed Systems & Databases
How each model performed on this task. Every chart states its N, and the reward carries a 95% confidence interval once there is more than one trial to measure.
Core metrics
Avg Cost
Mean API spend across trials; the line under a value is its observed range. N = 8 trials per model.
$0.59$0.53–$0.66
Fable-5high
$0.40$0.36–$0.46
Opus-4.8
$0.29$0.17–$0.37
Gemini-3
$0.13$0.10–$0.17
GPT-5.4
Extended metrics
Median Time
Wall-clock minutes, median across trials. N = 8 trials per model.
2m
Gemini-3
2m
Opus-4.8
2m
Fable-5high
1m
GPT-5.4
What the reward cost on this task
Reward vs Cost
Mean reward against mean spend per trial — top-left is more reward for less.
- Fable-5
- Opus-4.8
- Gemini-3
- GPT-5.4
$0.13Cost / trial$0.59