Terminal-Bench 3.0
Switch benchTerminal-Bench 3.0
Request dataset samplesby

Edl Drop FrameMedia & Graphics

Human baseline2.5h

TaxonomyMedia & GraphicsMedia & Graphics

Start with all trials for auto-written insights, or select a single trial to read its full trajectory.

Fable-5claude-code · high

All trials8 trialsTrial 01100%Trial 02100%Trial 03100%Trial 04100%Trial 05100%Trial 060%Trial 07100%Trial 080%

Mean reward

75%

All trials

Across 8 trials on this task, reward ran from 0% to 100% — 6 of 8 passed.

  • $0.93avg cost
  • 3mavg time

Summary · Fable-5

Fable-5 passed 6 of 8 trials.

Common pattern

6 of 8 trials passed on identical attempts: the capability is there, the reliability is not.

Opus-4.8claude-code

All trials8 trialsTrial 010%Trial 020%Trial 030%Trial 040%Trial 050%Trial 060%Trial 070%Trial 08100%

Mean reward

12.5%

All trials

Across 8 trials on this task, reward ran from 0% to 100% — 1 of 8 passed.

  • $0.73avg cost
  • 4mavg time

Summary · Opus-4.8

Opus-4.8 passed 1 of 8 trials.

Common pattern

1 of 8 trials passed on identical attempts: the capability is there, the reliability is not.

Gemini-3gemini-cli

All trials8 trialsTrial 010%Trial 020%Trial 03100%Trial 040%Trial 05100%Trial 060%Trial 07100%Trial 08100%

Mean reward

50%

All trials

Across 8 trials on this task, reward ran from 0% to 100% — 4 of 8 passed.

  • $0.59avg cost
  • 4mavg time

Summary · Gemini-3

Gemini-3 passed 4 of 8 trials.

Common pattern

4 of 8 trials passed on identical attempts: the capability is there, the reliability is not.

GPT-5.4codex

All trials8 trialsTrial 01trace0%Trial 020%Trial 030%Trial 040%Trial 050%Trial 060%Trial 070%Trial 080%

Mean reward

0%

All trials

Across 8 trials on this task, reward ran from 0% to 0% — 0 of 8 passed.

  • 36avg turns
  • $0.27avg cost
  • 2mavg time

Summary · GPT-5.4

GPT-5.4 passed none of its 8 trials.

Common pattern

No trial passed. Whatever this task demands, the model misses it every time — a reproducible miss, not variance.

Published trajectory

The drop publishes a full step record for one of the trials above — every other attempt shipped its score without its route.

GPT-5.4Trial 01Fail36 steps
The agent's last word

“Created `/logs/artifacts/re_cut.mp4`. Verification: - Video: `480x270`, `30000/1001` fps, `870` frames - Audio/video duration: both `29.029000` s - Audio sample duration: `1,393,392` samples at `48 kHz`, matching the cut length exactly - Cut-frame checks: output frames `0`, `300`, and `420` checksum-match master frames `150`, `1740`, and `4500`, so the segment starts land on the intended physical frames despite the DF/NDF mismatch The EDL was resolved as non-drop frame counts: - `00:00:05:00` to `00:00:15:00` -> frames `150` to `450` - `00:00:58:00` to `00:01:02:00` -> frames `1740` to `1860` - `00:02:30:00` to `00:02:45:00` -> frames `4500` to `4950`”

The full trajectory
  1. system

    <permissions instructions> Filesystem sandboxing defines which files can be read or written. `sandbox_mode` is `danger-full-access`: No filesystem sandboxing - all commands are permitted. Network access is enabled. Approval policy is currently never. Do not provide the `sandbox_permissions` for any reason, commands will be rejected. </permissions instructions><skills_instructions> ## Skills A skill is a set of instructions provided through a `SKILL.md` source. Below is the list of skills that can be used. Each entry includes a name, description, and source locator. `file` locators are on the host filesystem, `environment resource` locators are owned by an execution environment, `orchestrator resource` locators are opaque non-filesystem resources, and `custom resource` locators use their provider's access mechanism. ### Available skills - imagegen: Generate or edit raster images when the task benefits from AI-created bitmap visuals such as photos, illustrations, textures, sprites, mockups, or transparent-background cutouts. Use when Codex should create a brand-new image, transform an existing image, or derive visual variants from references, and the output should be a bitmap asset rather than repo-native code or vector. Do not use when the task is better handled by editing existing SVG/vector/code-native assets, extending an established icon or logo system, or building the visual directly in HTML/CSS/canvas. (file: /tmp/codex-home/skills/.system/imagegen/SKILL.md) - openai-docs: Use when the user asks how to build with OpenAI products or APIs, asks about Codex itself or choosing Codex surfaces, needs up-to-date official documentation with citations, help choosing the latest model for a use case, latest/current/default-model prompting guidance, or model upgrade and prompt-upgrade guidance; use OpenAI docs MCP tools for non-Codex docs questions, use the Codex manual helper first for broad Codex self-knowledge, and restrict fallback browsing to official OpenAI domains.

  2. user

    <environment_context> <cwd>/app</cwd> <shell>bash</shell> <current_date>2026-07-23</current_date> <timezone>Etc/UTC</timezone> <filesystem><workspace_roots><root>/app</root></workspace_roots><permission_profile type="disabled"><file_system type="unrestricted" /></permission_profile></filesystem> </environment_context>

  3. user

    # EDL Drop-Frame Recut You are given a 29.97 fps drop-frame master video (`/app/master.mov`) and an edit decision list (`/app/edit.edl`). The EDL lists source in/out points using non-drop-frame timecode (counting frames as if the video were exactly 30 fps). The master uses drop-frame timecode, which skips frame numbers `00` and `01` at the top of every minute except multiples of ten. Produce `/logs/artifacts/re_cut.mp4` containing the concatenation of the source clips described by the EDL, with every edit landing on the exact physical frame it refers to in the master. The output must preserve A/V sync and match the master's frame rate and resolution. The EDL file is plain text. Lines beginning with `#` are comments. Each edit event line has the form: ``` <event> <source_in> <source_out> ``` Timecodes are `HH:MM:SS:FF` with two-digit frames. Requirements: - Output file: `/logs/artifacts/re_cut.mp4` - Frame rate: `30000/1001` fps (≈29.97) - Resolution: same as the master - A/V sync within ±40 ms across the whole output - The first frame of each edit event must match the physical frame indicated by resolving the drop-frame/non-drop-frame mismatch.

  4. agent

    I’m checking the master metadata and the EDL first so I can map the non-drop edit points onto the drop-frame master without introducing frame drift or A/V offset.

  5. agent

    Executed exec_command call_vLGVgER5UKEQ3JADYQyJ7XF0

  6. agent

    Executed exec_command call_NYenPHU5khmoTwkiGQ429GH7

  7. agent

    Executed exec_command call_yTjFnel18ycw6Z9egwkZ5L7V

  8. agent

    I have the frame rate, duration, and the three EDL events. Next I’m verifying whether the MOV carries an embedded starting timecode; if not, I’ll treat frame 0 as `00:00:00;00` and map the non-drop counts onto physical frame numbers directly.

  9. agent

    Executed exec_command call_ppLzGOAjmvdkWM1SQrI9GKrT

  10. agent

    Executed exec_command call_E9jZYWFMtmQtxwtSiIXY0SBE

  11. agent

    Executed write_stdin call_KQBi76Imi0IVNCp3DkQAv7Bn

  12. agent

    Plan: 1. Resolve each EDL timecode to an absolute non-drop frame count. 2. Build a frame-accurate `ffmpeg` concat using `trim=start_frame/end_frame` for video and matching audio trims. 3. Encode `/logs/artifacts/re_cut.mp4` at `30000/1001`, then verify frame count, duration, and stream alignment.

  13. agent

    Executed update_plan call_4X2ahfbMgJGwO5bhuLK80j8V

  14. agent

    Executed exec_command call_Z2o63vxlCm0g47QdGxVPJUAy

  15. agent

    Executed exec_command call_IqKBw6J6j8BHqlpLbn0ipOdH

  16. agent

    I’ve resolved the edit points to exact physical frame numbers. I’m running a single decode/trim/concat pass now so the cuts are frame-accurate on video and share the same timing boundaries on audio.

  17. agent

    Executed exec_command call_r7GRO6e0XDmGEjO9vX6OZthX

  18. agent

    Executed write_stdin call_NckFPRnmOP4o9TD7PHSjjOg6

  19. agent

    The render completed with the expected 870 video frames. I’m probing the result now to make sure the output rate, dimensions, and stream durations line up with the cut list.

  20. agent

    Executed exec_command call_8OE9rsRkZWIKcZ1WBlr6HUnM

  21. agent

    Executed exec_command call_EVGsiiDOHt2AP8IZfGwH8c8p

  22. agent

    Executed exec_command call_RWrBVttKWJa2gUUvyqRSnKGG

  23. agent

    I’ve confirmed the output duration and audio sample count are exact. One more check: I’m matching the first frame of each output segment against the corresponding master frame so the drop-frame/non-drop-frame conversion is proven, not assumed.

  24. agent

    Executed exec_command call_QrhdsURluXRnEJNq45ksUJNa

  25. agent

    Executed exec_command call_tYhgGlHY301UkNMtKAJrfWGg

  26. agent

    Executed exec_command call_yVO989OzMeUUDqDOiceZU9wM

  27. agent

    Executed exec_command call_8EFPF8PtW40ohnO5HuSGor8i

  28. agent

    Executed write_stdin call_QeIvz2bVcsz8qhWEzaAZGhvf

  29. agent

    The current cut is timed correctly, but I want a stronger verification than timing alone. I’m re-encoding the video losslessly so I can checksum the decoded output frames against the master and prove the cut points landed on the intended source frames.

  30. agent

    Executed exec_command call_ZyNB8U0cAexoKCm0YQcs5OPb

  31. agent

    Executed write_stdin call_Elg5ND578XAa4DEeLfDNMDYo

  32. agent

    Executed exec_command call_dgd8icTnBAahegqoW2lqh9K1

  33. agent

    Executed exec_command call_rf3Segt1ElrAxEU4EgBKTbu3

  34. agent

    Executed exec_command call_sbDFc1kbRZ7vQRUD91GiqNjR

  35. agent

    Executed update_plan call_zZYSvUKFui31Vgar8f37c7xB

  36. agent

    Created `/logs/artifacts/re_cut.mp4`. Verification: - Video: `480x270`, `30000/1001` fps, `870` frames - Audio/video duration: both `29.029000` s - Audio sample duration: `1,393,392` samples at `48 kHz`, matching the cut length exactly - Cut-frame checks: output frames `0`, `300`, and `420` checksum-match master frames `150`, `1740`, and `4500`, so the segment starts land on the intended physical frames despite the DF/NDF mismatch The EDL was resolved as non-drop frame counts: - `00:00:05:00` to `00:00:15:00` -> frames `150` to `450` - `00:00:58:00` to `00:01:02:00` -> frames `1740` to `1860` - `00:02:30:00` to `00:02:45:00` -> frames `4500` to `4950`