Terminal-Bench 3.0

Taxonomy Definitions

Every task is labelled with one subdomain.

  • 9domains
  • 9subdomains
  • 100tasks

The definitions

Open a domain for its subdomains, and a subdomain for its definition and the tasks that reach it.

Distributed Systems & Databases

1 subdomain · 15 tasks
tb3-distributed-systems---databasesDistributed Systems & Databases15 tasks
  • distributed-systems
  • concurrency
  • kafka
  • transactions
  • saga-pattern
  • exactly-once

Business & Domain Workflows

1 subdomain · 15 tasks
tb3-business---domain-workflowsBusiness & Domain Workflows15 tasks
  • csv
  • accounting
  • json
  • concurrency
  • healthcare
  • audit-log

Developer Toolchain & Build

1 subdomain · 12 tasks
tb3-developer-toolchain---buildDeveloper Toolchain & Build12 tasks
  • static-analysis
  • python
  • cycle-detection
  • topological-sort
  • json
  • release-engineering

Media & Graphics

1 subdomain · 12 tasks
tb3-media---graphicsMedia & Graphics12 tasks
  • svg
  • audio
  • signal-processing
  • graphics
  • video
  • ffmpeg

Scientific Computing & Simulation

1 subdomain · 12 tasks
tb3-scientific-computing---simulationScientific Computing & Simulation12 tasks
  • csv
  • scheduling
  • optimization
  • gis
  • surveying
  • antitrust

ML & Forecasting

1 subdomain · 11 tasks
tb3-ml---forecastingML & Forecasting11 tasks
  • forecasting
  • time-series
  • seasonality
  • time_series
  • timezone
  • regression

Data Engineering & Pipelines

1 subdomain · 10 tasks
tb3-data-engineering---pipelinesData Engineering & Pipelines10 tasks
  • csv
  • cycle-detection
  • json
  • record-linkage
  • data-lineage
  • ooxml

Systems & SRE

1 subdomain · 7 tasks
tb3-systems---sreSystems & SRE7 tasks
  • cleanup
  • posix-acls
  • setfacl
  • chmod
  • permissions
  • inheritance

Security & Forensics

1 subdomain · 6 tasks
tb3-security---forensicsSecurity & Forensics6 tasks
  • cryptography
  • reverse_engineering
  • malware_analysis
  • sqlite
  • forensics
  • wal