Sources / Research and analysis

Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces (arXiv 2601.11868)

2 records were read from this source, dated Jan 2026.

Open it on arxiv.org

Its permanent copy is being made in the Wayback Machine.

Last checked 29 Oct 2026

What was read from it

Records read from this source
Record	Kind	As stated	Dated
Terminal-Bench 2.0 crowd-sourced tasks through open-source contributions: 93 contributors created 229 tasks and 89 were selected for the benchmark after a difficulty assessment and a review by three experienced humans; each kept task has its own environment, human-written solution and tests.	Supply fact	93 contributors, 229 tasks, 89 kept	Jan 2026
Cost and quote a task set per audited, accepted task, because review adds hours on top of authoring and most submitted tasks do not survive it	Negotiation lever	Terminal-Bench 2.0 (arXiv 2601.11868, January 2026) crowd-sourced 229 tasks from 93…	Jan 2026

Terminal-Bench 2.0 crowd-sourced tasks through open-source contributions: 93 contributors created 229 tasks and 89 were selected for the benchmark after a difficulty assessment and a review by three experienced humans; each kept task has its own environment, human-written solution and tests.
Kind
Supply fact
As stated
93 contributors, 229 tasks, 89 kept
Dated
Jan 2026
Cost and quote a task set per audited, accepted task, because review adds hours on top of authoring and most submitted tasks do not survive it
Kind
Negotiation lever
As stated
Terminal-Bench 2.0 (arXiv 2601.11868, January 2026) crowd-sourced 229 tasks from 93…
Dated
Jan 2026

Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces… | AIData.Broker