Which option best defines a ground-truth plan in evaluation?

• A. The ideal sequence of tool calls and steps used as a benchmark to assess agent decisions.
• B. A random set of tools chosen for exploration.
• C. A plan to maximize tool call concurrency without constraints.
• D. A historical log of actions without a predefined benchmark.

Answer: (A)

A ground-truth plan is a predefined, ideal sequence of actions the system should follow to complete a task. In evaluation, it serves as the benchmark for judging an agent's decisions because it encodes the best-known path, including which tools to use and in what order, under ideal conditions. This lets you measure how closely the agent's actual plan matches the intended one, revealing strengths and gaps in decision-making, tool use, and timing. A random collection of tools has no intended correct sequence and thus can't provide a meaningful baseline. A plan designed to maximize concurrency without constraints ignores the correctness of the steps themselves and focuses on performance, which isn't a benchmark of whether the chosen sequence is appropriate. A historical log records what happened but doesn't prescribe the optimal sequence to aim for, so it doesn't function as the standard for evaluation.