Note Human-validated subset. The verification step is the interesting part — the original split scored tasks that were not solvable.

Note Contamination-resistant variant.

Note General assistant tasks with unambiguous answers — hard to game, easy to score.

Note Trajectory-level evaluation. Scoring only the final answer misses agents that were right for fabricated reasons.

Note Agent security — the failure mode that does not show up in accuracy.

Note Tool-calling leaderboard.

Note Memory-focused agent evaluation.