ACEBench
Benchmarks
TaskDatasetSOTA resultTrendResults
ACEBench Agent
78Agent Score
36
ACEBench en
71.4Normal Score
34
ACEBench multi-turn (test)
76.5Process Accuracy
31
ACEBench
78Atom Score
20
ACEBench (agent-task)
85.8Average Success Rate
20
ACEBench Extended Setting
65.17Overall Score
18
ACEBench Standard Setting
68.92Overall Score
18
ACEBench Agent (test)
56.7Average Score
16
ACEBench
37.5ACE-E Score
16
ACEBench Parallel
81Accuracy
15
ACEBench Single
90Accuracy
15
ACEBench Agent
60End-to-End Accuracy
15
ACEBench agent subset Multi-Turn
84.4Success Rate
14
ACEBench
78.3Score (En)
14
ACEBench Agent
95Multi-Step Reasoning Score
13
ACEBench Normal
75.6Accuracy
13
ACEBench Normal (test)
53Summary Score
11
ACEBench Content OOD v1 (test)
40Adherence
8
ACEBench Format and Content OOD v1 (test)
38Adherence (Ad.)
8
ACEBench Special
87.6Overall Score
8
ACEBench
61.8Accuracy
8
ACEBench-en (out-of-distribution)
77.9Normal Score
8
ACEBench English
25Multistep (Agent)
7
ACEBench En
68MT Accuracy
7
ACEBench-en
56End-to-End Accuracy
7