V*Bench
Benchmarks
TaskDatasetSOTA resultTrendResults
V*Bench
98.95Accuracy
111
V*Bench
95.7Accuracy
62
V* Bench
98.3Attribute Score
49
V* Bench
90.4Accuracy
41
V* Bench
95.7Overall Score
38
V* Bench
96.34Overall Score
33
V* Bench
95.7Average Accuracy
32
V* Bench
89.01Score
28
V*Bench
92.15Overall Accuracy
28
V* Bench (Overall)
96.5Attribute Score
24
V* Bench
94.76Accuracy
19
V* Bench
85.6Accuracy
18
V* Bench
0.942Avg@8 EM
18
V* Bench
85.5General Score
18
V* Bench (test)
95Overall Accuracy
17
V* Bench
95.7Overall Success Rate
17
V* Bench Tool-needed
90.1Accuracy
15
V* Bench 1.0 (test)
83.48Attribute Score
13
V*Bench
37.17Accuracy
11
V*Bench
87.4Pass@1
10
V*-Bench v1.0 (test)
83.8Overall Score
10
V*Bench
84.3Accuracy
9
V* Bench
88.2Accuracy
9
V*Bench (REKEY renewal)
79.9Accuracy
8
V*Bench Relabel
75.9Accuracy
8