PaperBench
Benchmarks
TaskDatasetSOTA resultTrendResults
PaperBench Code (dev)
78.2Score
15
PaperBench Code lite (dev)
49.26MU-DPO
12
PaperBench Code (dev)
64.1Score
9
PaperBench
66.8Score
9
PaperBench Code (dev)
78.6Final Score
9
PaperBench Code (dev)
72.22FRE Score
7
PaperBench
33.26Adaptive Pruning Score
6
PaperBench Code ICML 2024 (dev)
0.786Average Score
6
PaperBench Category B: Detail
92.6Accuracy
2
PaperBench Category A: Fidelity
96.7Accuracy
2