MBPP
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MBPP
65.7Response Accuracy
16
MBPP
84.26Performance (%)
16
MBPP
2.032Acceptance Length (τ)
16
MBPP CodaSet OOD (test)
79.37Performance (%)
16
MBPP
80Knowledge Recall
16
MBPP
3.53Success Rate (SR)
16
MBPP
67.19Average Performance
16
MBPP (test)
25.18MAE
16
MBPP
87.53Execution Score
16
MBPP
65.1MBPP Pass@1
16
MBPP
99.2ASR
16
MBPP (test)
69.43-Shot Pass Rate
16
MBPP
99.78Pass%
15
MBPP Base
92Pass@1
15
MBPP
83.28Solve Rate
15
MBPP base standard (test)
66.17Pass@1
15
MBPP Plus 3-shot (test)
42.6Pass@1
15
MBPP
86.9Accuracy (CC→MBPP)
15
MBPP Code Llama
3.08MAT
15
MBPP+
79.2Accuracy
14
MBPP+
99.89Attack Success Rate (ASR)
14
MBPP
4Speedup
14
MBPP MBPP+
83.5MBPP Score
14
MBPP
97.1TPR @ 1% FPR
14
MBPP C++
89.2Compilation Success Rate
13