LiveCodeBench
Benchmarks
TaskDatasetSOTA resultTrendResults
LiveCodeBench v5
75.4Avg@8 Score
6
LiveCodeBench (LCB)
14.37Score
6
LiveCodeBench v5
70.8Accuracy
6
LiveCodeBench v6
42.4Score
6
LiveCodeBench (LCB) Pro-Medium
28.1Pass@1
6
LiveCodeBench Pro-Easy
81.4Pass@1
6
LiveCodeBench 2408-2505
76.43Pass@1
6
LiveCodeBench v6
64.6pass@5
6
LiveCodeBench v5 (880 samples)
0.46Cost
6
LiveCodeBench
56Pass@1
6
LiveCodeBench generation_lite LeetCode easy (test)
27.4Accuracy
6
LiveCodeBench 2024-07 to 2024-12 v5
82.1Accuracy
5
LiveCodeBench (LCB)
92.88LCB Accuracy (Easy)
5
LiveCodeBench Pro
90.8Score
5
LiveCodeBench v5
45.34Pass@16
5
LiveCodeBench 880 problems
1Tau of Mean
5
LiveCodeBench V5
37.6Pass@1
5
LiveCodeBench v6 (test)
37.53pass@8
5
LiveCodeBench Hard v6
74.6Accuracy
5
LiveCodeBench Medium v6
89.4Accuracy
5
LiveCodeBench Easy v6
99.1Accuracy
5
LiveCodeBench Overall v6
84.3Accuracy
5
LiveCodeBench HumanEval, HumanEval+, MBPP, MBPP+ (queries from Eurus-RL-Code) v6 (test)
43.2LCB v6 Accuracy
5
LiveCodeBench 50/50 calibration (test)
64.9CR (%)
5
LiveCodeBench
34.02Average Score
5