LiveCodeBench
Benchmarks
TaskDatasetSOTA resultTrendResults
LiveCodeBench
19.6pass@1
10
LiveCodeBench v5
29.94Score
10
LiveCodeBench
80.1Expected Utility
10
LiveCodeBench v6
61.5Pass Rate
10
LiveCodeBench
11Pass@1
10
LiveCodeBench v5 2024.10–2025.02
60.6Pass@1
10
LiveCodeBench
29.1Accuracy
10
LiveCodeBench
52.48Score
10
LiveCodeBench v3
70.2Pass@1
10
LiveCodeBench v6
0.611Solve Rate
9
LiveCodeBench v6
67.3Pass@1
9
LiveCodeBench
13.88Score
9
LiveCodeBench v6
94Pass@3
9
LiveCodeBench Post-March 2025 V6
94.4Pass@1 (Easy)
9
LiveCodeBench 2025/01/01 - 2025/05/01 v6 (test)
72.53Accuracy
9
LiveCodeBench
56.12Avg@3 Accuracy
9
LiveCodeBench May 2023 to February 2024 (start date 2023-05-01)
25Pass@1
9
LiveCodeBench May 2023 to February 2024 (start date 2023-07-01)
24.1Pass@1
9
LiveCodeBench start date 2023-09-01 May 2023 to February 2024
23.2Pass@1
9
LiveCodeBench v6
43.22Avg@8
8
LiveCodeBench v1-5
28.3Accuracy
8
LiveCodeBench
92Pass@1
8
LiveCodeBench
4.93Latency (ms/tok)
8
LiveCodeBench
5.29Latency (ms/tok)
8
LiveCodeBench (83 problems, 10 permutations)
179.9Average Cost
8