LIVECODE
Benchmarks
TaskDatasetSOTA resultTrendResults
LiveCode (test)
36.8Score
32
LiveCode
41.2LiveCode Score
31
LiveCode
83.6Accuracy
30
LiveCode
2.975Acceptance Length (τ)
16
LiveCode
42.8Pass@1
15
LIVECODE (held-out)
25.7DBS
9
LiveCode (test)
67.8Accuracy
8