Programming on LiveCodeBench V3 V4 (test)
61.4AccuracyT3-Struct
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| T3-StructModel=GPT-OSS 120B2026.05 | 61.4 | 6 | |
| CompactDSModel=GPT-52026.05 | 60.9 | 6.1 | |
| Chunked traj.Model=GPT-52026.05 | 60.9 | 6.1 | |
| T3-SemanticModel=GPT-OSS 120B2026.05 | 60.9 | 5.2 | |
| T3-StructModel=GPT-52026.05 | 60.4 | 5.2 | |
| Wikipedia-DPRModel=GPT-52026.05 | 59.9 | 4.4 | |
| Tavily Search APIModel=GPT-OSS 120B2026.05 | 59.9 | 3.5 | |
| T3-ReflectModel=GPT-52026.05 | 59.9 | 4.4 | |
| Wikipedia-DPRModel=GPT-OSS 120B2026.05 | 59.4 | 2.6 | |
| OpenWebMathModel=GPT-52026.05 | 58.9 | 2.6 | |
| Wikipedia-RPJModel=GPT-52026.05 | 58.9 | 2.6 | |
| Wikipedia-RPJModel=GPT-OSS 120B2026.05 | 58.9 | 1.7 | |
| Full traj.Model=GPT-OSS 120B2026.05 | 58.9 | 1.7 | |
| Chunked traj.Model=GPT-OSS 120B2026.05 | 58.9 | 1.7 | |
| T3-SemanticModel=GPT-52026.05 | 58.9 | 2.6 | |
| Tavily Search APIModel=GPT-52026.05 | 58.4 | 1.7 | |
| T3-ReflectModel=GPT-OSS 120B2026.05 | 58.4 | 0.9 | |
| No RAGModel=GPT-OSS 120B2026.05 | 57.9 | — | |
| ArxivModel=GPT-52026.05 | 57.9 | 0.9 | |
| Full traj.Model=GPT-52026.05 | 57.9 | 0.9 | |
| No RAGModel=GPT-52026.05 | 57.4 | — | |
| StackExchangeModel=GPT-52026.05 | 57.4 | 0 | |
| ArxivModel=GPT-OSS 120B2026.05 | 57.4 | -0.9 | |
| CompactDSModel=GPT-OSS 120B2026.05 | 57.4 | -0.9 | |
| GitHubModel=GPT-52026.05 | 56.9 | -0.9 | |
| StackExchangeModel=GPT-OSS 120B2026.05 | 55 | -5 | |
| GitHubModel=GPT-OSS 120B2026.05 | 54.5 | -5.9 | |
| Chunked traj.Model=Gemini-2.5 Flash2026.05 | 48 | 6.4 | |
| Tavily Search APIModel=Gemini-2.5 Flash2026.05 | 47.9 | 6.2 | |
| T3-StructModel=Gemini-2.5 Flash2026.05 | 47 | 4.2 | |
| ArxivModel=Gemini-2.5 Flash2026.05 | 46.9 | 4 | |
| Wikipedia-DPRModel=Gemini-2.5 Flash2026.05 | 46.5 | 3.1 | |
| Full traj.Model=Gemini-2.5 Flash2026.05 | 46.5 | 3.1 | |
| CompactDSModel=Gemini-2.5 Flash2026.05 | 45.4 | 0.7 | |
| No RAGModel=Gemini-2.5 Flash2026.05 | 45.1 | — | |
| Wikipedia-RPJModel=Gemini-2.5 Flash2026.05 | 45 | -0.2 | |
| T3-ReflectModel=Gemini-2.5 Flash2026.05 | 45 | -0.2 | |
| T3-SemanticModel=Gemini-2.5 Flash2026.05 | 45 | -0.2 | |
| StackExchangeModel=Gemini-2.5 Flash2026.05 | 42.1 | -6.7 | |
| GitHubModel=Gemini-2.5 Flash2026.05 | 42.1 | -6.7 | |
| OpenWebMathModel=Gemini-2.5 Flash2026.05 | 39.6 | -12.2 | |
| OpenWebMathModel=GPT-OSS 120B2026.05 | 37.6 | -35.1 |