Long-context code reasoning on InfiniteBench Code-Debug (test)
86AccuracyFull context
Evaluation Results
| Method | Links | |
|---|---|---|
| Full contextTarget model=GPT-OSS-120B, Ctx. budget=120k2026.05 | 86 | |
| Full contextTarget model=MiniMax-M2.5, Ctx. budget=152k2026.05 | 83.76 | |
| LongAttnComp (Stage 1)Target model=GPT-OSS-120B, Ctx. budget=16k2026.05 | 82.99 | |
| LongAttnComp (Stage 2, subq)Target model=MiniMax-M2.5, Ctx. budget=16k2026.05 | 82.23 | |
| LongAttnComp (Stage 2, nosubq)Target model=GPT-OSS-120B, Ctx. budget=16k2026.05 | 81.73 | |
| LongAttnComp (Stage 2, nosubq)Target model=MiniMax-M2.5, Ctx. budget=16k2026.05 | 81.72 | |
| LongAttnComp (Stage 1)Target model=MiniMax-M2.5, Ctx. budget=16k2026.05 | 81.22 | |
| LongAttnComp (Stage 2, subq)Target model=GPT-OSS-120B, Ctx. budget=16k2026.05 | 80.2 | |
| LongAttnComp (Stage 2, subq)Target model=DeepSeek-R1-0528, Ctx. budget=16k2026.05 | 76.9 | |
| LongAttnComp (Stage 2, subq)Target model=DeepSeek-R1-05282026.05 | 76.9 | |
| LongAttnComp (Stage 1)Target model=DeepSeek-R1-0528, Ctx. budget=16k2026.05 | 75.38 | |
| LongAttnComp (Stage 1)Target model=DeepSeek-R1-05282026.05 | 75.38 | |
| Full contextTarget model=DeepSeek-R1-0528, Ctx. budget=120k2026.05 | 74.37 | |
| Full contextTarget model=DeepSeek-R1-0528, Context budget=120k2026.05 | 74.37 | |
| LongAttnComp (Stage 2, nosubq)Target model=DeepSeek-R1-0528, Ctx. budget=16k2026.05 | 74.11 | |
| LongAttnComp (Stage 2, nosubq)Target model=DeepSeek-R1-05282026.05 | 74.11 | |
| Full contextTarget model=DeepSeek-V3.1, Ctx. budget=128k2026.05 | 67.51 | |
| LongAttnComp (Stage 2, nosubq)Target model=DeepSeek-V3.1, Ctx. budget=16k2026.05 | 65.99 | |
| LongAttnComp (Stage 1)Target model=DeepSeek-V3.1, Ctx. budget=16k2026.05 | 65.73 | |
| LongAttnComp (Stage 2, subq)Target model=DeepSeek-V3.1, Ctx. budget=16k2026.05 | 65.48 | |
| Spec PrefillTarget model=DeepSeek-R1-0528, Ctx. budget=16k2026.05 | 62.44 | |
| Speculative PrefillTarget model=DeepSeek-R1-05282026.05 | 62.44 | |
| Spec PrefillTarget model=DeepSeek-V3.1, Ctx. budget=16k2026.05 | 59.14 | |
| Spec PrefillTarget model=MiniMax-M2.5, Ctx. budget=16k2026.05 | 57.1 | |
| Spec PrefillTarget model=GPT-OSS-120B, Ctx. budget=16k2026.05 | 52.28 |