Long-context reasoning on OOLONG trec_coarse
86.6ScoreKimi K2
Evaluation results
| Method | Links | |
|---|---|---|
| Kimi K2Architecture=Base LLM2026.03 | 86.6 | |
| SRLMBackbone=GPT-52026.03 | 65.5 | |
| SRLM (no sub-calls)Backbone=GPT-52026.03 | 60.7 | |
| RLMBackbone=Kimi K2, Recursion Depth=12026.03 | 60 | |
| RLMBackbone=GPT-5, Recursion Depth=12026.03 | 56.5 | |
| RLMBackbone=Kimi K2, Recursion Depth=22026.03 | 55 | |
| RLMBackbone=GPT-52026.03 | 53 | |
| SRLMBackbone=Qwen3-Coder-480B2026.03 | 51.8 | |
| RLM (no sub-calls)Backbone=GPT-52026.03 | 50.5 | |
| RLMBackbone=Qwen3-Coder-480B-A35B, Recursion Depth=12026.03 | 48 | |
| Summary agentBackbone=GPT-52026.03 | 46 | |
| SRLM (no sub-calls)Backbone=Qwen3-Coder-480B2026.03 | 45.9 | |
| RLMBackbone=Qwen3-Coder-480B2026.03 | 45.7 | |
| Summary agentBackbone=Qwen3-Coder-480B2026.03 | 44.1 | |
| GPT-5Architecture=Base LLM2026.03 | 44 | |
| Base ModelBackbone=GPT-52026.03 | 44 | |
| RLMBackbone=DeepSeek v3.2, Recursion Depth=12026.03 | 42.1 | |
| CodeAct (+ sub-calls)Backbone=GPT-52026.03 | 40 | |
| RLM (no sub-calls)Backbone=Qwen3-Coder-480B2026.03 | 39.1 | |
| CodeAct (+ BM25)Backbone=Qwen3-Coder-480B2026.03 | 38 | |
| CodeAct (+ BM25)Backbone=GPT-52026.03 | 38 | |
| Qwen3-Coder-480B-A35BArchitecture=Base LLM2026.03 | 36 | |
| Base ModelBackbone=Qwen3-Coder-480B2026.03 | 36 | |
| RLMBackbone=DeepSeek v3.2, Recursion Depth=22026.03 | 33.7 | |
| CodeAct (+ sub-calls)Backbone=Qwen3-Coder-480B2026.03 | 32 | |
| RLMBackbone=Qwen3-8B, Recursion Depth=12026.03 | 24 | |
| Qwen3-8BArchitecture=Base LLM2026.03 | 0 | |
| DeepSeek v3.2Architecture=Base LLM2026.03 | 0 |