Code Question Answering on LongBench CodeQA v2
0.741AccuracySRLM (no sub-calls)
Evaluation results
| Method | Links | |
|---|---|---|
| SRLM (no sub-calls)Backbone=GPT-52026.03 | 0.741 | |
| SRLMBackbone=GPT-52026.03 | 0.689 | |
| RLM (no sub-calls)Backbone=GPT-52026.03 | 0.652 | |
| SRLMBackbone=Qwen3-Coder-480B2026.03 | 0.649 | |
| RLMBackbone=Qwen3-Coder-480B2026.03 | 0.598 | |
| RLMBackbone=GPT-52026.03 | 0.595 | |
| SRLM (no sub-calls)Backbone=Qwen3-Coder-480B2026.03 | 0.59 | |
| Summary agentBackbone=GPT-52026.03 | 0.58 | |
| RLM (no sub-calls)Backbone=Qwen3-Coder-480B2026.03 | 0.538 | |
| Summary agentBackbone=Qwen3-Coder-480B2026.03 | 0.5 | |
| CodeAct (+ sub-calls)Backbone=Qwen3-Coder-480B2026.03 | 0.26 | |
| CodeAct (+ BM25)Backbone=Qwen3-Coder-480B2026.03 | 0.24 | |
| Base ModelBackbone=GPT-52026.03 | 0.24 | |
| CodeAct (+ sub-calls)Backbone=GPT-52026.03 | 0.24 | |
| CodeAct (+ BM25)Backbone=GPT-52026.03 | 0.22 | |
| Base ModelBackbone=Qwen3-Coder-480B2026.03 | 0.2 |