Question Answering on TriviaQA (EM, F1)
89.02F1IndexLM-1.7B
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| IndexLM-1.7Banswer model=Gemma3-27B-it, Avg. Tokens=20432025.12 | 89.02 | — | |
| IndexLM-4Banswer model=Qwen3-4B, Avg. Tokens=19282025.12 | 89.01 | — | |
| IndexLM-1.7Banswer model=Qwen3-4B, Avg. Tokens=20432025.12 | 88.92 | — | |
| IndexLM-0.6Banswer model=Gemma3-27B-it, Avg. Tokens=19662025.12 | 88.79 | — | |
| IndexLM-4Banswer model=Gemma3-27B-it, Avg. Tokens=19282025.12 | 88.75 | — | |
| Firecrawl Extractanswer model=Qwen3-4B, Avg. Tokens=13192025.12 | 87.92 | — | |
| Firecrawl Extractanswer model=Gemma3-27B-it, Avg. Tokens=13192025.12 | 87.92 | — | |
| IndexLM-0.6Banswer model=Qwen3-4B, Avg. Tokens=19662025.12 | 87.76 | — | |
| Qwen3-4B + promptanswer model=Gemma3-27B-it, Avg. Tokens=25632025.12 | 85.96 | — | |
| Qwen3-4B + promptanswer model=Qwen3-4B, Avg. Tokens=25632025.12 | 84.43 | — | |
| FlowSteerMethodology=Ours, Backbone=GPT-4o-mini2026.02 | 84.11 | 79.69 | |
| HtmlRAGanswer model=Gemma3-27B-it, Avg. Tokens=35622025.12 | 83.72 | — | |
| Orchestrator (Agent+RL)Methodology=Agent+RL, Backbone=GPT-4o-mini, Agent Configuration=Orchestrator2026.02 | 83.23 | 77.36 | |
| AFlow (4o-mini)Methodology=AFlow, Backbone=GPT-4o-mini2026.02 | 82.5 | 73.44 | |
| Agentflow (Agent+RL)Methodology=Agent+RL, Backbone=GPT-4o-mini, Agent Configuration=Agentflow2026.02 | 81.47 | 75 | |
| Baseline (4o-mini)Methodology=Baseline, Backbone=GPT-4o-mini2026.02 | 81.4 | 71.09 | |
| Router-R1 (Agent+RL)Methodology=Agent+RL, Backbone=GPT-4o-mini, Agent Configuration=Router-R12026.02 | 80.43 | 75.78 | |
| ReaderLM-v2answer model=Gemma3-27B-it, Avg. Tokens=40002025.12 | 79.74 | — | |
| Chunk-Rerankanswer model=Qwen3-4B, Avg. Tokens=40942025.12 | 79.52 | — | |
| Markdown (raw)answer model=Gemma3-27B-it, Avg. Tokens=40962025.12 | 79.32 | — | |
| HtmlRAGanswer model=Qwen3-4B, Avg. Tokens=35622025.12 | 78.97 | — | |
| ComprExITBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.02 | 78.86 | 72.97 | |
| Chunk-Rerankanswer model=Gemma3-27B-it, Avg. Tokens=40942025.12 | 77.64 | — | |
| ReaderLM-v2answer model=Qwen3-4B, Avg. Tokens=40002025.12 | 76.48 | — | |
| Prompt tuningBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Prompt tuning, Context Setting=w/ context2026.02 | 76.3 | 71.23 | |
| ICAEBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.02 | 74.13 | 68.62 | |
| Llama 3.3 70BModel Size=70B2025.01 | 73 | 56.6 | |
| Markdown (raw)answer model=Qwen3-4B, Avg. Tokens=40962025.12 | 72.56 | — | |
| 500xBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.02 | 71.59 | 65.77 | |
| ComprExITBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Context Compression2026.02 | 70.93 | 64.62 | |
| SFT (Qwen3-8B)Methodology=SFT, Backbone=Qwen3-8B2026.02 | 69.88 | 60.94 | |
| GRPO (Qwen3-8B)Methodology=GRPO, Backbone=Qwen3-8B2026.02 | 69.23 | 59.38 | |
| Baseline (Qwen3-8B)Methodology=Baseline, Backbone=Qwen3-8B2026.02 | 69.17 | 60.16 | |
| Zero-shotBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Zero-shot, Context Setting=w/ context2026.02 | 67.83 | 57.52 | |
| Prompt tuningBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Prompt tuning, Context Setting=w/ context2026.02 | 66.75 | 61.04 | |
| Llama 3.2 3BModel Size=3B2025.01 | 66.7 | 50.4 | |
| HTML (raw)answer model=Qwen3-4B, Avg. Tokens=40962025.12 | 64.25 | — | |
| ICAEBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Context Compression2026.02 | 63.54 | 57.59 | |
| HTML (raw)answer model=Gemma3-27B-it, Avg. Tokens=40962025.12 | 59.98 | — | |
| GeAR2025.01 | 59.1 | 46.5 | |
| Zero-shotBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Zero-shot, Context Setting=w/o context2026.02 | 52.06 | 46.2 | |
| Zero-shotBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Zero-shot, Context Setting=w/ context2026.02 | 49.31 | 35.5 | |
| 500xBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Context Compression2026.02 | 32.66 | 26.09 | |
| Zero-shotBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Zero-shot, Context Setting=w/o context2026.02 | 21.6 | 15.63 | |
| BeaconBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.02 | 4.72 | 1.36 | |
| BeaconBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Context Compression2026.02 | 2.15 | 0.51 |