List your company
ResearchFormerly WizWand

Question Answering on TriviaQA (EM, F1)

89.02F1IndexLM-1.7B
-1.324822.130145.58569.0399Jan 6, 2025Mar 12, 2025May 17, 2025Jul 21, 2025Sep 25, 2025Nov 29, 2025Feb 3, 2026
Updated 8mo ago

Evaluation results

MethodLinks
IndexLM-1.7Banswer model=Gemma3-27B-it, Avg. Tokens=20432025.1289.02—
IndexLM-4Banswer model=Qwen3-4B, Avg. Tokens=19282025.1289.01—
IndexLM-1.7Banswer model=Qwen3-4B, Avg. Tokens=20432025.1288.92—
IndexLM-0.6Banswer model=Gemma3-27B-it, Avg. Tokens=19662025.1288.79—
IndexLM-4Banswer model=Gemma3-27B-it, Avg. Tokens=19282025.1288.75—
Firecrawl Extractanswer model=Qwen3-4B, Avg. Tokens=13192025.1287.92—
Firecrawl Extractanswer model=Gemma3-27B-it, Avg. Tokens=13192025.1287.92—
IndexLM-0.6Banswer model=Qwen3-4B, Avg. Tokens=19662025.1287.76—
Qwen3-4B + promptanswer model=Gemma3-27B-it, Avg. Tokens=25632025.1285.96—
Qwen3-4B + promptanswer model=Qwen3-4B, Avg. Tokens=25632025.1284.43—
FlowSteerMethodology=Ours, Backbone=GPT-4o-mini2026.0284.1179.69
HtmlRAGanswer model=Gemma3-27B-it, Avg. Tokens=35622025.1283.72—
Orchestrator (Agent+RL)Methodology=Agent+RL, Backbone=GPT-4o-mini, Agent Configuration=Orchestrator2026.0283.2377.36
AFlow (4o-mini)Methodology=AFlow, Backbone=GPT-4o-mini2026.0282.573.44
Agentflow (Agent+RL)Methodology=Agent+RL, Backbone=GPT-4o-mini, Agent Configuration=Agentflow2026.0281.4775
Baseline (4o-mini)Methodology=Baseline, Backbone=GPT-4o-mini2026.0281.471.09
Router-R1 (Agent+RL)Methodology=Agent+RL, Backbone=GPT-4o-mini, Agent Configuration=Router-R12026.0280.4375.78
ReaderLM-v2answer model=Gemma3-27B-it, Avg. Tokens=40002025.1279.74—
Chunk-Rerankanswer model=Qwen3-4B, Avg. Tokens=40942025.1279.52—
Markdown (raw)answer model=Gemma3-27B-it, Avg. Tokens=40962025.1279.32—
HtmlRAGanswer model=Qwen3-4B, Avg. Tokens=35622025.1278.97—
ComprExITBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.0278.8672.97
Chunk-Rerankanswer model=Gemma3-27B-it, Avg. Tokens=40942025.1277.64—
ReaderLM-v2answer model=Qwen3-4B, Avg. Tokens=40002025.1276.48—
Prompt tuningBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Prompt tuning, Context Setting=w/ context2026.0276.371.23
ICAEBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.0274.1368.62
Llama 3.3 70BModel Size=70B2025.017356.6
Markdown (raw)answer model=Qwen3-4B, Avg. Tokens=40962025.1272.56—
500xBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.0271.5965.77
ComprExITBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Context Compression2026.0270.9364.62
SFT (Qwen3-8B)Methodology=SFT, Backbone=Qwen3-8B2026.0269.8860.94
GRPO (Qwen3-8B)Methodology=GRPO, Backbone=Qwen3-8B2026.0269.2359.38
Baseline (Qwen3-8B)Methodology=Baseline, Backbone=Qwen3-8B2026.0269.1760.16
Zero-shotBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Zero-shot, Context Setting=w/ context2026.0267.8357.52
Prompt tuningBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Prompt tuning, Context Setting=w/ context2026.0266.7561.04
Llama 3.2 3BModel Size=3B2025.0166.750.4
HTML (raw)answer model=Qwen3-4B, Avg. Tokens=40962025.1264.25—
ICAEBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Context Compression2026.0263.5457.59
HTML (raw)answer model=Gemma3-27B-it, Avg. Tokens=40962025.1259.98—
GeAR2025.0159.146.5
Zero-shotBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Zero-shot, Context Setting=w/o context2026.0252.0646.2
Zero-shotBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Zero-shot, Context Setting=w/ context2026.0249.3135.5
500xBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Context Compression2026.0232.6626.09
Zero-shotBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Zero-shot, Context Setting=w/o context2026.0221.615.63
BeaconBackbone=Llama-3.2-3B-Base, Evaluation Protocol=Context Compression2026.024.721.36
BeaconBackbone=Llama-3.2-1B-Base, Evaluation Protocol=Context Compression2026.022.150.51