List your company
ResearchFormerly WizWand

Question Answering on WebQuestions (WebQs)

63.5AccuracyGemini2.5-Pro
-2.5414.60531.7548.895May 28, 2020May 10, 2021Apr 23, 2022Apr 6, 2023Mar 19, 2024Mar 2, 2025Feb 13, 2026
Updated 8mo ago

Evaluation results

MethodLinks
Gemini2.5-ProEvaluation Protocol=Closed-Source2026.0163.5
GPT-4oEvaluation Protocol=Closed-Source2026.0163
GPT-5Evaluation Protocol=Closed-Source2026.0161.5
GPT-4.1Evaluation Protocol=Closed-Source2026.0161.5
ATLAS (cluster)Evaluation Protocol=In-Distribution2026.0153.6
CDKCBackbone=Qwen2.5-14B-Instruct2026.0252.51
ATLAS (RL)Evaluation Protocol=Out-of-Distribution2026.0152.2
BertRouterEvaluation Protocol=Out-of-Distribution2026.0151.4
ATLAS (cluster)Evaluation Protocol=Out-of-Distribution2026.0151.4
GRPOBackbone=Qwen2.5-14B-Instruct2026.0251.23
RouterDCEvaluation Protocol=Out-of-Distribution2026.0150.8
BertRouterEvaluation Protocol=In-Distribution2026.0150.4
GradOTSetting=Plug-in, Backbone=LLaMA-13B2025.0748.1
RouterDCEvaluation Protocol=In-Distribution2026.0147.6
RAGSetting=Fine-tuned, QA Environment=Open-Domain2020.0545.5
T5-11B+SSMSetting=Fine-tuned, QA Environment=Closed-Book, Auxiliary Training=SSM2020.0544.7
MLPRouterEvaluation Protocol=Out-of-Distribution2026.0143.7
CDKCBackbone=Qwen2.5-3B-Instruct2026.0243.26
GRPOBackbone=Qwen2.5-3B-Instruct2026.0242.96
CGKEBackbone=Qwen2.5-14B-Instruct2026.0241.93
GPT-3Learning Protocol=Few-Shot, QA Environment=Closed-Book2020.0541.5
MLPRouterEvaluation Protocol=In-Distribution2026.0140.4
ZS RouterEvaluation Protocol=Training-free, Prompting Strategy=Zero-shot2026.0139.2
CoTBackbone=Qwen2.5-14B-Instruct2026.0238.83
EmulatorSetting=Fine-tuned, Backbone=LLaMA-13B2025.0737.9
T5-11BSetting=Fine-tuned, QA Environment=Closed-Book2020.0537.4
Vanilla SFTBackbone=Qwen2.5-14B-Instruct2026.0235.97
FS RouterEvaluation Protocol=Training-free, Prompting Strategy=Few-shot2026.0135.8
OT†Setting=Plug-in, Backbone=LLaMA-13B2025.0735.4
CGKEBackbone=Qwen2.5-3B-Instruct2026.0234.79
RAGBackbone=Qwen2.5-14B-Instruct2026.0234.35
Vanilla LLMBackbone=Qwen2.5-14B-Instruct2026.0233.46
Random RouterEvaluation Protocol=Training-free2026.0132.1
Full LLMSetting=Fine-tuning (FT), Backbone=OPT-1.3B2025.0731.2
CoTBackbone=Qwen2.5-3B-Instruct2026.0231.2
RAGBackbone=Qwen2.5-3B-Instruct2026.0230.51
GradOTSetting=Plug-in, Backbone=OPT-1.3B2025.0729.2
GPT-3number of parameters=175B2023.0229
Vanilla SFTBackbone=Qwen2.5-3B-Instruct2026.0228.74
GradOTSetting=Emulator Fine-tuning (Emu. FT), Backbone=OPT-1.3B2025.0728.4
ScaleOTSetting=Plug-in, Backbone=OPT-1.3B2025.0728.2
Vanilla LLMBackbone=Qwen2.5-3B-Instruct2026.0226.38
ToolformerAPI calls=enabled2023.0226.3
OTSetting=Plug-in, Backbone=OPT-1.3B2025.0726.2
ScaleOTSetting=Emulator Fine-tuning (Emu. FT), Backbone=OPT-1.3B2025.0725.3
GPT-3Learning Protocol=One-Shot, QA Environment=Closed-Book2020.0525.3
OTSetting=Emulator Fine-tuning (Emu. FT), Backbone=OPT-1.3B2025.0724.3
CRaShSetting=Plug-in, Backbone=OPT-1.3B2025.0723.7
OT†Setting=Emulator Fine-tuning (Emu. FT), Backbone=OPT-1.3B2025.0721.8
CRaShSetting=Emulator Fine-tuning (Emu. FT), Backbone=OPT-1.3B2025.0721.8
OT†Setting=Plug-in, Backbone=OPT-1.3B2025.0721.4
GPT-3 (Original)few-shot (k)=642021.0819.6
GPT-3 (SLW 8x Bsz)few-shot (k)=642021.0819.4
ToolformerAPI calls=disabled2023.0218.9
OPTnumber of parameters=66B2023.0218.6
GPT-J2023.0218.5
GPT-3 (Baseline repro)few-shot (k)=642021.0818.4
GPT-J + CCpre-training=CCNet2023.0218.4
GPT-3Learning Protocol=Zero-Shot, QA Environment=Closed-Book2020.0514.4
CRaShSetting=Emulator Zero-shot (Emu. ZS), Backbone=OPT-1.3B2025.074.7
Full LLMSetting=Zero-shot (ZS), Backbone=OPT-1.3B2025.074.6
OTSetting=Emulator Zero-shot (Emu. ZS), Backbone=OPT-1.3B2025.071.3
ScaleOTSetting=Emulator Zero-shot (Emu. ZS), Backbone=OPT-1.3B2025.070.1
OT†Setting=Emulator Zero-shot (Emu. ZS), Backbone=OPT-1.3B2025.070
GradOTSetting=Emulator Zero-shot (Emu. ZS), Backbone=OPT-1.3B2025.070
Full ModelSetting=Zero-shot, Backbone=LLaMA-13B2025.070
EmulatorSetting=Zero-shot, Backbone=LLaMA-13B2025.070