List your company
ResearchFormerly WizWand

Question Answering on TyDi QA

69.4AccuracyFlan-PaLM 2 (L)
19.37632.36345.3558.337May 17, 2023Nov 1, 2023Apr 17, 2024Oct 2, 2024Mar 19, 2025Sep 3, 2025Feb 19, 2026
Updated 8mo ago

Evaluation results

MethodLinks
Flan-PaLM 2 (L)shot=1-shot2023.0569.4
Flan-U-PaLM-540Bshot=1-shot2023.0568.3
PaLM 2 (L)shot=1-shot2023.0559.2
U-PaLM-540Bshot=1-shot2023.0554.6
IPROXTarget Model=Qwen2-7B, Sparsity Level (ρ)=0.3, #Params=5.8B2026.0253.56
Qwen2-7BTarget Model=Qwen2-7B, Proxy Model=Qwen2-7B, #Params=7B2026.0251.46
IPROXTarget Model=Qwen2-7B, Sparsity Level (ρ)=0.5, #Params=4.4B2026.0251.36
IPROXTarget Model=Qwen3-4B, Sparsity Level (ρ)=0.3, #Params=3.1B2026.0250.63
IPROXTarget Model=Qwen2-7B, Sparsity Level (ρ)=0.7, #Params=3.3B2026.0250.61
Qwen3-4BTarget Model=Qwen3-4B, Proxy Model=Qwen3-4B, #Params=4B2026.0249.56
Contrastive Decoding2025.1249.1
IPROXTarget Model=Qwen3-4B, Sparsity Level (ρ)=0.5, #Params=2.2B2026.0248.45
LLaMA2-7B2025.1248.1
DeVecvariant=second2025.1248.1
IPROXTarget Model=Qwen3-4B, Sparsity Level (ρ)=0.7, #Params=1.5B2026.0247.98
Qwen3-1.7BTarget Model=Qwen3-4B, Proxy Model=Qwen3-1.7B, #Params=1.7B2026.0247.35
Qwen2-1.5BTarget Model=Qwen2-7B, Proxy Model=Qwen2-1.5B, #Params=1.5B2026.0247.29
Safe Activation2025.1246.9
Model Surgery2025.1245.6
DeVecvariant=first2025.1244.4
IPROXTarget Model=Llama3.2-3B, Sparsity (ρ)=0.32026.0244.35
Task Vector2025.1243.4
Llama3.2-3BTarget Model=Llama3.2-3B, Proxy Model=Llama3.2-3B, #Params=3B2026.0243.1
Llama3.2-3BTarget Model=Llama3.2-3B2026.0241.88
IPROXTarget Model=Llama3.2-3B, Sparsity (ρ)=0.52026.0241.57
IPROXTarget Model=Llama3.2-3B, Sparsity Level (ρ)=0.3, #Params=2.5B2026.0240.98
IPROXTarget Model=Llama3.2-3B, Sparsity Level (ρ)=0.5, #Params=1.8B2026.0239.77
IPROXTarget Model=Llama3.2-3B, Sparsity (ρ)=0.72026.0239.05
IPROXTarget Model=Llama3.2-3B, Sparsity Level (ρ)=0.7, #Params=1.3B2026.0239.04
Llama3.2-1BTarget Model=Llama3.2-3B, Proxy Model=Llama3.2-1B, #Params=1B2026.0238.84
Llama3.2-1BTarget Model=Llama3.2-3B2026.0238.13
IPROXTarget Model=Gemma3-4B, Sparsity (ρ)=0.32026.0234.18
IPROXTarget Model=Gemma3-4B, Sparsity (ρ)=0.52026.0233.94
Gemma3-4BTarget Model=Gemma3-4B2026.0232.48
IPROXTarget Model=Gemma3-4B, Sparsity Level (ρ)=0.3, #Params=3B2026.0232.19
IPROXTarget Model=Gemma3-4B, Sparsity Level (ρ)=0.5, #Params=2.3B2026.0231.42
Gemma3-1BTarget Model=Gemma3-4B2026.0230.01
IPROXTarget Model=Gemma3-4B, Sparsity Level (ρ)=0.7, #Params=1.6B2026.0229.12
IPROXTarget Model=Gemma3-4B, Sparsity (ρ)=0.72026.0228.44
Gemma3-4BTarget Model=Gemma3-4B, Proxy Model=Gemma3-4B, #Params=4B2026.0228.14
Gemma3-1BTarget Model=Gemma3-4B, Proxy Model=Gemma3-1B, #Params=1B2026.0225.43
SFT2025.1222.8
Feature Subtraction2025.1221.3