Question Answering on TyDi QA
69.4AccuracyFlan-PaLM 2 (L)
Evaluation Results
| Method | Links | |
|---|---|---|
| Flan-PaLM 2 (L)shot=1-shot2023.05 | 69.4 | |
| Flan-U-PaLM-540Bshot=1-shot2023.05 | 68.3 | |
| PaLM 2 (L)shot=1-shot2023.05 | 59.2 | |
| U-PaLM-540Bshot=1-shot2023.05 | 54.6 | |
| IPROXTarget Model=Qwen2-7B, Sparsity Level (ρ)=0.3, #Params=5.8B2026.02 | 53.56 | |
| Qwen2-7BTarget Model=Qwen2-7B, Proxy Model=Qwen2-7B, #Params=7B2026.02 | 51.46 | |
| IPROXTarget Model=Qwen2-7B, Sparsity Level (ρ)=0.5, #Params=4.4B2026.02 | 51.36 | |
| IPROXTarget Model=Qwen3-4B, Sparsity Level (ρ)=0.3, #Params=3.1B2026.02 | 50.63 | |
| IPROXTarget Model=Qwen2-7B, Sparsity Level (ρ)=0.7, #Params=3.3B2026.02 | 50.61 | |
| Qwen3-4BTarget Model=Qwen3-4B, Proxy Model=Qwen3-4B, #Params=4B2026.02 | 49.56 | |
| Contrastive Decoding2025.12 | 49.1 | |
| IPROXTarget Model=Qwen3-4B, Sparsity Level (ρ)=0.5, #Params=2.2B2026.02 | 48.45 | |
| LLaMA2-7B2025.12 | 48.1 | |
| DeVecvariant=second2025.12 | 48.1 | |
| IPROXTarget Model=Qwen3-4B, Sparsity Level (ρ)=0.7, #Params=1.5B2026.02 | 47.98 | |
| Qwen3-1.7BTarget Model=Qwen3-4B, Proxy Model=Qwen3-1.7B, #Params=1.7B2026.02 | 47.35 | |
| Qwen2-1.5BTarget Model=Qwen2-7B, Proxy Model=Qwen2-1.5B, #Params=1.5B2026.02 | 47.29 | |
| Safe Activation2025.12 | 46.9 | |
| Model Surgery2025.12 | 45.6 | |
| DeVecvariant=first2025.12 | 44.4 | |
| IPROXTarget Model=Llama3.2-3B, Sparsity (ρ)=0.32026.02 | 44.35 | |
| Task Vector2025.12 | 43.4 | |
| Llama3.2-3BTarget Model=Llama3.2-3B, Proxy Model=Llama3.2-3B, #Params=3B2026.02 | 43.1 | |
| Llama3.2-3BTarget Model=Llama3.2-3B2026.02 | 41.88 | |
| IPROXTarget Model=Llama3.2-3B, Sparsity (ρ)=0.52026.02 | 41.57 | |
| IPROXTarget Model=Llama3.2-3B, Sparsity Level (ρ)=0.3, #Params=2.5B2026.02 | 40.98 | |
| IPROXTarget Model=Llama3.2-3B, Sparsity Level (ρ)=0.5, #Params=1.8B2026.02 | 39.77 | |
| IPROXTarget Model=Llama3.2-3B, Sparsity (ρ)=0.72026.02 | 39.05 | |
| IPROXTarget Model=Llama3.2-3B, Sparsity Level (ρ)=0.7, #Params=1.3B2026.02 | 39.04 | |
| Llama3.2-1BTarget Model=Llama3.2-3B, Proxy Model=Llama3.2-1B, #Params=1B2026.02 | 38.84 | |
| Llama3.2-1BTarget Model=Llama3.2-3B2026.02 | 38.13 | |
| IPROXTarget Model=Gemma3-4B, Sparsity (ρ)=0.32026.02 | 34.18 | |
| IPROXTarget Model=Gemma3-4B, Sparsity (ρ)=0.52026.02 | 33.94 | |
| Gemma3-4BTarget Model=Gemma3-4B2026.02 | 32.48 | |
| IPROXTarget Model=Gemma3-4B, Sparsity Level (ρ)=0.3, #Params=3B2026.02 | 32.19 | |
| IPROXTarget Model=Gemma3-4B, Sparsity Level (ρ)=0.5, #Params=2.3B2026.02 | 31.42 | |
| Gemma3-1BTarget Model=Gemma3-4B2026.02 | 30.01 | |
| IPROXTarget Model=Gemma3-4B, Sparsity Level (ρ)=0.7, #Params=1.6B2026.02 | 29.12 | |
| IPROXTarget Model=Gemma3-4B, Sparsity (ρ)=0.72026.02 | 28.44 | |
| Gemma3-4BTarget Model=Gemma3-4B, Proxy Model=Gemma3-4B, #Params=4B2026.02 | 28.14 | |
| Gemma3-1BTarget Model=Gemma3-4B, Proxy Model=Gemma3-1B, #Params=1B2026.02 | 25.43 | |
| SFT2025.12 | 22.8 | |
| Feature Subtraction2025.12 | 21.3 |