Long Context Question Answering on MultiFieldQA (Accuracy)
57.33AccuracyPOP
Evaluation results
| Method | Links | |
|---|---|---|
| POPBackbone=Gemma-3-12B-It, Pruning Ratio=33.3%2026.02 | 57.33 | |
| Full ModelBackbone=Gemma-3-12B-It, Pruning Ratio=0%2026.02 | 55.9 | |
| WandaBackbone=Gemma-3-12B-It, Pruning Ratio=30%2026.02 | 55.28 | |
| Full ModelBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=0%2026.02 | 54.57 | |
| Full ModelBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=0%2026.02 | 53.53 | |
| POPBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=31.25%2026.02 | 52.88 | |
| WandaBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=30%2026.02 | 52.87 | |
| WandaBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=30%2026.02 | 52.8 | |
| POPBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=33.3%2026.02 | 52.34 | |
| SliceGPTBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=25%2026.02 | 40.76 | |
| ShortGPTBackbone=Qwen3-VL-8B-Instruct, Pruning Ratio=25%2026.02 | 21.44 | |
| SliceGPTBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=25%2026.02 | 12.35 | |
| SliceGPTBackbone=Gemma-3-12B-It, Pruning Ratio=25%2026.02 | 10.83 | |
| ShortGPTBackbone=Llama-3.1-8B-Instruct, Pruning Ratio=25%2026.02 | 6.8 | |
| ShortGPTBackbone=Gemma-3-12B-It, Pruning Ratio=25%2026.02 | 1.58 |