Multi-document Question Answering on MDQA (Accuracy Thresholds)
72.6Accuracy @ 0% ThresholdAttention Buckets
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Attention BucketsModel=Vicuna-7B-v1.52026.06 | 72.6 | 61.4 | 60.6 | 60.8 | 59.6 | 63 | |
| Ms-PoEModel=Vicuna-7B-v1.52026.06 | 72.6 | 61.4 | 61.8 | 62 | 59 | 63.5 | |
| MoICEModel=Vicuna-7B-v1.52026.06 | 71.6 | 61.2 | 60.6 | 60.8 | 62.4 | 63.3 | |
| LPESModel=Vicuna-7B-v1.52026.06 | 71.4 | 62.2 | 62 | 61 | 61.6 | 63.6 | |
| Positional InterpolationModel=Vicuna-7B-v1.52026.06 | 71.2 | 59.6 | 58.8 | 56.4 | 56.2 | 60.4 | |
| BaselineModel=Vicuna-7B-v1.52026.06 | 70.4 | 58 | 55.4 | 55.4 | 60.4 | 59.9 | |
| Positional InterpolationModel=StableBeluga-7B2026.06 | 69.6 | 58.6 | 58.2 | 60 | 65.4 | 62.4 | |
| Attention BucketsModel=Qwen2.5-7B2026.06 | 69.6 | 62.2 | 63 | 60.2 | 62 | 63.4 | |
| LPESModel=Qwen2.5-7B2026.06 | 69.6 | 64.8 | 69.2 | 63 | 65.4 | 66.4 | |
| BaselineModel=Qwen2.5-7B2026.06 | 69.4 | 61 | 62.6 | 58.6 | 63.6 | 63 | |
| Ms-PoEModel=Qwen2.5-7B2026.06 | 69.4 | 61.8 | 63.4 | 60.2 | 61.4 | 63.2 | |
| Attention BucketsModel=StableBeluga-7B2026.06 | 69.2 | 59 | 59.8 | 59.2 | 67.4 | 63 | |
| LPESModel=StableBeluga-7B2026.06 | 68.8 | 60 | 60.8 | 61 | 68.2 | 64.5 | |
| Positional InterpolationModel=Qwen2.5-7B2026.06 | 68.6 | 62 | 62.2 | 58.4 | 64 | 63 | |
| Ms-PoEModel=StableBeluga-7B2026.06 | 68.4 | 57 | 60.2 | 61 | 68.4 | 63 | |
| MoICEModel=Qwen2.5-7B2026.06 | 68.4 | 61.2 | 63 | 61 | 63.8 | 63.5 | |
| BaselineModel=StableBeluga-7B2026.06 | 67.8 | 59.2 | 59.6 | 59.4 | 68.2 | 62.8 | |
| MoICEModel=StableBeluga-7B2026.06 | 67.4 | 60 | 60.2 | 60 | 68.6 | 63.2 |