Long-context Question Answering on 2WikiMQA
79.5SubEMLongMab
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| LongMabBackbone=Qwen2.5-14B-Instruct2025.08 | 79.5 | 68.83 | |
| LongFaith-POBackbone=Qwen2.5-14B-Instruct2025.08 | 78 | 67.85 | |
| LongMabBackbone=Qwen3-8B2025.08 | 77.5 | 74.32 | |
| Logo-POBackbone=Qwen2.5-14B-Instruct2025.08 | 77.5 | 66.52 | |
| Vanilla LLMBackbone=Qwen3-8B2025.08 | 77 | 72.4 | |
| SeaLong-POBackbone=Qwen3-8B2025.08 | 77 | 73.38 | |
| LongReward-POBackbone=Qwen3-8B2025.08 | 76 | 70.08 | |
| LongMabBackbone=Llama-3.1-8B-Instruct2025.08 | 76 | 68.6 | |
| LongMab-SFTBackbone=Qwen3-8B2025.08 | 75.5 | 73.16 | |
| Logo-POBackbone=Qwen3-8B2025.08 | 75.5 | 71.34 | |
| Vanilla LLMBackbone=Qwen2.5-14B-Instruct2025.08 | 75.5 | 63.88 | |
| LongFaith-POBackbone=Llama-3.1-8B-Instruct2025.08 | 75.5 | 72.8 | |
| SeaLong-POBackbone=Qwen2.5-14B-Instruct2025.08 | 75 | 57.91 | |
| LongReward-POBackbone=Qwen2.5-14B-Instruct2025.08 | 73.5 | 62.3 | |
| LongMab-SFTBackbone=Qwen2.5-14B-Instruct2025.08 | 73 | 64.01 | |
| LongAlpacaBackbone=Qwen3-8B2025.08 | 71 | 68.76 | |
| LongAlpacaBackbone=Qwen2.5-14B-Instruct2025.08 | 70 | 61.55 | |
| Vanilla LLMBackbone=Llama-3.1-8B-Instruct2025.08 | 69.5 | 65.45 | |
| LongAlpacaBackbone=Llama-3.1-8B-Instruct2025.08 | 69.5 | 63.64 | |
| LongAlignBackbone=Llama-3.1-8B-Instruct2025.08 | 69 | 59.35 | |
| LongMab-SFTBackbone=Llama-3.1-8B-Instruct2025.08 | 69 | 65.45 | |
| LongAlignBackbone=Qwen2.5-14B-Instruct2025.08 | 68.5 | 65.26 | |
| LongReward-POBackbone=Llama-3.1-8B-Instruct2025.08 | 68.5 | 63.23 | |
| SeaLong-POBackbone=Llama-3.1-8B-Instruct2025.08 | 68 | 67.07 | |
| LongMabBackbone=Qwen-2.5-7B-Instruct2025.08 | 67.5 | 62.97 | |
| SeaLong-POBackbone=Qwen-2.5-7B-Instruct2025.08 | 67 | 36.39 | |
| LongFaith-POBackbone=Qwen3-8B2025.08 | 66 | 66.79 | |
| LongFaith-POBackbone=Qwen-2.5-7B-Instruct2025.08 | 66 | 53.93 | |
| Logo-POBackbone=Llama-3.1-8B-Instruct2025.08 | 63 | 61.74 | |
| Logo-POBackbone=Qwen-2.5-7B-Instruct2025.08 | 63 | 57.2 | |
| LongAlignBackbone=Qwen3-8B2025.08 | 62 | 54.34 | |
| LongReward-POBackbone=Qwen-2.5-7B-Instruct2025.08 | 62 | 50.75 | |
| LongMab-SFTBackbone=Qwen-2.5-7B-Instruct2025.08 | 59.5 | 58.34 | |
| Vanilla LLMBackbone=Qwen-2.5-7B-Instruct2025.08 | 58 | 50.14 | |
| LongAlpacaBackbone=Qwen-2.5-7B-Instruct2025.08 | 55.5 | 50.75 | |
| LongAlignBackbone=Qwen-2.5-7B-Instruct2025.08 | 52 | 52.02 |