Long-context Question Answering on NarrativeQA (SubEM, F1)
22SubEMLongMab
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| LongMabBackbone=Qwen2.5-14B-Instruct2025.08 | 22 | 26.83 | |
| LongReward-POBackbone=Qwen2.5-14B-Instruct2025.08 | 21 | 20.33 | |
| LongMab-SFTBackbone=Qwen2.5-14B-Instruct2025.08 | 20.5 | 22.67 | |
| Logo-POBackbone=Qwen2.5-14B-Instruct2025.08 | 20.5 | 23.41 | |
| LongFaith-POBackbone=Qwen2.5-14B-Instruct2025.08 | 20 | 24.37 | |
| LongMabBackbone=Llama-3.1-8B-Instruct2025.08 | 20 | 28.61 | |
| Vanilla LLMBackbone=Qwen2.5-14B-Instruct2025.08 | 19.5 | 22.54 | |
| Vanilla LLMBackbone=Qwen3-8B2025.08 | 19 | 26.26 | |
| LongAlpacaBackbone=Qwen2.5-14B-Instruct2025.08 | 19 | 21.62 | |
| SeaLong-POBackbone=Qwen2.5-14B-Instruct2025.08 | 19 | 21.17 | |
| LongMabBackbone=Qwen3-8B2025.08 | 18.5 | 27.34 | |
| LongAlignBackbone=Llama-3.1-8B-Instruct2025.08 | 18.5 | 26.38 | |
| Logo-POBackbone=Llama-3.1-8B-Instruct2025.08 | 18.5 | 28.05 | |
| SeaLong-POBackbone=Qwen3-8B2025.08 | 18 | 25.15 | |
| Vanilla LLMBackbone=Llama-3.1-8B-Instruct2025.08 | 18 | 26.61 | |
| SeaLong-POBackbone=Qwen-2.5-7B-Instruct2025.08 | 18 | 16.27 | |
| LongMabBackbone=Qwen-2.5-7B-Instruct2025.08 | 18 | 25.14 | |
| LongAlignBackbone=Qwen2.5-14B-Instruct2025.08 | 17.5 | 21.52 | |
| LongAlpacaBackbone=Qwen-2.5-7B-Instruct2025.08 | 17.5 | 19.72 | |
| LongMab-SFTBackbone=Qwen3-8B2025.08 | 17 | 25.86 | |
| LongReward-POBackbone=Qwen3-8B2025.08 | 17 | 23.13 | |
| Logo-POBackbone=Qwen3-8B2025.08 | 17 | 24.48 | |
| LongAlpacaBackbone=Llama-3.1-8B-Instruct2025.08 | 16.5 | 26.88 | |
| LongReward-POBackbone=Llama-3.1-8B-Instruct2025.08 | 16.5 | 25.92 | |
| SeaLong-POBackbone=Llama-3.1-8B-Instruct2025.08 | 16 | 26.45 | |
| LongMab-SFTBackbone=Llama-3.1-8B-Instruct2025.08 | 15.5 | 24.98 | |
| LongAlignBackbone=Qwen-2.5-7B-Instruct2025.08 | 15.5 | 21.98 | |
| LongMab-SFTBackbone=Qwen-2.5-7B-Instruct2025.08 | 15.5 | 22.17 | |
| LongReward-POBackbone=Qwen-2.5-7B-Instruct2025.08 | 15.5 | 17.67 | |
| Logo-POBackbone=Qwen-2.5-7B-Instruct2025.08 | 15.5 | 18.28 | |
| LongAlpacaBackbone=Qwen3-8B2025.08 | 15 | 22.31 | |
| LongAlignBackbone=Qwen3-8B2025.08 | 15 | 22.89 | |
| Vanilla LLMBackbone=Qwen-2.5-7B-Instruct2025.08 | 15 | 18.29 | |
| LongFaith-POBackbone=Qwen-2.5-7B-Instruct2025.08 | 12 | 17.03 | |
| LongFaith-POBackbone=Qwen3-8B2025.08 | 11.5 | 18.31 | |
| LongFaith-POBackbone=Llama-3.1-8B-Instruct2025.08 | 9.5 | 23.26 |