List your company
ResearchFormerly WizWand

Long-context Question Answering on NarrativeQA (SubEM, F1)

22SubEMLongMab
912.37515.7519.125Aug 19, 2025
Updated 6mo ago

Evaluation results

MethodLinks
LongMabBackbone=Qwen2.5-14B-Instruct2025.082226.83
LongReward-POBackbone=Qwen2.5-14B-Instruct2025.082120.33
LongMab-SFTBackbone=Qwen2.5-14B-Instruct2025.0820.522.67
Logo-POBackbone=Qwen2.5-14B-Instruct2025.0820.523.41
LongFaith-POBackbone=Qwen2.5-14B-Instruct2025.082024.37
LongMabBackbone=Llama-3.1-8B-Instruct2025.082028.61
Vanilla LLMBackbone=Qwen2.5-14B-Instruct2025.0819.522.54
Vanilla LLMBackbone=Qwen3-8B2025.081926.26
LongAlpacaBackbone=Qwen2.5-14B-Instruct2025.081921.62
SeaLong-POBackbone=Qwen2.5-14B-Instruct2025.081921.17
LongMabBackbone=Qwen3-8B2025.0818.527.34
LongAlignBackbone=Llama-3.1-8B-Instruct2025.0818.526.38
Logo-POBackbone=Llama-3.1-8B-Instruct2025.0818.528.05
SeaLong-POBackbone=Qwen3-8B2025.081825.15
Vanilla LLMBackbone=Llama-3.1-8B-Instruct2025.081826.61
SeaLong-POBackbone=Qwen-2.5-7B-Instruct2025.081816.27
LongMabBackbone=Qwen-2.5-7B-Instruct2025.081825.14
LongAlignBackbone=Qwen2.5-14B-Instruct2025.0817.521.52
LongAlpacaBackbone=Qwen-2.5-7B-Instruct2025.0817.519.72
LongMab-SFTBackbone=Qwen3-8B2025.081725.86
LongReward-POBackbone=Qwen3-8B2025.081723.13
Logo-POBackbone=Qwen3-8B2025.081724.48
LongAlpacaBackbone=Llama-3.1-8B-Instruct2025.0816.526.88
LongReward-POBackbone=Llama-3.1-8B-Instruct2025.0816.525.92
SeaLong-POBackbone=Llama-3.1-8B-Instruct2025.081626.45
LongMab-SFTBackbone=Llama-3.1-8B-Instruct2025.0815.524.98
LongAlignBackbone=Qwen-2.5-7B-Instruct2025.0815.521.98
LongMab-SFTBackbone=Qwen-2.5-7B-Instruct2025.0815.522.17
LongReward-POBackbone=Qwen-2.5-7B-Instruct2025.0815.517.67
Logo-POBackbone=Qwen-2.5-7B-Instruct2025.0815.518.28
LongAlpacaBackbone=Qwen3-8B2025.081522.31
LongAlignBackbone=Qwen3-8B2025.081522.89
Vanilla LLMBackbone=Qwen-2.5-7B-Instruct2025.081518.29
LongFaith-POBackbone=Qwen-2.5-7B-Instruct2025.081217.03
LongFaith-POBackbone=Qwen3-8B2025.0811.518.31
LongFaith-POBackbone=Llama-3.1-8B-Instruct2025.089.523.26