Long-context Question Answering on MuSiQue (F1)
51.02F1 ScoreLongMab
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LongMabBackbone=Qwen3-8B2025.08 | 51.02 | 48 | |
| LongFaith-POBackbone=Qwen3-8B2025.08 | 46.35 | 41 | |
| LongMab-SFTBackbone=Qwen3-8B2025.08 | 45.62 | 41.5 | |
| Logo-POBackbone=Qwen3-8B2025.08 | 44.98 | 41 | |
| Vanilla LLMBackbone=Qwen3-8B2025.08 | 44.24 | 41.5 | |
| SeaLong-POBackbone=Qwen3-8B2025.08 | 44.11 | 42.5 | |
| LongReward-POBackbone=Qwen3-8B2025.08 | 41.94 | 40.5 | |
| LongAlpacaBackbone=Qwen3-8B2025.08 | 38.06 | 36.5 | |
| GLM-4.1V-9B-Thinking VERARAG Strategy=Attention-Guided RAG2026.02 | 30.58 | — | |
| GLM-4.1V-9B-Thinking ColPali RAGRAG Strategy=ColPali RAG2026.02 | 30.24 | — | |
| LongAlignBackbone=Qwen3-8B2025.08 | 28.76 | 29 | |
| GLM-4.1V-9B-Thinking Random RAGRAG Strategy=Random RAG2026.02 | 28.71 | — | |
| GLM-4.1V-9B-Thinking Embedding RAGRAG Strategy=Embedding RAG2026.02 | 28.08 | — | |
| GLM-4.1V-9B-ThinkingRAG Strategy=Direct (None)2026.02 | 27.85 | — | |
| GlyphRAG Strategy=Direct (None)2026.02 | 24.87 | — | |
| Qwen3-VL-8B-Instruct VERARAG Strategy=Attention-Guided RAG2026.02 | 17.84 | — | |
| Qwen3-VL-8B-Instruct OCR RAGRAG Strategy=OCR RAG2026.02 | 14.7 | — | |
| Qwen3-VL-8B-Instruct Random RAGRAG Strategy=Random RAG2026.02 | 14.68 | — | |
| Qwen3-VL-8B-InstructRAG Strategy=Direct (None)2026.02 | 9.25 | — |