Question Answering on NarrativeQA (Score)
21.96ScoreLSTM-MAS
Evaluation results
| Method | Links | |
|---|---|---|
| LSTM-MASBackbone=Qwen2.5-32B, Context Window=8K2026.01 | 21.96 | |
| LSTM-MASBackbone=Phi-3-small-8k-instruct, Context Window=8K2026.01 | 18.95 | |
| LSTM-MASBackbone=Llama-3.2-3B-Instruct, Context Window=8K2026.01 | 17.72 | |
| CoABackbone=Qwen2.5-32B, Context Window=8K2026.01 | 15.03 | |
| LSTM-MASBackbone=Qwen2.5-1.5B, Context Window=8K2026.01 | 14.98 | |
| LSTM-MASBackbone=Llama-3.1-8B-Instruct, Context Window=8K2026.01 | 14.83 | |
| LSTM-MASBackbone=Qwen2.5-7B, Context Window=8K2026.01 | 14.6 | |
| CoABackbone=Phi-3-small-8k-instruct, Context Window=8K2026.01 | 13.9 | |
| LSTM-MASBackbone=Phi-3.5-mini-instruct, Context Window=8K2026.01 | 13.01 | |
| LONGAGENTBackbone=Llama-3.1-8B-Instruct, Context Window=8K2026.01 | 11.87 | |
| LSTM-MASBackbone=Qwen2.5-14B, Context Window=8K2026.01 | 11.51 | |
| CoABackbone=Llama-3.2-3B-Instruct, Context Window=8K2026.01 | 11.2 | |
| CoABackbone=Qwen2.5-1.5B, Context Window=8K2026.01 | 10.58 | |
| LONGAGENTBackbone=Qwen2.5-14B, Context Window=8K2026.01 | 10.56 | |
| CoABackbone=Qwen2.5-7B, Context Window=8K2026.01 | 10.36 | |
| CoABackbone=Qwen2.5-14B, Context Window=8K2026.01 | 9.77 | |
| CoABackbone=Llama-3.1-8B-Instruct, Context Window=8K2026.01 | 9.02 | |
| LSTM-MASBackbone=Qwen2.5-0.5B, Context Window=8K2026.01 | 8.2 | |
| LONGAGENTBackbone=Qwen2.5-7B, Context Window=8K2026.01 | 7.07 | |
| VanillaBackbone=Qwen2.5-14B, Context Window=8K2026.01 | 6.14 | |
| LightRAGBackbone=Qwen2.5-1.5B2026.01 | 6.09 | |
| CoABackbone=Phi-3.5-mini-instruct, Context Window=8K2026.01 | 5.46 | |
| LightRAGBackbone=Llama-3.2-3B-Instruct2026.01 | 5.3 | |
| LightRAGBackbone=Llama-3.1-8B-Instruct2026.01 | 5.23 | |
| LONGAGENTBackbone=Qwen2.5-32B, Context Window=8K2026.01 | 5.07 | |
| CoABackbone=Qwen2.5-0.5B, Context Window=8K2026.01 | 4.98 | |
| LONGAGENTBackbone=Llama-3.2-3B-Instruct, Context Window=8K2026.01 | 4.82 | |
| LONGAGENTBackbone=Phi-3.5-mini-instruct, Context Window=8K2026.01 | 4.27 | |
| LONGAGENTBackbone=Phi-3-small-8k-instruct, Context Window=8K2026.01 | 2.71 | |
| LONGAGENTBackbone=Qwen2.5-1.5B, Context Window=8K2026.01 | 1.91 | |
| VanillaBackbone=Qwen2.5-32B, Context Window=8K2026.01 | 1.87 | |
| LightRAGBackbone=Qwen2.5-0.5B2026.01 | 1.56 | |
| VanillaBackbone=Qwen2.5-7B, Context Window=8K2026.01 | 1.26 | |
| LONGAGENTBackbone=Qwen2.5-0.5B, Context Window=8K2026.01 | 0.57 | |
| VanillaBackbone=Qwen2.5-0.5B, Context Window=8K2026.01 | 0 | |
| VanillaBackbone=Qwen2.5-1.5B, Context Window=8K2026.01 | 0 | |
| VanillaBackbone=Llama-3.2-3B-Instruct, Context Window=8K2026.01 | 0 | |
| VanillaBackbone=Llama-3.1-8B-Instruct, Context Window=8K2026.01 | 0 | |
| VanillaBackbone=Phi-3.5-mini-instruct, Context Window=8K2026.01 | 0 | |
| VanillaBackbone=Phi-3-small-8k-instruct, Context Window=8K2026.01 | 0 |