Long-context Question Answering on NarrativeQA
53.56F1 ScoreMiA-RAG
Evaluation Results
| Method | Links | |
|---|---|---|
| MiA-RAGRetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=MiA-Gen-14B, +S (Generator)=true, top-k=102025.12 | 53.56 | |
| MiA-RAGRetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=MiA-Gen-14B, +S (Generator)=true, top-k=52025.12 | 53.52 | |
| MiARetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-72B, +S (Generator)=true, top-k=102025.12 | 53.15 | |
| MiA-RAGRetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=MiA-Gen-14B, +S (Generator)=true, top-k=32025.12 | 52.48 | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=MiA-Gen-14B, +S (Generator)=true, top-k=102025.12 | 51.99 | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-72B, +S (Generator)=true, top-k=102025.12 | 51.81 | |
| MiARetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-72B, +S (Generator)=true, top-k=52025.12 | 51.04 | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=MiA-Gen-14B, +S (Generator)=true, top-k=32025.12 | 50.55 | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=MiA-Gen-14B, +S (Generator)=true, top-k=52025.12 | 50.08 | |
| MiARetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-72B, +S (Generator)=true, top-k=32025.12 | 50.05 | |
| MiA (Emb-Only)Retriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-72B, +S (Generator)=false, top-k=102025.12 | 49.88 | |
| VanillaRetriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-72B, +S (Generator)=false, top-k=102025.12 | 49.06 | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-72B, +S (Generator)=true, top-k=52025.12 | 48.46 | |
| MiA (Emb-Only)Retriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-72B, +S (Generator)=false, top-k=52025.12 | 48.06 | |
| MiARetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-14B, +S (Generator)=true, top-k=102025.12 | 47.87 | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-72B, +S (Generator)=true, top-k=32025.12 | 47.67 | |
| MiA (Emb-Only)Retriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-14B, +S (Generator)=false, top-k=102025.12 | 47.13 | |
| MiA (Emb-Only)Retriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-14B, +S (Generator)=false, top-k=52025.12 | 46.77 | |
| MiARetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-14B, +S (Generator)=true, top-k=52025.12 | 46.66 | |
| MiA (Emb-Only)Retriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-72B, +S (Generator)=false, top-k=32025.12 | 46.38 | |
| MiA (Emb-Only)Retriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-14B, +S (Generator)=false, top-k=32025.12 | 45.89 | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-14B, +S (Generator)=true, top-k=102025.12 | 45.83 | |
| VanillaRetriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-72B, +S (Generator)=false, top-k=52025.12 | 45.51 | |
| VanillaRetriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-14B, +S (Generator)=false, top-k=102025.12 | 44.38 | |
| MiARetriever=MiA-Emb-8B, +S (Retriever)=true, Generator=Qwen2.5-14B, +S (Generator)=true, top-k=32025.12 | 44.38 | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-14B, +S (Generator)=true, top-k=52025.12 | 43.32 | |
| MiA (Gen-Only)Retriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-14B, +S (Generator)=true, top-k=32025.12 | 43.04 | |
| VanillaRetriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-72B, +S (Generator)=false, top-k=32025.12 | 41.13 | |
| VanillaRetriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-14B, +S (Generator)=false, top-k=52025.12 | 40.99 | |
| VanillaRetriever=Qwen3-8B, +S (Retriever)=false, Generator=Qwen2.5-14B, +S (Generator)=false, top-k=32025.12 | 39.32 | |
| Summary-OnlyGenerator=Qwen2.5-72B, +S (Generator)=true2025.12 | 39.24 | |
| Summary-OnlyGenerator=Qwen2.5-14B, +S (Generator)=true2025.12 | 38.03 | |
| MambaTraining Protocol=Trained from scratch, Training Tokens=>1T, Input Context Length=16K2024.09 | 27.9 | |
| MistralTraining Protocol=Trained from scratch, Training Tokens=>1T, Input Context Length=16K2024.09 | 25.1 | |
| GSATraining Protocol=Finetuned from Mistral 7B, Training Tokens=20B, Input Context Length=16K, Base Model=Mistral 7B2024.09 | 19.2 | |
| GLATraining Protocol=Finetuned from Mistral 7B, Training Tokens=20B, Input Context Length=16K, Base Model=Mistral 7B2024.09 | 17.2 | |
| RWKV6Training Protocol=Trained from scratch, Training Tokens=>1T, Input Context Length=16K2024.09 | 14.4 | |
| RetNetTraining Protocol=Finetuned from Mistral 7B, Training Tokens=20B, Input Context Length=16K, Base Model=Mistral 7B2024.09 | 0 |