Question Answering on NarrativeQA Helmet benchmark
49.5F1 ScoreMiA-Emb-8B + MiA-Gen-14B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MiA-Emb-8B + MiA-Gen-14B+Summ=true, Tokens=13k2025.12 | 49.5 | 29.8 | |
| MiA-Emb-8B + MiA-Gen-14B+Summ=true, Tokens=4k2025.12 | 48.7 | 28.9 | |
| GPT4o-2405+Summ=false, Tokens=128k, Context Setting=Full Context2025.12 | 46.5 | — | |
| GPT4o-2408+Summ=false, Tokens=128k, Context Setting=Full Context2025.12 | 43.1 | — | |
| Gemini-1.5-Pro+Summ=false, Tokens=2M, Context Setting=Full Context2025.12 | 42.8 | — | |
| MiA-Emb-8BGenerative Model=Qwen2.5-14B, +Summ=true, Tokens=13k2025.12 | 39.1 | 20.4 | |
| MiA-Emb-8BGenerative Model=GPT4o-2405, +Summ=false, Tokens=12k2025.12 | 38.9 | 21.9 | |
| MiA-Emb-8BGenerative Model=Qwen2.5-14B, +Summ=false, Tokens=12k2025.12 | 36.7 | 18.2 | |
| Qwen3-Emb-8BGenerative Model=Qwen2.5-14B, +Summ=false, Tokens=12k2025.12 | 34.8 | 17.7 |