Question Answering on QASPER Long-doc
52.76Gen-F1Base+ctx
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Base+ctxLLM Backbone=Qwen3-4B-Instruct, Evaluation Protocol=In-domain2026.06 | 52.76 | — | — | — | — | 50.72 | |
| Base+ctxLLM Backbone=Gemma-2-2B-It, Evaluation Protocol=In-domain2026.06 | 48.62 | — | — | — | — | 47.26 | |
| UAE + SePerMethod category=Rerank Ablation, Latency (ms)=17042026.04 | 34 | 61.91 | 74.11 | 2.4194 | 53.39 | 33.3 | |
| SePerMethod category=Multi-Stage (LLM) Rerank, Latency (ms)=16962026.04 | 32.6 | 59.84 | 72.71 | 2.3445 | 50.12 | 32.1 | |
| Doc2AtomLLM Backbone=Qwen3-4B-Instruct, Evaluation Protocol=In-domain2026.06 | 31.03 | — | — | — | — | 30.7 | |
| Doc2AtomLLM Backbone=Gemma-2-2B-It, Evaluation Protocol=In-domain2026.06 | 30.78 | — | — | — | — | 30.44 | |
| UAE + LLM RerankerMethod category=Rerank Ablation, Latency (ms)=16172026.04 | 29.8 | 50.48 | 75.7 | 1.2631 | 50.65 | 29.7 | |
| RankGPTMethod category=Multi-Stage (LLM) Rerank, Latency (ms)=16632026.04 | 29.2 | 48.89 | 71.52 | 1.2328 | 46.13 | 29.3 | |
| UAE + BGE-rerankerMethod category=Rerank Ablation, Latency (ms)=432026.04 | 28.5 | 44.45 | 70.71 | 1.1458 | 45.94 | 28.5 | |
| GainRAGMethod category=Multi-Stage (LLM) Rerank, Latency (ms)=22932026.04 | 28 | 46.3 | 69.23 | 1.3057 | 45.29 | 27.9 | |
| NV-Embed V2Method category=LLM Dense Retriever, Latency (ms)=522026.04 | 27.8 | 47.04 | 71.75 | 0.9749 | 38.67 | 27.7 | |
| BGE-V2-M3 rerankerMethod category=Multi-Stage (LLM) Rerank, Latency (ms)=522026.04 | 27.4 | 43.05 | 66.79 | 1.0882 | 42.93 | 27.5 | |
| D2LckptLLM Backbone=Qwen3-4B-Instruct, Evaluation Protocol=In-domain2026.06 | 27.03 | — | — | — | — | 25.39 | |
| UAEMethod category=Utility-Aligned Retrieval, Latency (ms)=92026.04 | 27 | 48.15 | 70.56 | 1.3269 | 49.49 | 27 | |
| ColBERTV2Method category=Classic Retrieval, Latency (ms)=462026.04 | 26.7 | 40.75 | 64.87 | 0.979 | 33.98 | 26.8 | |
| D2LckptLLM Backbone=Gemma-2-2B-It, Evaluation Protocol=In-domain2026.06 | 26.34 | — | — | — | — | 25.39 | |
| E5-MistralMethod category=LLM Dense Retriever, Latency (ms)=392026.04 | 26 | 40.46 | 65.24 | 0.8688 | 33.93 | 26.1 | |
| SPLADE V3Method category=Classic Retrieval, Latency (ms)=52026.04 | 25.2 | 38.46 | 61.76 | 0.9836 | 32.13 | 25.2 | |
| UtilityQwenMethod category=Multi-Stage (LLM) Rerank, Latency (ms)=32002026.04 | 24 | 40.95 | 64.25 | 1.096 | 37.66 | 24.1 | |
| BGE-BaseMethod category=Classic Retrieval, Latency (ms)=82026.04 | 23.9 | 36.87 | 61.28 | 0.8201 | 38.02 | 24.1 | |
| BM25Method category=Classic Retrieval, Latency (ms)=52026.04 | 20.5 | 26.26 | 49.7 | 0.6792 | 27.81 | 20.7 | |
| D2LatomLLM Backbone=Qwen3-4B-Instruct, Evaluation Protocol=In-domain2026.06 | 20.07 | — | — | — | — | 19.83 | |
| D2LrawLLM Backbone=Qwen3-4B-Instruct, Evaluation Protocol=In-domain2026.06 | 19.17 | — | — | — | — | 18.91 | |
| D2LatomLLM Backbone=Gemma-2-2B-It, Evaluation Protocol=In-domain2026.06 | 18.39 | — | — | — | — | 18.19 | |
| D2LrawLLM Backbone=Gemma-2-2B-It, Evaluation Protocol=In-domain2026.06 | 16.46 | — | — | — | — | 16.25 | |
| InfoNCE TunedMethod category=Classic Retrieval, Latency (ms)=92026.04 | 5.9 | 6.21 | 18.23 | 0.0932 | 8.61 | 6 |