Question Answering on NewsQA Short-doc
66.43R@1UAE + SePer
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| UAE + SePerMethod category=Rerank Ablation, Protocol=Zero-shot, Latency (ms)=17092026.04 | 66.43 | 88.1 | 6.2394 | 73.15 | 50.5 | 51 | |
| UAE + BGE-rerankerMethod category=Rerank Ablation, Protocol=Zero-shot, Latency (ms)=332026.04 | 65.65 | 86.22 | 5.8727 | 72.04 | 49.8 | 50.3 | |
| UAE + LLM RerankerMethod category=Rerank Ablation, Protocol=Zero-shot, Latency (ms)=15992026.04 | 60.87 | 87.32 | 5.7416 | 67.92 | 49.8 | 50.3 | |
| BGE-V2-M3 rerankerMethod category=Multi-Stage (LLM) Rerank, Latency (ms)=342026.04 | 60.67 | 77.62 | 5.7339 | 65.49 | 46.5 | 47 | |
| SePerMethod category=Multi-Stage (LLM) Rerank, Latency (ms)=17112026.04 | 60.61 | 78.65 | 6.1131 | 66.17 | 47.6 | 48 | |
| ColBERTV2Method category=Classic Retrieval, Latency (ms)=352026.04 | 55.11 | 76.53 | 5.5024 | 58.31 | 47.3 | 47.8 | |
| UAEMethod category=Utility-Aligned Retrieval, Protocol=Zero-shot, Latency (ms)=92026.04 | 54.9 | 77.3 | 5.8184 | 62.31 | 45.6 | 46.1 | |
| SPLADE V3Method category=Classic Retrieval, Latency (ms)=42026.04 | 50.58 | 73.67 | 5.3588 | 53.34 | 44.7 | 45.2 | |
| RankGPTMethod category=Multi-Stage (LLM) Rerank, Latency (ms)=16102026.04 | 49.68 | 71.54 | 4.968 | 46.78 | 43.7 | 44.2 | |
| BM25Method category=Classic Retrieval, Latency (ms)=62026.04 | 49.61 | 69.79 | 5.1859 | 52.13 | 41.8 | 42.2 | |
| NV-Embed V2Method category=LLM Dense Retriever, Latency (ms)=542026.04 | 47.35 | 72.06 | 5.3325 | 51.28 | 43.4 | 43.9 | |
| E5-MistralMethod category=LLM Dense Retriever, Latency (ms)=382026.04 | 45.15 | 69.99 | 5.2489 | 47.64 | 41.9 | 42.5 | |
| UtilityQwenMethod category=Multi-Stage (LLM) Rerank, Latency (ms)=31942026.04 | 42.77 | 65.35 | 5.013 | 47.9 | 38.9 | 39.3 | |
| GainRAGMethod category=Multi-Stage (LLM) Rerank, Latency (ms)=21042026.04 | 41.2 | 64.17 | 5.0329 | 50.96 | 40.7 | 41.4 | |
| BGE-BaseMethod category=Classic Retrieval, Latency (ms)=82026.04 | 34.15 | 57.37 | 4.7378 | 43.84 | 38.6 | 39 | |
| InfoNCE TunedMethod category=Classic Retrieval, Latency (ms)=92026.04 | 22.45 | 51.29 | 4.1819 | 32.6 | 25.5 | 25.9 |