End-to-End Performance on Clipper
47.7Top-3 RecallQRhead+ReAttn
Evaluation results
| Method | Links | ||
|---|---|---|---|
| QRhead+ReAttnBase LLM=Llama-3.1-8B-Instruct2026.02 | 47.7 | 42 | |
| QRheadBase LLM=Llama-3.1-8B-Instruct2026.02 | 47.6 | 41.9 | |
| ICR+ReAttnBase LLM=Llama-3.1-8B-Instruct2026.02 | 43.9 | 42.8 | |
| ICRBase LLM=Llama-3.1-8B-Instruct2026.02 | 43.8 | 42.5 | |
| StellaBase LLM=Llama-3.1-8B-Instruct2026.02 | 38.8 | 39.6 | |
| BM25Base LLM=Llama-3.1-8B-Instruct2026.02 | 37.9 | 37.9 | |
| Full contextBase LLM=Llama-3.1-8B-Instruct2026.02 | 31.3 | — | |
| ContrieverBase LLM=Llama-3.1-8B-Instruct2026.02 | 28.2 | 31.1 | |
| Full contextBase LLM=Llama-3.2-3B-Instruct2026.02 | 25.2 | — | |
| QRheadBase LLM=Llama-3.2-3B-Instruct2026.02 | 23.4 | 26.9 | |
| QRhead+ReAttnBase LLM=Llama-3.2-3B-Instruct2026.02 | 23.4 | 27.2 | |
| StellaBase LLM=Llama-3.2-3B-Instruct2026.02 | 21.3 | 25.1 | |
| BM25Base LLM=Llama-3.2-3B-Instruct2026.02 | 20 | 22.8 | |
| RankGPTBubbleBase LLM=Llama-3.1-8B-Instruct2026.02 | 19.7 | 20.4 | |
| ICRBase LLM=Llama-3.2-3B-Instruct2026.02 | 19.4 | 23.6 | |
| ICR+ReAttnBase LLM=Llama-3.2-3B-Instruct2026.02 | 19.4 | 23.8 | |
| RankGPTw/oBase LLM=Llama-3.1-8B-Instruct2026.02 | 15.9 | 19.4 | |
| ContrieverBase LLM=Llama-3.2-3B-Instruct2026.02 | 12.6 | 18.4 | |
| RankGPTBubbleBase LLM=Llama-3.2-3B-Instruct2026.02 | 3.8 | 8.8 | |
| RankGPTw/oBase LLM=Llama-3.2-3B-Instruct2026.02 | 3.6 | 8.8 |