Information Retrieval on ArguAna (NDCG Variants)
41.6NDCG@10 (Dense)PromptReps
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PromptRepsProtocol=Fine-tuned, Backbone=LLaMA-3, Variant=multi-token2026.05 | 41.6 | 29 | 39 | |
| PromptRepsProtocol=Fine-tuned, Backbone=LLaMA-3, Variant=single-token2026.05 | 41.4 | 31.1 | 39.1 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=LLaDA, Variant=single-token2026.05 | 41.4 | 29.4 | 39.2 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=LLaDA, Variant=multi-token2026.05 | 41.2 | 29.1 | 38.9 | |
| RepLLaMAProtocol=Fine-tuned, Backbone=LLaMA, Variant=single-vector2026.05 | 41.1 | — | — | |
| PromptRepsProtocol=Fine-tuned, Backbone=Qwen2.5, Variant=single-token2026.05 | 40.7 | 30 | 38.6 | |
| PromptRepsProtocol=Fine-tuned, Backbone=Qwen2.5, Variant=multi-token2026.05 | 40.7 | 28.7 | 38.1 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=Dream, Variant=single-token2026.05 | 40.6 | 28.8 | 38.7 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=Dream, Variant=multi-token2026.05 | 40.3 | 30.5 | 38.2 | |
| DiffEmbedProtocol=Fine-tuned, Backbone=LLaDA, Variant=single-vector2026.05 | 38.7 | — | — | |
| DiffRetrieverProtocol=Zero-shot, Backbone=LLaDA, Variant=multi-token2026.05 | 38.6 | 17.9 | 35.7 | |
| DiffRetrieverProtocol=Zero-shot, Backbone=LLaDA, Variant=single-token2026.05 | 37.9 | 23.7 | 37 | |
| DiffEmbedProtocol=Fine-tuned, Backbone=Dream, Variant=single-vector2026.05 | 37.5 | — | — | |
| DiffRetrieverProtocol=Zero-shot, Backbone=Dream, Variant=multi-token2026.05 | 35 | 13.7 | 32.3 | |
| DiffEmbedProtocol=Zero-shot, Backbone=LLaDA, Variant=single-vector2026.05 | 32.6 | — | — | |
| PromptRepsProtocol=Zero-shot, Backbone=LLaMA-3, Variant=multi-token2026.05 | 31.5 | 13.3 | 31.6 | |
| DiffEmbedProtocol=Zero-shot, Backbone=Dream, Variant=single-vector2026.05 | 30.8 | — | — | |
| DiffRetrieverProtocol=Zero-shot, Backbone=Dream, Variant=single-token2026.05 | 29 | 13.9 | 27.3 | |
| PromptRepsProtocol=Zero-shot, Backbone=Qwen2.5, Variant=multi-token2026.05 | 23.1 | 12.1 | 24 | |
| PromptRepsProtocol=Zero-shot, Backbone=LLaMA-3, Variant=single-token2026.05 | 22.9 | 17.3 | 24.8 | |
| PromptRepsProtocol=Zero-shot, Backbone=Qwen2.5, Variant=single-token2026.05 | 18.9 | 17.9 | 20.9 | |
| BM25Protocol=Zero-shot, Backbone=-, Variant=sparse2026.05 | — | 27.5 | — |