Information Retrieval on SciFact (NDCG@10 Variants)
77.3NDCG@10 (Dense)PromptReps
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PromptRepsProtocol=Fine-tuned, Backbone=Qwen2.5, Variant=multi-token2026.05 | 77.3 | 68.1 | 74.8 | |
| PromptRepsProtocol=Fine-tuned, Backbone=Qwen2.5, Variant=single-token2026.05 | 75.6 | 66.2 | 72.7 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=Dream, Variant=multi-token2026.05 | 75.2 | 66.6 | 72.9 | |
| PromptRepsProtocol=Fine-tuned, Backbone=LLaMA-3, Variant=single-token2026.05 | 75 | 65.9 | 73.1 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=LLaDA, Variant=multi-token2026.05 | 74.4 | 69.5 | 74.6 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=Dream, Variant=single-token2026.05 | 73.9 | 67 | 72.9 | |
| DiffEmbedProtocol=Fine-tuned, Backbone=Dream, Variant=single-vector2026.05 | 73.5 | — | — | |
| PromptRepsProtocol=Fine-tuned, Backbone=LLaMA-3, Variant=multi-token2026.05 | 73.5 | 67.3 | 73.7 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=LLaDA, Variant=single-token2026.05 | 73.3 | 68.1 | 74.3 | |
| DiffEmbedProtocol=Fine-tuned, Backbone=LLaDA, Variant=single-vector2026.05 | 71.4 | — | — | |
| RepLLaMAProtocol=Fine-tuned, Backbone=LLaMA, Variant=single-vector2026.05 | 65.3 | — | — | |
| DiffRetrieverProtocol=Zero-shot, Backbone=LLaDA, Variant=multi-token2026.05 | 59.2 | 62.4 | 66 | |
| DiffRetrieverProtocol=Zero-shot, Backbone=Dream, Variant=multi-token2026.05 | 55.4 | 56.4 | 63.6 | |
| PromptRepsProtocol=Zero-shot, Backbone=LLaMA-3, Variant=multi-token2026.05 | 53.9 | 60.5 | 63.9 | |
| DiffRetrieverProtocol=Zero-shot, Backbone=LLaDA, Variant=single-token2026.05 | 52.5 | 57.6 | 61.8 | |
| PromptRepsProtocol=Zero-shot, Backbone=LLaMA-3, Variant=single-token2026.05 | 51.7 | 58.4 | 64.3 | |
| PromptRepsProtocol=Zero-shot, Backbone=Qwen2.5, Variant=multi-token2026.05 | 48.9 | 59.5 | 60.9 | |
| PromptRepsProtocol=Zero-shot, Backbone=Qwen2.5, Variant=single-token2026.05 | 45.5 | 57 | 61.8 | |
| DiffEmbedProtocol=Zero-shot, Backbone=LLaDA, Variant=single-vector2026.05 | 38.5 | — | — | |
| DiffEmbedProtocol=Zero-shot, Backbone=Dream, Variant=single-vector2026.05 | 38.3 | — | — | |
| DiffRetrieverProtocol=Zero-shot, Backbone=Dream, Variant=single-token2026.05 | 33.4 | 27.6 | 38 | |
| BM25Protocol=Zero-shot, Backbone=-, Variant=sparse2026.05 | — | 66.4 | — |