Information Retrieval on HQA
69.6NDCG@10 (Dense)PromptReps
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PromptRepsProtocol=Fine-tuned, Backbone=LLaMA-3, Variant=multi-token2026.05 | 69.6 | 61.3 | 70.9 | |
| PromptRepsProtocol=Fine-tuned, Backbone=Qwen2.5, Variant=multi-token2026.05 | 68.6 | 60.1 | 70.1 | |
| PromptRepsProtocol=Fine-tuned, Backbone=LLaMA-3, Variant=single-token2026.05 | 68.5 | 59.5 | 69.5 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=Dream, Variant=multi-token2026.05 | 68.3 | 60.3 | 70.5 | |
| DiffEmbedProtocol=Fine-tuned, Backbone=Dream, Variant=single-vector2026.05 | 66.8 | — | — | |
| PromptRepsProtocol=Fine-tuned, Backbone=Qwen2.5, Variant=single-token2026.05 | 66.1 | 58 | 68.4 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=Dream, Variant=single-token2026.05 | 65 | 58.7 | 68.4 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=LLaDA, Variant=multi-token2026.05 | 64.7 | 61.3 | 68.7 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=LLaDA, Variant=single-token2026.05 | 64 | 59.5 | 67.4 | |
| DiffEmbedProtocol=Fine-tuned, Backbone=LLaDA, Variant=single-vector2026.05 | 60.1 | — | — | |
| RepLLaMAProtocol=Fine-tuned, Backbone=LLaMA, Variant=single-vector2026.05 | 50.3 | — | — | |
| DiffRetrieverProtocol=Zero-shot, Backbone=LLaDA, Variant=multi-token2026.05 | 32.8 | 46.8 | 49.4 | |
| DiffRetrieverProtocol=Zero-shot, Backbone=Dream, Variant=multi-token2026.05 | 29.4 | 41.1 | 44.9 | |
| PromptRepsProtocol=Zero-shot, Backbone=LLaMA-3, Variant=multi-token2026.05 | 28.7 | 50.8 | 50.7 | |
| PromptRepsProtocol=Zero-shot, Backbone=Qwen2.5, Variant=multi-token2026.05 | 26.7 | 48.4 | 47 | |
| PromptRepsProtocol=Zero-shot, Backbone=LLaMA-3, Variant=single-token2026.05 | 19.9 | 45.2 | 45.4 | |
| DiffEmbedProtocol=Zero-shot, Backbone=LLaDA, Variant=single-vector2026.05 | 16.7 | — | — | |
| DiffEmbedProtocol=Zero-shot, Backbone=Dream, Variant=single-vector2026.05 | 14.9 | — | — | |
| DiffRetrieverProtocol=Zero-shot, Backbone=LLaDA, Variant=single-token2026.05 | 12.9 | 32.2 | 29.9 | |
| PromptRepsProtocol=Zero-shot, Backbone=Qwen2.5, Variant=single-token2026.05 | 6.6 | 39.4 | 32.2 | |
| DiffRetrieverProtocol=Zero-shot, Backbone=Dream, Variant=single-token2026.05 | 3.8 | 5.6 | 6.6 | |
| BM25Protocol=Zero-shot, Backbone=-, Variant=sparse2026.05 | — | 56.7 | — |