Information Retrieval on COVID
78.1nDCG@10LW-window (IQ)
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| LW-window (IQ)Query Variant=IQ, Mode=window2026.03 | 78.1 | 9.5 | — | — | — | — | — | — | |
| BAGELLLM Backbone=Qwen3-14B, Ranking Protocol=ER, LLM budget per query=502026.04 | 77.2 | — | 63.6 | 2.6 | 9.7 | — | — | — | |
| ReBOL-QR-GreedyQuery Variant=QR, Acquisition=Greedy2026.03 | 76.8 | 15.6 | — | — | — | — | — | — | |
| BAGELLLM Backbone=Qwen3-14B, Ranking Protocol=PR, LLM budget per query=502026.04 | 76.6 | — | 61.4 | 2.6 | 9.5 | — | — | — | |
| Pointwise LLMLLM Backbone=Qwen3-14B, Ranking Protocol=ER, LLM budget per query=502026.04 | 76.5 | — | 53.6 | 2.6 | 7.7 | — | — | — | |
| ReBOL-IQ-Top-kQuery Variant=IQ, Acquisition=Top-k2026.03 | 75.9 | 13.7 | — | — | — | — | — | — | |
| MedEmbedType=Black-Box2026.03 | 75.73 | — | — | — | — | — | — | — | |
| ReBOL-QR-UCBQuery Variant=QR, Acquisition=UCB2026.03 | 75.4 | 15.6 | — | — | — | — | — | — | |
| BAGELLLM Backbone=GPT-4o, Ranking Protocol=ER, LLM budget per query=502026.04 | 74.7 | — | 62.1 | 2.6 | 9.9 | — | — | — | |
| ReBOL-IQ-GreedyQuery Variant=IQ, Acquisition=Greedy2026.03 | 74.6 | 15.3 | — | — | — | — | — | — | |
| Pointwise LLMLLM Backbone=GPT-4o, Ranking Protocol=ER, LLM budget per query=502026.04 | 74.6 | — | 52.8 | 2.6 | 7.7 | — | — | — | |
| LW-one-call (IQ)Query Variant=IQ, Mode=one-call2026.03 | 74.5 | 9.5 | — | — | — | — | — | — | |
| Pointwise LLMLLM Backbone=Qwen3-14B, Ranking Protocol=PR, LLM budget per query=502026.04 | 74.2 | — | 52.9 | 2.5 | 7.7 | — | — | — | |
| RankGPTLLM Backbone=GPT-4o, Ranking Method=Listwise, LLM budget per query=502026.04 | 73.2 | — | 52.6 | 2.5 | 7.7 | — | — | — | |
| ReBOL-QR-Top-kQuery Variant=QR, Acquisition=Top-k2026.03 | 73 | 12.9 | — | — | — | — | — | — | |
| SPLADE2026.03 | 72.7 | 12.8 | — | — | — | — | — | — | |
| LW-window (QR)Query Variant=QR, Mode=window2026.03 | 72.6 | 8 | — | — | — | — | — | — | |
| Pointwise LLMLLM Backbone=GPT-4o, Ranking Protocol=PR, LLM budget per query=502026.04 | 71.9 | — | 52.4 | 2.5 | 7.7 | — | — | — | |
| RankGPTLLM Backbone=Baseline, Ranking Method=Listwise, LLM budget per query=502026.04 | 71.8 | — | 52.3 | 2.4 | 7.7 | — | — | — | |
| PW (IQ)Query Variant=IQ2026.03 | 71.3 | 9.5 | — | — | — | — | — | — | |
| ReBOL-IQ-UCBQuery Variant=IQ, Acquisition=UCB2026.03 | 70.8 | 15.4 | — | — | — | — | — | — | |
| LW-one-call (QR)Query Variant=QR, Mode=one-call2026.03 | 70.6 | 8 | — | — | — | — | — | — | |
| BAGELLLM Backbone=GPT-4o, Ranking Protocol=PR, LLM budget per query=502026.04 | 70.4 | — | 57.7 | 2.5 | 9.2 | — | — | — | |
| PW (QR)Query Variant=QR2026.03 | 66.5 | 8 | — | — | — | — | — | — | |
| Cross EncoderLLM Backbone=Baseline, LLM budget per query=502026.04 | 66.5 | — | 51.1 | 2.3 | 7.7 | — | — | — | |
| BM25 MF2026.03 | 65.6 | 11.4 | — | — | — | — | — | — | |
| QIME-TF-MMRType=Interpretable2026.03 | 64.65 | — | — | — | — | — | — | — | |
| ReBOL-QR-Rand.Query Variant=QR, Acquisition=Random2026.03 | 64 | 12.7 | — | — | — | — | — | — | |
| BM25 Flat2026.03 | 59.5 | 10.9 | — | — | — | — | — | — | |
| QIME-TFType=Interpretable2026.03 | 57.96 | — | — | — | — | — | — | — | |
| Dense RetrieverLLM Backbone=Baseline, LLM budget per query=502026.04 | 57 | — | 48.7 | 2 | 7.7 | — | — | — | |
| BioLORDType=Black-Box2026.03 | 54.89 | — | — | — | — | — | — | — | |
| MedCPTType=Black-Box2026.03 | 54.66 | — | — | — | — | — | — | — | |
| Dense (IQ)Query Variant=IQ2026.03 | 51.8 | 9.5 | — | — | — | — | — | — | |
| BM25LLM Backbone=Baseline, LLM budget per query=502026.04 | 50.4 | — | 42.8 | 1.7 | 7.3 | — | — | — | |
| EmbeddingGemmaType=Black-Box2026.03 | 50.36 | — | — | — | — | — | — | — | |
| LDIR-500Type=Interpretable2026.03 | 47.04 | — | — | — | — | — | — | — | |
| ReBOL-IQ-Rand.Query Variant=IQ, Acquisition=Random2026.03 | 46.7 | 8 | — | — | — | — | — | — | |
| QIMEType=Interpretable2026.03 | 46.31 | — | — | — | — | — | — | — | |
| Dense (QR)Query Variant=QR2026.03 | 45.4 | 8 | — | — | — | — | — | — | |
| PubMedBERTType=Black-Box2026.03 | 44.76 | — | — | — | — | — | — | — | |
| GloVeType=Black-Box2026.03 | 36.22 | — | — | — | — | — | — | — | |
| SapBERTType=Black-Box2026.03 | 33.4 | — | — | — | — | — | — | — | |
| SimCSE (Unsup)Type=Black-Box2026.03 | 32.71 | — | — | — | — | — | — | — | |
| SimCSE (Sup)Type=Black-Box2026.03 | 30.83 | — | — | — | — | — | — | — | |
| CQG-MBQAType=Interpretable2026.03 | 28.49 | — | — | — | — | — | — | — | |
| QA-EmbType=Interpretable2026.03 | 22.42 | — | — | — | — | — | — | — | |
| Bag-of-WordsType=Interpretable2026.03 | 19.23 | — | — | — | — | — | — | — | |
| BMRetrieverType=Black-Box2026.03 | 18.64 | — | — | — | — | — | — | — | |
| BERTType=Black-Box2026.03 | 14.78 | — | — | — | — | — | — | — | |
| BM25Protocol=Zero-shot, Backbone=-, Variant=sparse2026.05 | — | — | — | — | — | — | 53 | — | |
| DiffEmbedProtocol=Zero-shot, Backbone=Dream, Variant=single-vector2026.05 | — | — | — | — | — | 31.6 | — | — | |
| DiffEmbedProtocol=Zero-shot, Backbone=LLaDA, Variant=single-vector2026.05 | — | — | — | — | — | 40.7 | — | — | |
| DiffEmbedProtocol=Fine-tuned, Backbone=Dream, Variant=single-vector2026.05 | — | — | — | — | — | 75.8 | — | — | |
| DiffEmbedProtocol=Fine-tuned, Backbone=LLaDA, Variant=single-vector2026.05 | — | — | — | — | — | 57.8 | — | — | |
| DiffRetrieverProtocol=Zero-shot, Backbone=Dream, Variant=single-token2026.05 | — | — | — | — | — | 50.1 | 33.9 | 49.7 | |
| DiffRetrieverProtocol=Zero-shot, Backbone=Dream, Variant=multi-token2026.05 | — | — | — | — | — | 62.2 | 48.1 | 59 | |
| DiffRetrieverProtocol=Zero-shot, Backbone=LLaDA, Variant=single-token2026.05 | — | — | — | — | — | 67.9 | 66.6 | 75.2 | |
| DiffRetrieverProtocol=Zero-shot, Backbone=LLaDA, Variant=multi-token2026.05 | — | — | — | — | — | 66.2 | 69 | 71.7 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=Dream, Variant=single-token2026.05 | — | — | — | — | — | 84.1 | 62.9 | 78.9 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=Dream, Variant=multi-token2026.05 | — | — | — | — | — | 84.7 | 66.5 | 83 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=LLaDA, Variant=single-token2026.05 | — | — | — | — | — | 84 | 69.1 | 82.3 | |
| DiffRetrieverProtocol=Fine-tuned, Backbone=LLaDA, Variant=multi-token2026.05 | — | — | — | — | — | 84.6 | 71 | 81.9 | |
| PromptRepsProtocol=Zero-shot, Backbone=Qwen2.5, Variant=single-token2026.05 | — | — | — | — | — | 49 | 54.5 | 57.7 | |
| PromptRepsProtocol=Zero-shot, Backbone=Qwen2.5, Variant=multi-token2026.05 | — | — | — | — | — | 41.1 | 51.3 | 53.8 | |
| PromptRepsProtocol=Zero-shot, Backbone=LLaMA-3, Variant=single-token2026.05 | — | — | — | — | — | 61.2 | 55.8 | 64.5 | |
| PromptRepsProtocol=Zero-shot, Backbone=LLaMA-3, Variant=multi-token2026.05 | — | — | — | — | — | 53 | 55.2 | 61.3 | |
| PromptRepsProtocol=Fine-tuned, Backbone=Qwen2.5, Variant=single-token2026.05 | — | — | — | — | — | 84.3 | 63.2 | 79.7 | |
| PromptRepsProtocol=Fine-tuned, Backbone=Qwen2.5, Variant=multi-token2026.05 | — | — | — | — | — | 84.8 | 66.4 | 81.8 | |
| PromptRepsProtocol=Fine-tuned, Backbone=LLaMA-3, Variant=single-token2026.05 | — | — | — | — | — | 81.5 | 59.6 | 75.6 | |
| PromptRepsProtocol=Fine-tuned, Backbone=LLaMA-3, Variant=multi-token2026.05 | — | — | — | — | — | 80.9 | 65.9 | 79.2 | |
| RepLLaMAProtocol=Fine-tuned, Backbone=LLaMA, Variant=single-vector2026.05 | — | — | — | — | — | 56.4 | — | — |