Information Retrieval on SciFact (test)
0.906NDCG@10Ours
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| OursBackbone=GritLM, Training Protocol=supervised with relevance labels2026.02 | 0.906 | 0.4 | — | — | — | |
| Ours@30%Backbone=GritLM, Training Protocol=supervised with relevance labels, Fixed Dimension Ratio=30%2026.02 | 0.902 | 0.3 | — | — | — | |
| OursBackbone=Qwen-4B, Training Protocol=supervised with relevance labels2026.02 | 0.899 | 0.56 | — | — | — | |
| OursBackbone=OpenAI, Training Protocol=supervised with relevance labels2026.02 | 0.897 | 0.32 | — | — | — | |
| Ours@30%Backbone=OpenAI, Training Protocol=supervised with relevance labels, Fixed Dimension Ratio=30%2026.02 | 0.897 | 0.3 | — | — | — | |
| Ours@30%Backbone=Qwen-4B, Training Protocol=supervised with relevance labels, Fixed Dimension Ratio=30%2026.02 | 0.895 | 0.3 | — | — | — | |
| OursBackbone=L2V-LLaMA, Training Protocol=supervised with relevance labels2026.02 | 0.884 | 0.1 | — | — | — | |
| AdapterBackbone=Qwen-8B, Training Protocol=supervised with relevance labels2026.02 | 0.883 | 1 | — | — | — | |
| AdapterBackbone=GritLM, Training Protocol=supervised with relevance labels2026.02 | 0.883 | 1 | — | — | — | |
| OursBackbone=Qwen-8B, Training Protocol=supervised with relevance labels2026.02 | 0.883 | 0.32 | — | — | — | |
| AdapterBackbone=L2V-LLaMA, Training Protocol=supervised with relevance labels2026.02 | 0.882 | 1 | — | — | — | |
| Ours@30%Backbone=Qwen-8B, Training Protocol=supervised with relevance labels, Fixed Dimension Ratio=30%2026.02 | 0.881 | 0.3 | — | — | — | |
| Ours@30%Backbone=L2V-LLaMA, Training Protocol=supervised with relevance labels, Fixed Dimension Ratio=30%2026.02 | 0.881 | 0.3 | — | — | — | |
| AdapterBackbone=OpenAI, Training Protocol=supervised with relevance labels2026.02 | 0.871 | 1 | — | — | — | |
| AdapterBackbone=Qwen-4B, Training Protocol=supervised with relevance labels2026.02 | 0.849 | 1 | — | — | — | |
| OursBackbone=Qwen-0.6B, Training Protocol=supervised with relevance labels2026.02 | 0.845 | 0.32 | — | — | — | |
| AdapterBackbone=L2V-Mistral, Training Protocol=supervised with relevance labels2026.02 | 0.843 | 1 | — | — | — | |
| Ours@30%Backbone=Qwen-0.6B, Training Protocol=supervised with relevance labels, Fixed Dimension Ratio=30%2026.02 | 0.839 | 0.3 | — | — | — | |
| OursBackbone=L2V-Mistral, Training Protocol=supervised with relevance labels2026.02 | 0.83 | 0.2 | — | — | — | |
| Ours@30%Backbone=L2V-Mistral, Training Protocol=supervised with relevance labels, Fixed Dimension Ratio=30%2026.02 | 0.823 | 0.3 | — | — | — | |
| CutoffBackbone=GritLM, Training Protocol=fully unsupervised2026.02 | 0.792 | 0.6 | — | — | — | |
| NormBackbone=GritLM, Training Protocol=fully unsupervised2026.02 | 0.791 | 0.28 | — | — | — | |
| NormBackbone=Qwen-4B, Training Protocol=fully unsupervised2026.02 | 0.789 | 0.4 | — | — | — | |
| CutoffBackbone=Qwen-4B, Training Protocol=fully unsupervised2026.02 | 0.788 | 0.86 | — | — | — | |
| CutoffBackbone=L2V-LLaMA, Training Protocol=fully unsupervised2026.02 | 0.788 | 0.84 | — | — | — | |
| DIMEBackbone=L2V-LLaMA, Training Protocol=PRF-based2026.02 | 0.788 | 0.98 | — | — | — | |
| EclipseBackbone=L2V-LLaMA, Training Protocol=PRF-based2026.02 | 0.788 | 0.98 | — | — | — | |
| BaselineBackbone=L2V-LLaMA, Training Protocol=fully unsupervised2026.02 | 0.787 | 1 | — | — | — | |
| NormBackbone=L2V-LLaMA, Training Protocol=fully unsupervised2026.02 | 0.787 | 0.92 | — | — | — | |
| DIMEBackbone=Qwen-4B, Training Protocol=PRF-based2026.02 | 0.787 | 0.94 | — | — | — | |
| DIMEBackbone=GritLM, Training Protocol=PRF-based2026.02 | 0.787 | 0.98 | — | — | — | |
| EclipseBackbone=Qwen-4B, Training Protocol=PRF-based2026.02 | 0.787 | 0.98 | — | — | — | |
| EclipseBackbone=GritLM, Training Protocol=PRF-based2026.02 | 0.787 | 0.98 | — | — | — | |
| BaselineBackbone=GritLM, Training Protocol=fully unsupervised2026.02 | 0.786 | 1 | — | — | — | |
| BaselineBackbone=Qwen-4B, Training Protocol=fully unsupervised2026.02 | 0.785 | 1 | — | — | — | |
| NormBackbone=Qwen-8B, Training Protocol=fully unsupervised2026.02 | 0.785 | 0.32 | — | — | — | |
| DIMEBackbone=Qwen-8B, Training Protocol=PRF-based2026.02 | 0.785 | 0.94 | — | — | — | |
| EclipseBackbone=Qwen-8B, Training Protocol=PRF-based2026.02 | 0.785 | 0.94 | — | — | — | |
| CutoffBackbone=Qwen-8B, Training Protocol=fully unsupervised2026.02 | 0.784 | 0.98 | — | — | — | |
| BaselineBackbone=Qwen-8B, Training Protocol=fully unsupervised2026.02 | 0.783 | 1 | — | — | — | |
| EclipseBackbone=OpenAI, Training Protocol=PRF-based2026.02 | 0.783 | 0.94 | — | — | — | |
| DIMEBackbone=OpenAI, Training Protocol=PRF-based2026.02 | 0.781 | 0.82 | — | — | — | |
| NormBackbone=OpenAI, Training Protocol=fully unsupervised2026.02 | 0.778 | 0.8 | — | — | — | |
| BaselineBackbone=OpenAI, Training Protocol=fully unsupervised2026.02 | 0.776 | 1 | — | — | — | |
| CutoffBackbone=OpenAI, Training Protocol=fully unsupervised2026.02 | 0.776 | 0.98 | — | — | — | |
| NormBackbone=L2V-Mistral, Training Protocol=fully unsupervised2026.02 | 0.774 | 0.62 | — | — | — | |
| AdapterBackbone=Qwen-0.6B, Training Protocol=supervised with relevance labels2026.02 | 0.774 | 1 | — | — | — | |
| BaselineBackbone=L2V-Mistral, Training Protocol=fully unsupervised2026.02 | 0.773 | 1 | — | — | — | |
| CutoffBackbone=L2V-Mistral, Training Protocol=fully unsupervised2026.02 | 0.773 | 1 | — | — | — | |
| DIMEBackbone=L2V-Mistral, Training Protocol=PRF-based2026.02 | 0.773 | 1 | — | — | — | |
| EclipseBackbone=L2V-Mistral, Training Protocol=PRF-based2026.02 | 0.773 | 1 | — | — | — | |
| Our ModelRetriever Type=Inference-free Sparse Retriever2024.11 | 0.729 | — | — | — | — | |
| SPLADE-doc-distillRetriever Type=Inference-free Sparse Retriever2024.11 | 0.708 | — | — | — | — | |
| NormBackbone=Qwen-0.6B, Training Protocol=fully unsupervised2026.02 | 0.706 | 0.64 | — | — | — | |
| DIMEBackbone=Qwen-0.6B, Training Protocol=PRF-based2026.02 | 0.705 | 0.98 | — | — | — | |
| EclipseBackbone=Qwen-0.6B, Training Protocol=PRF-based2026.02 | 0.704 | 0.98 | — | — | — | |
| BaselineBackbone=Qwen-0.6B, Training Protocol=fully unsupervised2026.02 | 0.702 | 1 | — | — | — | |
| CutoffBackbone=Qwen-0.6B, Training Protocol=fully unsupervised2026.02 | 0.702 | 1 | — | — | — | |
| SPLADE++-SelfDistilRetriever Type=Sparse Retriever2024.11 | 0.693 | — | — | — | — | |
| ColBERTv2Retriever Type=Dense Retriever2024.11 | 0.693 | — | — | — | — | |
| BM25Retriever Type=Inference-free Sparse Retriever2024.11 | 0.69 | — | — | — | — | |
| SPLADE-v3-DocRetriever Type=Inference-free Sparse Retriever2024.11 | 0.688 | — | — | — | — | |
| SPLADE-v3-DistilRetriever Type=Sparse Retriever2024.11 | 0.685 | — | — | — | — | |
| ContrieverRetriever Type=Dense Retriever2024.11 | 0.677 | — | — | — | — | |
| TAS-BRetriever Type=Dense Retriever2024.11 | 0.643 | — | — | — | — | |
| Best FKN=5,1832026.05 | — | — | 85.6 | — | — | |
| BoRN=5,1832026.05 | — | — | 78.9 | 7.2 | 9.76 | |
| F1N=5,1832026.05 | — | — | 66.7 | 5 | — |