Scientific Paper Retrieval on SCIFULLBENCH ACL-EMNLP (Citations) 1.0
36.87nDCG@300COR
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| CORRetrieval Protocol=COR, Backbone Retriever=Inf-Retriever-v1-1.5B, LLM Engine=QWEN-2.5-3B-Instruct2025.07 | 36.87 | 53.59 | |
| CORRetrieval Protocol=COR, Backbone Retriever=Inf-Retriever-v1-1.5B, LLM Engine=Llama-3.2-3B-Instruct2025.07 | 36.18 | 52.95 | |
| CORRetrieval Protocol=COR, Backbone Retriever=Jina-Embeddings-v2-BASE-EN, LLM Engine=QWEN-2.5-3B-Instruct2025.07 | 32.54 | 48.3 | |
| CORRetrieval Protocol=COR, Backbone Retriever=Jina-Embeddings-v2-BASE-EN, LLM Engine=Llama-3.2-3B-Instruct2025.07 | 31.8 | 47.03 | |
| Inf-Retriever-v1-1.5BRetrieval Protocol=Abstract-to-Abstract (A2A)2025.07 | 30.94 | 45.53 | |
| Inf-Retriever-v1-1.5BRetrieval Protocol=Full-to-Full (F2F)2025.07 | 30.31 | 43.64 | |
| CORRetrieval Protocol=COR, Backbone Retriever=BGE-M3, LLM Engine=QWEN-2.5-3B-Instruct2025.07 | 29.33 | 43.72 | |
| CORRetrieval Protocol=COR, Backbone Retriever=BGE-M3, LLM Engine=Llama-3.2-3B-Instruct2025.07 | 28.87 | 42.67 | |
| Jina-Embeddings-v2-BASE-ENRetrieval Protocol=Full-to-Full (F2F)2025.07 | 28.34 | 40.65 | |
| BM-25Retrieval Protocol=Full-to-Full (F2F)2025.07 | 28.08 | 38.58 | |
| SPECTER2-BaseRetrieval Protocol=Abstract-to-Abstract (A2A)2025.07 | 26.85 | 40.21 | |
| Jina-Embeddings-v2-BASE-ENRetrieval Protocol=Abstract-to-Abstract (A2A)2025.07 | 26.45 | 38.92 | |
| SciNCLRetrieval Protocol=Abstract-to-Abstract (A2A)2025.07 | 25.58 | 38.91 | |
| SPECTER2-Adapter-MTL CTRLRetrieval Protocol=Abstract-to-Abstract (A2A)2025.07 | 25.48 | 38.54 | |
| BGE-M3Retrieval Protocol=Full-to-Full (F2F)2025.07 | 24.85 | 35.87 | |
| BGE-M3Retrieval Protocol=Abstract-to-Abstract (A2A)2025.07 | 24.41 | 36.28 | |
| SciMult-MHAExpertRetrieval Protocol=Abstract-to-Abstract (A2A)2025.07 | 22.57 | 33.79 | |
| BM-25Retrieval Protocol=Abstract-to-Abstract (A2A)2025.07 | 20.86 | 29.37 |