Information Retrieval on SciFact (nDCG@10, Overlap%, Spearman)
0.77nDCG@10Naive
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| NaiveRetriever Model=Jasper, Embedding Dimension=512d2026.04 | 0.77 | 100 | 1 | |
| NaiveRetriever Model=BGE, Embedding Dimension=768d2026.04 | 0.747 | 100 | 1 | |
| Trans-RAGRetriever Model=Jasper, Embedding Dimension=512d2026.04 | 0.745 | 92.2 | 0.747 | |
| NaiveRetriever Model=UAE, Embedding Dimension=1024d2026.04 | 0.743 | 100 | 1 | |
| NaiveRetriever Model=Ember, Embedding Dimension=1024d2026.04 | 0.737 | 100 | 1 | |
| NaiveRetriever Model=Linq, Embedding Dimension=4096d2026.04 | 0.725 | 100 | 1 | |
| Trans-RAGRetriever Model=BGE, Embedding Dimension=768d2026.04 | 0.721 | 92.7 | 0.791 | |
| Trans-RAGRetriever Model=UAE, Embedding Dimension=1024d2026.04 | 0.716 | 93.2 | 0.843 | |
| Trans-RAGRetriever Model=Ember, Embedding Dimension=1024d2026.04 | 0.712 | 93.2 | 0.844 | |
| Trans-RAGRetriever Model=Linq, Embedding Dimension=4096d2026.04 | 0.701 | 93.2 | 0.841 | |
| NaiveRetriever Model=MPNet, Embedding Dimension=768d2026.04 | 0.634 | 100 | 1 | |
| Trans-RAGRetriever Model=MPNet, Embedding Dimension=768d2026.04 | 0.613 | 92.9 | 0.809 | |
| NaiveRetriever Model=Stella, Embedding Dimension=8192d2026.04 | 0.531 | 100 | 1 | |
| Trans-RAGRetriever Model=Stella, Embedding Dimension=8192d2026.04 | 0.513 | 94.4 | 0.95 | |
| NaiveRetriever Model=GTE, Embedding Dimension=1024d2026.04 | 0.354 | 100 | 1 | |
| Trans-RAGRetriever Model=GTE, Embedding Dimension=1024d2026.04 | 0.343 | 93.4 | 0.857 |