Reranking on SciRAG-SSLI hard 1.0 (test)
66.6Hit Rate @ 1DeepEra
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| DeepEraSetting=Ours2026.01 | 66.6 | 76.4 | 71.96 | |
| MXBAISetting=Ours2026.01 | 62.6 | 83.2 | 63.21 | |
| DeARrerankSetting=Ours2026.01 | 61.85 | 73.13 | 62.9 | |
| ZephyrRerankerSetting=Ours2026.01 | 61.69 | 72.95 | 69.02 | |
| JinaSetting=Ours2026.01 | 60.8 | 76.2 | 68.36 | |
| BGESetting=Ours2026.01 | 59.4 | 82.6 | 71.61 | |
| TwoLARSetting=Ours2026.01 | 58.8 | 83.6 | 71.47 | |
| VicunaRerankerSetting=Ours2026.01 | 58.67 | 74.95 | 67.56 | |
| In-RankerSetting=Ours2026.01 | 53.4 | 81.4 | 67.53 | |
| LLM2VecSetting=Ours2026.01 | 53.4 | 76.4 | 65.9 | |
| RankT5Setting=Ours2026.01 | 52.6 | 78.4 | 65.95 | |
| BCESetting=Ours2026.01 | 51.4 | 73.2 | 62.91 | |
| SPLADESetting=Ours2026.01 | 51.4 | 75.6 | 64.38 | |
| RankGPTSetting=Ours2026.01 | 50.02 | 70.1 | 57.98 | |
| ColBERTSetting=Ours2026.01 | 48.8 | 75.8 | 63.24 | |
| MonoT5Setting=Ours2026.01 | 45.58 | 72.29 | 59.79 | |
| limrankSetting=Ours2026.01 | 35.14 | 57.23 | 47.63 | |
| MiniLMSetting=Ours2026.01 | 34.2 | 58.6 | 47.74 | |
| ListT5Setting=Ours2026.01 | 0.6 | 1.8 | 1.2 |