Reranking on SciRAG-SSLI easy 1.0 (test)
56.2Hit Rate @ 1DeepEra
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DeepEraSetting=LMQG2026.01 | 56.2 | 66.6 | 63.65 | |
| RankT5Setting=LMQG2026.01 | 55.4 | 65.4 | 60.7 | |
| In-RankerSetting=LMQG2026.01 | 54.4 | 65.4 | 60.33 | |
| BCESetting=LMQG2026.01 | 53.8 | 64.8 | 59.73 | |
| BGESetting=LMQG2026.01 | 53.6 | 65.2 | 59.53 | |
| DeARrerankSetting=LMQG2026.01 | 53.01 | 63.65 | 58.7 | |
| JinaSetting=LMQG2026.01 | 52.8 | 64.2 | 59.04 | |
| MiniLMSetting=LMQG2026.01 | 52.6 | 63.6 | 58.38 | |
| ColBERTSetting=LMQG2026.01 | 51.8 | 63.4 | 58.15 | |
| MonoT5Setting=LMQG2026.01 | 51.41 | 64.06 | 57.66 | |
| ZephyrRerankerSetting=LMQG2026.01 | 50.8 | 61.24 | 56.39 | |
| TwoLARSetting=LMQG2026.01 | 48.8 | 64.2 | 56.76 | |
| SPLADESetting=LMQG2026.01 | 48.6 | 62.8 | 56.07 | |
| MXBAISetting=LMQG2026.01 | 47.8 | 62.2 | 55.35 | |
| RankGPTSetting=LMQG2026.01 | 47.2 | 62.2 | 54.64 | |
| VicunaRerankerSetting=LMQG2026.01 | 45.18 | 59.24 | 52.15 | |
| LLM2VecSetting=LMQG2026.01 | 20.4 | 39.4 | 32.86 | |
| limrankSetting=LMQG2026.01 | 14.86 | 32.93 | 25.95 | |
| ListT5Setting=LMQG2026.01 | 0.2 | 0.2 | 0.25 |