Scientific Question Answering on SciRAG-SSLI hard 1.0 (test)
46.86F1 ScoreBCE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| BCESetting=Ours2026.01 | 46.86 | 39.33 | 53.05 | 3.31 | |
| BGESetting=Ours2026.01 | 46.4 | 40.98 | 53.4 | 3.4 | |
| TwoLARSetting=Ours2026.01 | 46.01 | 38.61 | 53.76 | 3.43 | |
| RankGPTSetting=Ours2026.01 | 45.92 | 40.02 | 52.88 | 3.84 | |
| MXBAISetting=Ours2026.01 | 45.79 | 39.76 | 53.5 | 3.55 | |
| RankT5Setting=Ours2026.01 | 45.48 | 40.14 | 52.72 | 3.37 | |
| ColBERTSetting=Ours2026.01 | 45.39 | 39.43 | 52.52 | 3.32 | |
| In-RankerSetting=Ours2026.01 | 45.27 | 38.75 | 53.29 | 3.38 | |
| MiniLMSetting=Ours2026.01 | 45.25 | 38.91 | 51.43 | 3.2 | |
| SPLADESetting=Ours2026.01 | 45.07 | 38.74 | 51.4 | 3.28 | |
| LLM2VecSetting=Ours2026.01 | 45.02 | 38.69 | 52.36 | 3.34 | |
| JinaSetting=Ours2026.01 | 44.93 | 43.61 | 48.39 | 3.61 | |
| DeepEraSetting=Ours2026.01 | 43.38 | 42.97 | 60.54 | 3.94 | |
| VicunaRerankerSetting=Ours2026.01 | 41.12 | 41.49 | 54.38 | 3.65 | |
| ZephyrRerankerSetting=Ours2026.01 | 41 | 40.63 | 54.41 | 3.59 | |
| DeARrerankSetting=Ours2026.01 | 39.49 | 39.35 | 54.04 | 3.51 | |
| MonoT5Setting=Ours2026.01 | 38.95 | 39.08 | 51.85 | 3.32 | |
| limrankSetting=Ours2026.01 | 37.44 | 36.73 | 52.6 | 3.46 | |
| ListT5Setting=Ours2026.01 | 21.34 | 23.97 | 26.52 | 1.81 |