Information Retrieval on Scientific QA Base setting
58.66HitRate@1DeepEra
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DeepEraSetting=Ours2026.01 | 58.66 | 88.63 | 63.36 | |
| ZephyrRerankerSetting=Ours2026.01 | 56.35 | 85.58 | 60.87 | |
| JinaSetting=Ours2026.01 | 53.45 | 71.62 | 62.45 | |
| MonoT5Setting=Ours2026.01 | 53.33 | 73.78 | 58.49 | |
| VicunaRerankerSetting=Ours2026.01 | 52.53 | 83.98 | 58.45 | |
| limrankSetting=Ours2026.01 | 52.05 | 83.13 | 62.5 | |
| RankGPTSetting=Ours2026.01 | 52 | 74 | 60 | |
| DeepEraSetting=LMQG2026.01 | 50 | 71.4 | 66.34 | |
| MXBAISetting=Ours2026.01 | 48.49 | 70.61 | 62.61 | |
| DeARrerankSetting=LMQG2026.01 | 48.07 | 58.09 | 50.08 | |
| VicunaRerankerSetting=LMQG2026.01 | 47.85 | 65.08 | 60.14 | |
| RankGPTSetting=LMQG2026.01 | 47.8 | 63.8 | 56.08 | |
| MonoT5Setting=LMQG2026.01 | 47.27 | 60.48 | 58.99 | |
| RankT5Setting=LMQG2026.01 | 46.9 | 61.92 | 52.4 | |
| DeARrerankSetting=Ours2026.01 | 46.14 | 75.18 | 50.61 | |
| In-RankerSetting=LMQG2026.01 | 45.5 | 58.46 | 52.38 | |
| BGESetting=Ours2026.01 | 44.72 | 72.84 | 58.63 | |
| JinaSetting=LMQG2026.01 | 44.52 | 58.63 | 52.77 | |
| BCESetting=LMQG2026.01 | 44.51 | 59.74 | 52.24 | |
| BGESetting=LMQG2026.01 | 44.23 | 59.85 | 51.85 | |
| ZephyrRerankerSetting=LMQG2026.01 | 43.45 | 69.68 | 63.64 | |
| MiniLMSetting=LMQG2026.01 | 42.38 | 57.14 | 49.79 | |
| limrankSetting=LMQG2026.01 | 40.96 | 57.43 | 49.74 | |
| MXBAISetting=LMQG2026.01 | 39.4 | 54.75 | 47.29 | |
| LLM2VecSetting=Ours2026.01 | 39.27 | 62.37 | 52.31 | |
| ColBERTSetting=LMQG2026.01 | 38.89 | 55.11 | 49.35 | |
| TwoLARSetting=Ours2026.01 | 38.57 | 74.91 | 58.84 | |
| TwoLARSetting=LMQG2026.01 | 37.36 | 57.97 | 47.89 | |
| BCESetting=Ours2026.01 | 37.19 | 59.77 | 48.64 | |
| SPLADESetting=LMQG2026.01 | 36.83 | 54.14 | 47.33 | |
| In-RankerSetting=Ours2026.01 | 33.8 | 69.56 | 51.29 | |
| RankT5Setting=Ours2026.01 | 32.88 | 64.67 | 51.49 | |
| SPLADESetting=Ours2026.01 | 29.42 | 63.26 | 50.09 | |
| ColBERTSetting=Ours2026.01 | 28.47 | 63.62 | 48.91 | |
| MiniLMSetting=Ours2026.01 | 23.58 | 43.47 | 34.76 | |
| LLM2VecSetting=LMQG2026.01 | 15.34 | 27.22 | 22.06 | |
| ListT5Setting=LMQG2026.01 | 1.2 | 1.2 | 1.25 | |
| ListT5Setting=Ours2026.01 | 0.4 | 1.6 | 1 |