Citation-based retrieval on SOPHIA-BENCH tacd query v1 (test)
58.2R@10 (XY)QaECTER
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| QaECTER2026.04 | 58.2 | 87.3 | 47.3 | 48.6 | 83.1 | 37.1 | 52.9 | 84.9 | 53.8 | |
| Octen-Embed-8BParameters=8B2026.04 | 53.6 | 83.6 | 43.6 | 45.2 | 79.8 | 35 | 49.1 | 81.3 | 50.2 | |
| PaECTER2026.04 | 52.8 | 83.4 | 43.6 | 42.8 | 78.1 | 33.4 | 47.3 | 80.2 | 49.1 | |
| Octen-Embed-4BParameters=4B2026.04 | 52.7 | 83.3 | 42.8 | 44.5 | 79.7 | 34.4 | 48.2 | 81.1 | 49.3 | |
| patembed-largeModel size=large2026.04 | 51.9 | 82.4 | 42.3 | 43.2 | 77.9 | 33.4 | 47.1 | 79.7 | 48.3 | |
| modernbert-embed-baseModel size=base2026.04 | 38.3 | 68.4 | 31.5 | 32.4 | 63.9 | 25.8 | 35.2 | 65.9 | 36.8 | |
| bert-for-patents2026.04 | 27.9 | 52.1 | 23.4 | 20.8 | 45.2 | 17.2 | 24 | 48.1 | 26.1 |