Citation-based retrieval on SOPHIA-BENCH ai_feat query v1 (test)
57.7Recall@10 (XY)QaECTER
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| QaECTER2026.04 | 57.7 | 87.5 | 47.2 | 48.2 | 83.3 | 36.6 | 52.7 | 85.1 | 53.5 | |
| Octen-Embed-8BParameters=8B2026.04 | 54.6 | 84.4 | 44.5 | 45.1 | 80.4 | 34.6 | 49.4 | 82.1 | 50.5 | |
| Octen-Embed-4BParameters=4B2026.04 | 53.2 | 83.8 | 43.4 | 44.1 | 79.8 | 33.8 | 48.2 | 81.4 | 49.3 | |
| PaECTER2026.04 | 51.1 | 82.5 | 42.4 | 41.7 | 77.2 | 32.2 | 46.1 | 79.5 | 47.7 | |
| patembed-largeModel size=large2026.04 | 50.7 | 82.2 | 41.1 | 42.4 | 78 | 32.3 | 46.1 | 79.7 | 46.9 | |
| modernbert-embed-baseModel size=base2026.04 | 36.7 | 68.2 | 29.8 | 30.9 | 63.4 | 24.1 | 33.7 | 65.7 | 34.7 | |
| bert-for-patents2026.04 | 21.8 | 47.5 | 17.9 | 17.1 | 42.1 | 13.6 | 19.4 | 44.8 | 20.3 |