Academic Paper Retrieval on AutoScholarQuery (test)
70.99Recall@100PaSa-7b ensemble
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| PaSa-7b ensemblebackbone=7b, mode=ensemble, sampling decoding=2 runs2025.01 | 70.99 | 82.65 | 14.1 | 49.85 | 63.86 | 53.26 | |
| PaSa-7bbackbone=7b, exploration depth=32025.01 | 69.47 | 79.31 | 14.48 | 48.34 | 63.34 | 53.01 | |
| Google with GPT-4oplatform=Google Search, post-processor=GPT-4o2025.01 | 26.83 | — | — | — | 24.5 | 19.21 | |
| Googleplatform=Google Search2025.01 | 20.15 | — | — | — | 18.91 | 15.68 | |
| Google Scholarplatform=Google Scholar2025.01 | 11.3 | — | — | — | 9.7 | 6.09 | |
| ChatGPTevaluation_subset=100 randomly sampled instances2025.01 | — | — | 5.07 | 30.46 | — | — | |
| GPT-o1model=GPT-o12025.01 | — | — | 4.13 | 19.25 | — | — | |
| PaSa-GPT-4obackbone=GPT-4o, exploration depth=32025.01 | — | 75.65 | 14.57 | 38.73 | — | — |