Citation Recommendation on Scientific Paper Domains Overall (test)
73.6Precision@3H-LM
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| H-LMLLM version=GPT-4o, Testing set note=Randomly tested 10% due to API limits2024.10 | 73.6 | 65.5 | 74.3 | 68.6 | |
| H-LMLLM version=GPT-3.52024.10 | 72.5 | 64.4 | 73.5 | 67.7 | |
| GTE-large-v1.52024.10 | 64.9 | 56.3 | 67.1 | 60.6 | |
| GTE-large2024.10 | 64.1 | 55.7 | 66.2 | 59.9 | |
| GTE-base2024.10 | 63.9 | 55.6 | 65.9 | 59.7 | |
| GTE-base-v1.52024.10 | 63.8 | 55.1 | 66.1 | 59.4 | |
| OpenAI-32024.10 | 63.3 | 54.5 | 65.6 | 58.9 | |
| OpenAI-ada-0022024.10 | 62.4 | 53.6 | 64.8 | 58.1 | |
| SPECTER-2.02024.10 | 60.2 | 51.5 | 62.7 | 56 | |
| SciMult-MoE2024.10 | 57.9 | 49.6 | 60.3 | 53.9 | |
| SciNCL2024.10 | 57.7 | 49.7 | 60 | 53.9 | |
| SciMult-vanilla2024.10 | 56.9 | 48.5 | 59.3 | 52.9 | |
| SPECTER2024.10 | 54.5 | 46 | 57 | 50.4 | |
| SciPATTON2024.10 | 44.7 | 37.1 | 47.2 | 41.3 | |
| Keywords overlap2024.10 | 33.6 | 26.9 | 30.4 | 26.4 | |
| PATTON2024.10 | 25.3 | 20.5 | 27.1 | 23.4 | |
| METAG2024.10 | 11.4 | 9 | 12.6 | 10.6 | |
| SciBERT2024.10 | 5.4 | 4.6 | 5.7 | 5.1 | |
| BERT-base2024.10 | 3.9 | 3.6 | 3.9 | 3.8 | |
| BERT-large2024.10 | 2.6 | 2.9 | 2.5 | 2.7 |