Reviewer Assignment on LR-Bench
0.1795Loss (LR-PC)RATE
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| RATEProfiling LLM=Qwen3-Max, Embedding Model=RATE-8B2026.01 | 0.1795 | 0.1926 | 0.1991 | 0.1904 | 76.89 | 75.25 | 80.09 | 77.41 | |
| RATEProfiling LLM=GLM-4.6, Embedding Model=RATE-8B2026.01 | 0.1875 | 0.1895 | 0.2125 | 0.1965 | 75.94 | 75.51 | 78.05 | 76.78 | |
| SPECTER2 PRXMethod Category=Embedding-based, Pooling strategy=Best performing2026.01 | 0.1902 | 0.2176 | 0.2144 | 0.2074 | 74.06 | 72.89 | 78.56 | 75.17 | |
| RATEProfiling LLM=Qwen3-Max, Embedding Model=RATE-0.6B2026.01 | 0.1955 | 0.2035 | 0.2378 | 0.2123 | 75.47 | 74.32 | 76.22 | 75.34 | |
| RATEProfiling LLM=GLM-4.6, Embedding Model=RATE-0.6B2026.01 | 0.2008 | 0.1989 | 0.235 | 0.2116 | 74.53 | 74.83 | 76.5 | 75.29 | |
| SPECTERMethod Category=Embedding-based, Pooling strategy=Best performing2026.01 | 0.2048 | 0.2171 | 0.2672 | 0.2297 | 73.58 | 73.31 | 73.28 | 73.39 | |
| SciNCLMethod Category=Embedding-based, Pooling strategy=Best performing2026.01 | 0.2354 | 0.2114 | 0.2141 | 0.2203 | 69.34 | 73.9 | 78.59 | 73.64 | |
| TPMSMethod Category=Statistical-based2026.01 | 0.2646 | 0.2333 | 0.2811 | 0.2597 | 70.28 | 72.3 | 71.89 | 71.49 | |
| Qwen3-maxMethod Category=LLM-based2026.01 | 0.2713 | 0.2289 | 0.2246 | 0.2416 | 47.17 | 55.32 | 77.54 | 60.01 | |
| DeepSeek-V3.2Method Category=LLM-based2026.01 | 0.2779 | 0.2351 | 0.2237 | 0.2456 | 50 | 53.89 | 77.36 | 60.42 | |
| BERTScoreMethod Category=Embedding-based, Pooling strategy=Best performing2026.01 | 0.2846 | 0.339 | 0.3216 | 0.3153 | 65.57 | 62.08 | 67.84 | 65.16 | |
| CoFMethod Category=Embedding-based, Pooling strategy=Best performing2026.01 | 0.2939 | 0.2218 | 0.2564 | 0.2574 | 65.57 | 73.31 | 74.36 | 71.08 | |
| ACLMethod Category=Embedding-based, Pooling strategy=Best performing2026.01 | 0.3338 | 0.3038 | 0.3163 | 0.318 | 61.32 | 65.96 | 68.37 | 65.22 | |
| SciBERTMethod Category=Embedding-based, Pooling strategy=Best performing2026.01 | 0.4016 | 0.441 | 0.3505 | 0.3977 | 57.55 | 55.15 | 64.95 | 59.22 |