Large Model Performance Prediction on OpenCompass 95% masking Sep 30 2024 (temporal split)
8.75RMSESTAR
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| STARMasking level=95%, Number of runs=5, backbone=GPT-5.12026.02 | 8.75 | 5.69 | 7.22 | 96.1 | 88.64 | 38.23 | 74.32 | 67.1 | |
| Semantic-Augmented CPMFMasking level=95%, Number of runs=52026.02 | 9.95 | 6.47 | 8.21 | 95.27 | 87.56 | 33.94 | 72.26 | 64.05 | |
| PMFMasking level=95%, Number of runs=52026.02 | 12.82 | 8.18 | 10.5 | 92.46 | 84.42 | 30.48 | 69.12 | 58.62 | |
| Mean Of MeansMasking level=95%, Number of runs=52026.02 | 12.93 | 9.74 | 11.33 | 92.28 | 83.76 | 33.74 | 69.93 | 58.6 | |
| RegressionMasking level=95%, Number of runs=52026.02 | 13.01 | 8.98 | 11 | 91.7 | 83.58 | 38.04 | 71.11 | 60.11 | |
| NCFMasking level=95%, Number of runs=52026.02 | 13.13 | 8.95 | 11.04 | 90.44 | 81.8 | 22.45 | 64.9 | 53.86 | |
| LLM-RAGMasking level=95%, Number of runs=5, backbone=GPT-5.12026.02 | 14.64 | 9.25 | 11.94 | 76.65 | 61.06 | 27.35 | 55.02 | 43.08 | |
| PCAMasking level=95%, Number of runs=52026.02 | 15.33 | 11.82 | 13.58 | 86.86 | 78.04 | 14.99 | 59.96 | 46.38 | |
| Global MeanMasking level=95%, Number of runs=52026.02 | 16.19 | 13.01 | 14.6 | 85.61 | 77.02 | 10.75 | 57.79 | 43.19 | |
| Pure LLMMasking level=95%, Number of runs=5, backbone=GPT-5.12026.02 | 23.33 | 16.55 | 19.94 | 70.6 | 65.3 | 24.1 | 53.33 | 33.39 |