Personalization Behavior Classification on MMLU Topic Distributions
80Generalized ScoreDeepSeek-R1
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-R1Paradigm=LLM-as-a-Judge, Items=6272026.06 | 80 | 0 | 48 | 46.25 | 32 | 0.48 | 32 | 75.92 | 30.66 | 57.8 | 36,241.58 | 7,353.7 | 4,610,768 | |
| Llama-3-8BParadigm=LLM-as-a-Judge, Items=1,2372026.06 | 77 | 0 | 50 | 61.68 | 42 | 0 | 44 | 47.37 | 27.26 | 29.21 | 36,133.03 | 5,658.14 | 6,999,116 | |
| Qwen3-8BParadigm=LLM-as-a-Judge, Items=1,7232026.06 | 74 | 18.69 | 42 | 33.08 | 30 | 6.79 | 34 | 65.76 | 31.08 | 20.97 | 36,134.73 | 4,763.17 | 8,206,940 | |
| Falcon-H1RParadigm=LLM-as-a-Judge, Items=1622026.06 | 72 | 3.09 | 45 | 41.98 | 25 | 6.17 | 26 | 79.01 | 32.56 | 228.72 | 37,053.08 | 7,406.99 | 1,199,933 | |
| DeBERTa-LargeParadigm=NLICV, Items=11,4402026.06 | 50 | 93.23 | 63 | 43.5 | 43 | 56.32 | 10 | 80.71 | 68.44 | 0.07 | 7,826.17 | — | — | |
| DeBERTa-SmallParadigm=NLICV, Items=11,4402026.06 | 49 | 89.1 | 53 | 34.68 | 34 | 49.95 | 14 | 74.33 | 62.02 | 0.03 | 3,901.49 | — | — | |
| RoBERTaParadigm=NLICV, Items=11,4402026.06 | 49 | 93.68 | 37 | 16.85 | 24 | 27.24 | 11 | 80.85 | 54.66 | 0.02 | 2,525.86 | — | — | |
| DistilRoBERTaParadigm=NLICV, Items=11,4402026.06 | 44 | 91.42 | 16 | 2.74 | 13 | 5.67 | 13 | 78.21 | 44.51 | 0.02 | 2,124.91 | — | — | |
| MiniLM2Paradigm=NLICV, Items=11,4402026.06 | 41 | 86.9 | 12 | 1.44 | 10 | 2.88 | 10 | 83.04 | 43.56 | 0.02 | 2,124.79 | — | — |