Utility Evaluation on MMLU
0.2ΔMMLUCNT
Evaluation results
| Method | Links | |
|---|---|---|
| CNTModel=DeepSeek-V2-Lite, Transfer Rate (TR)=0.00082026.03 | 0.2 | |
| CNTModel=Mistral-Nemo-Instruct, Transfer Rate (TR)=0.00242026.03 | 0.12 | |
| RDModel=DeepSeek-V2-Lite2026.03 | 0.01 | |
| CNTModel=Llama3-8B, Transfer Rate (TR)=0.00232026.03 | -0.18 | |
| RDModel=Llama3.2-3B2026.03 | -0.2 | |
| RDModel=Mistral-Nemo-Instruct2026.03 | -0.44 | |
| RDModel=Llama3-8B2026.03 | -0.65 | |
| CNTModel=Llama3.2-3B, Transfer Rate (TR)=0.00112026.03 | -0.81 | |
| CNTModel=Average, Transfer Rate (TR)=0.00142026.03 | -1.37 | |
| RDModel=Average2026.03 | -1.38 | |
| RDModel=Yi-1.5-6B2026.03 | -5.64 | |
| CNTModel=Yi-1.5-6B, Transfer Rate (TR)=0.00022026.03 | -6.21 | |
| ActSVD-OPModel=Llama3.2-3B2026.03 | -6.62 | |
| ActSVD-OPModel=Yi-1.5-6B2026.03 | -7.24 | |
| ActSVD-OPModel=Mistral-Nemo-Instruct2026.03 | -7.66 | |
| ActSVD-OPModel=Average2026.03 | -13.49 | |
| ActSVD-OPModel=Llama3-8B2026.03 | -32.43 |