Language Understanding on MMLU (Delta Accuracy)
37.28Delta AccuracyLMNet
Evaluation Results
| Method | Links | |
|---|---|---|
| LMNetBackbone=Qwen2.5-1.5B2025.05 | 37.28 | |
| PromptBackbone=Qwen2.5-1.5B2025.05 | 35.83 | |
| CODIBackbone=Qwen2.5-1.5B2025.05 | 35.16 | |
| SFTBackbone=Qwen2.5-1.5B2025.05 | 35.02 | |
| PredBackbone=Qwen2.5-1.5B2025.05 | 34.75 | |
| COCONUTBackbone=Qwen2.5-1.5B2025.05 | 33.85 | |
| LMNetBackbone=Llama3.2-1B-Instruct2025.05 | 27.57 | |
| LMNetBackbone=Llama3.2-1B2025.05 | 27.19 | |
| SFTBackbone=Llama3.2-1B-Instruct2025.05 | 24.83 | |
| SFTBackbone=Llama3.2-1B2025.05 | 24.65 | |
| PromptBackbone=Llama3.2-1B-Instruct2025.05 | 24.3 | |
| LMNetBackbone=Qwen2.5-0.5B2025.05 | 23.35 | |
| PromptBackbone=Qwen2.5-0.5B2025.05 | 22.5 | |
| PredBackbone=Qwen2.5-0.5B2025.05 | 22.36 | |
| SFTBackbone=Qwen2.5-0.5B2025.05 | 21.5 | |
| CODIBackbone=Llama3.2-1B-Instruct2025.05 | 21.3 | |
| PredBackbone=Llama3.2-1B-Instruct2025.05 | 20.93 | |
| CODIBackbone=Llama3.2-1B2025.05 | 20.69 | |
| CODIBackbone=Qwen2.5-0.5B2025.05 | 19.48 | |
| COCONUTBackbone=Qwen2.5-0.5B2025.05 | 19.22 | |
| COCONUTBackbone=Llama3.2-1B-Instruct2025.05 | 18.54 | |
| LMNetBackbone=GPT2-XL2025.05 | 13.1 | |
| PromptBackbone=Llama3.2-1B2025.05 | 11.69 | |
| COCONUTBackbone=Llama3.2-1B2025.05 | 9.88 | |
| SFTBackbone=GPT2-XL2025.05 | 7.4 | |
| PredBackbone=Llama3.2-1B2025.05 | 6.05 | |
| CODIBackbone=GPT2-XL2025.05 | 5.01 | |
| COCONUTBackbone=GPT2-XL2025.05 | 3.63 | |
| PromptBackbone=GPT2-XL2025.05 | 1.69 | |
| PredBackbone=GPT2-XL2025.05 | 0.24 |