Code Generation on Code latest (test)
83.9HumanEvalWMSS
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| WMSSBase Model=Qwen3-8B-Base, Fine-tuning Epochs=2, Runs=3 (averaged)2026.02 | 83.9 | 71.3 | 77.6 | |
| NEFTuneBase Model=Qwen3-8B-Base, Fine-tuning Epochs=2, Runs=3 (averaged)2026.02 | 80.5 | 64.3 | 72.4 | |
| UNDIALBase Model=Qwen3-8B-Base, Fine-tuning Epochs=2, Runs=3 (averaged)2026.02 | 79.7 | 61 | 70.4 | |
| SFTBase Model=Qwen3-8B-Base, Fine-tuning Epochs=2, Runs=3 (averaged)2026.02 | 78.3 | 64 | 71.2 | |
| WMSSBase Model=Qwen3-4B-Base, Fine-tuning Epochs=2, Runs=3 (averaged)2026.02 | 74.4 | 59.2 | 66.8 | |
| NEFTuneBase Model=Qwen3-4B-Base, Fine-tuning Epochs=2, Runs=3 (averaged)2026.02 | 74.2 | 55.1 | 64.7 | |
| UNDIALBase Model=Qwen3-4B-Base, Fine-tuning Epochs=2, Runs=3 (averaged)2026.02 | 71.3 | 54.6 | 63 | |
| SFTBase Model=Qwen3-4B-Base, Fine-tuning Epochs=2, Runs=3 (averaged)2026.02 | 68.5 | 57.7 | 63.1 | |
| WMSSBase Model=Qwen2.5-3B, Fine-tuning Epochs=2, Runs=3 (averaged)2026.02 | 53.7 | 43.3 | 48.5 | |
| NEFTuneBase Model=Qwen2.5-3B, Fine-tuning Epochs=2, Runs=3 (averaged)2026.02 | 51.8 | 41.5 | 46.7 | |
| SFTBase Model=Qwen2.5-3B, Fine-tuning Epochs=2, Runs=3 (averaged)2026.02 | 51.4 | 42.7 | 47.1 | |
| UNDIALBase Model=Qwen2.5-3B, Fine-tuning Epochs=2, Runs=3 (averaged)2026.02 | 50.8 | 42.5 | 46.7 |