Preference Alignment on U10 (held-out evaluation set)
610.7Delta RewardRobust PL
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Robust PLBackbone=Qwen3-8B, K=4, ρ=0.052026.07 | 610.7 | 71.9 | 28.9 | 48 | 41.3 | 63.2 | 24.3 | |
| Robust PLBackbone=Qwen3-8B, K=2, ρ=0.052026.07 | 584.6 | 69.5 | 27.7 | 48.8 | 39.3 | 62.4 | 22.5 | |
| Robust PLBackbone=Qwen3-8B, K=4, ρ=0.102026.07 | 484 | 68.4 | 25.8 | 46.5 | 43.6 | 64.6 | 26.5 | |
| Robust PLBackbone=Qwen3-8B, K=2, ρ=0.002026.07 | 482.1 | 67.2 | 25 | 43.8 | 38.9 | 62.1 | 22.3 | |
| Robust PLBackbone=Qwen3-8B, K=4, ρ=0.002026.07 | 467 | 65.6 | 23.8 | 45.9 | 39.1 | 62.1 | 21.8 | |
| Robust PLBackbone=Qwen3-8B, K=4, ρ=0.022026.07 | 426.7 | 65.2 | 24.2 | 46.1 | 40.1 | 62.4 | 22.6 | |
| Robust PLBackbone=Qwen3-8B, K=2, ρ=0.022026.07 | 407.4 | 66 | 25 | 47.5 | 38.2 | 63 | 23.8 | |
| Robust PLBackbone=Qwen3-0.6B, K=4, ρ=0.022026.07 | 350.1 | 64.1 | 10.5 | 18.8 | 38.8 | 59.5 | 16.3 | |
| Robust PLBackbone=Qwen3-0.6B, K=4, ρ=0.052026.07 | 341.3 | 61.7 | 9.4 | 18.6 | 37.1 | 57 | 12.5 | |
| Robust PLBackbone=Qwen3-0.6B, K=2, ρ=0.022026.07 | 333 | 62.9 | 10.2 | 17.8 | 37.3 | 59.1 | 17.2 | |
| Robust PLBackbone=Qwen3-0.6B, K=4, ρ=0.002026.07 | 261.5 | 61.7 | 5.9 | 14.8 | 36.3 | 58.5 | 15.2 | |
| Robust PLBackbone=Qwen3-0.6B, K=2, ρ=0.002026.07 | 252.5 | 61.7 | 7.8 | 15.8 | 35 | 56.8 | 12.1 | |
| Robust PLBackbone=Qwen3-0.6B, K=2, ρ=0.052026.07 | 189.4 | 56.2 | 7.1 | 16.9 | 33.2 | 54.8 | 8.5 | |
| SFT referenceBackbone=Qwen3-0.6B2026.07 | 0 | 50 | 5.9 | 14.5 | 31.5 | 58.6 | 15.5 | |
| SFT referenceBackbone=Qwen3-8B2026.07 | 0 | 50 | 24.6 | 42.6 | 24.6 | 51.6 | 2.6 |