Instruction hierarchy resolution on 5-level evaluation suite 1.0 (full)
83.8PPA (Macro)GW-DPO (bi.)
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| GW-DPO (bi.)Margin Schedule=bilateral, Levels=52026.06 | 83.8 | 88.5 | 79.3 | 0.044 | 0.084 | 5.7 | 4.41 | 100 | 100 | |
| DPOMargin Schedule=standard2026.06 | 81.4 | 85.2 | 79 | 0.024 | 0.053 | 12 | 4.3 | 100 | 100 | |
| 3-level (lin.)Margin Schedule=linear, Levels=32026.06 | 77.8 | 82.6 | 78 | 0.002 | 0.084 | 7.4 | 4.26 | 100 | 100 | |
| GW-DPO (lin.)Margin Schedule=linear, Levels=52026.06 | 71.9 | 75.1 | 84.3 | 0.125 | 0.143 | 2.4 | 3.88 | 100 | 100 | |
| No-ISEMargin Schedule=linear, Levels=5, ISE Component=false2026.06 | 67.7 | 72.8 | 81 | 0.133 | 0.149 | 57.6 | 3.75 | 100 | 100 | |
| SFTConfiguration=SFT2026.06 | 33.2 | 38.8 | 49.3 | 0.161 | 0.214 | 9.2 | 2.42 | 48.3 | 52.4 | |
| baselineConfiguration=baseline2026.06 | 24.5 | 27.1 | 66 | 0.415 | 0.415 | 4.4 | 2.15 | 49.1 | 56.3 |