Ordinal Regression on STSB
0.65L1 LossTask Calibration
Evaluation Results
| Method | Links | |
|---|---|---|
| Task CalibrationModel=Gemma-3-12B, Decoding Strategy=Ours2026.05 | 0.65 | |
| Task CalibrationModel=Qwen3-30B-A3B, Decoding Strategy=Ours2026.05 | 0.67 | |
| Task CalibrationModel=Qwen-3-4B, Decoding Strategy=Ours2026.05 | 0.68 | |
| Task CalibrationModel=Gemma-3-4B, Decoding Strategy=Ours2026.05 | 0.8 | |
| BeamModel=Gemma-3-12B, Decoding Strategy=Beam2026.05 | 0.98 | |
| ContrastiveModel=Gemma-3-12B, Decoding Strategy=Contrastive2026.05 | 0.99 | |
| GreedyModel=Gemma-3-12B, Decoding Strategy=Greedy2026.05 | 1 | |
| AncestralModel=Gemma-3-12B, Decoding Strategy=Ancestral2026.05 | 1.01 | |
| MBRModel=Gemma-3-12B, Decoding Strategy=MBR2026.05 | 1.01 | |
| AncestralModel=Gemma-3-4B, Decoding Strategy=Ancestral2026.05 | 1.05 | |
| BeamModel=Gemma-3-4B, Decoding Strategy=Beam2026.05 | 1.06 | |
| GreedyModel=Gemma-3-4B, Decoding Strategy=Greedy2026.05 | 1.07 | |
| MBRModel=Gemma-3-4B, Decoding Strategy=MBR2026.05 | 1.07 | |
| ContrastiveModel=Gemma-3-4B, Decoding Strategy=Contrastive2026.05 | 1.08 | |
| ContrastiveModel=Qwen3-30B-A3B, Decoding Strategy=Contrastive2026.05 | 1.17 | |
| MBRModel=Qwen3-30B-A3B, Decoding Strategy=MBR2026.05 | 1.18 | |
| AncestralModel=Qwen3-30B-A3B, Decoding Strategy=Ancestral2026.05 | 1.19 | |
| BeamModel=Qwen3-30B-A3B, Decoding Strategy=Beam2026.05 | 1.19 | |
| GreedyModel=Qwen3-30B-A3B, Decoding Strategy=Greedy2026.05 | 1.19 | |
| AncestralModel=Qwen-3-4B, Decoding Strategy=Ancestral2026.05 | 1.33 | |
| BeamModel=Qwen-3-4B, Decoding Strategy=Beam2026.05 | 1.33 | |
| ContrastiveModel=Qwen-3-4B, Decoding Strategy=Contrastive2026.05 | 1.33 | |
| MBRModel=Qwen-3-4B, Decoding Strategy=MBR2026.05 | 1.33 | |
| GreedyModel=Qwen-3-4B, Decoding Strategy=Greedy2026.05 | 1.34 |