Multiple Choice Classification on When2Call
78.63AccuracyTask Calibration
Evaluation Results
| Method | Links | |
|---|---|---|
| Task CalibrationModel=Qwen3-30B-A3B, Decoding Strategy=Ours2026.05 | 78.63 | |
| GreedyModel=Qwen3-30B-A3B, Decoding Strategy=Greedy2026.05 | 74.8 | |
| MBRModel=Qwen3-30B-A3B, Decoding Strategy=MBR2026.05 | 74.59 | |
| AncestralModel=Qwen3-30B-A3B, Decoding Strategy=Ancestral2026.05 | 74.49 | |
| ContrastiveModel=Qwen3-30B-A3B, Decoding Strategy=Contrastive2026.05 | 74.39 | |
| Task CalibrationModel=Gemma-3-12B, Decoding Strategy=Ours2026.05 | 74.38 | |
| BeamModel=Qwen3-30B-A3B, Decoding Strategy=Beam2026.05 | 74.19 | |
| Task CalibrationModel=Qwen-3-4B, Decoding Strategy=Ours2026.05 | 74.1 | |
| MBRModel=Qwen-3-4B, Decoding Strategy=MBR2026.05 | 73 | |
| ContrastiveModel=Qwen-3-4B, Decoding Strategy=Contrastive2026.05 | 72.1 | |
| GreedyModel=Qwen-3-4B, Decoding Strategy=Greedy2026.05 | 71.3 | |
| BeamModel=Qwen-3-4B, Decoding Strategy=Beam2026.05 | 70.4 | |
| AncestralModel=Qwen-3-4B, Decoding Strategy=Ancestral2026.05 | 69.8 | |
| BeamModel=Gemma-3-12B, Decoding Strategy=Beam2026.05 | 69.17 | |
| MBRModel=Gemma-3-12B, Decoding Strategy=MBR2026.05 | 68.87 | |
| GreedyModel=Gemma-3-12B, Decoding Strategy=Greedy2026.05 | 68.57 | |
| ContrastiveModel=Gemma-3-12B, Decoding Strategy=Contrastive2026.05 | 67.67 | |
| AncestralModel=Gemma-3-12B, Decoding Strategy=Ancestral2026.05 | 67.47 | |
| Task CalibrationModel=Gemma-3-4B, Decoding Strategy=Ours2026.05 | 60.63 | |
| MBRModel=Gemma-3-4B, Decoding Strategy=MBR2026.05 | 51.14 | |
| AncestralModel=Gemma-3-4B, Decoding Strategy=Ancestral2026.05 | 50.81 | |
| BeamModel=Gemma-3-4B, Decoding Strategy=Beam2026.05 | 50.49 | |
| ContrastiveModel=Gemma-3-4B, Decoding Strategy=Contrastive2026.05 | 50.49 | |
| GreedyModel=Gemma-3-4B, Decoding Strategy=Greedy2026.05 | 49.84 |