Question Answering on HotPotQA (Success Rate and Best Delta)
82.7Success RateOPT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OPTRole=Teacher, Size=13B2025.05 | 82.7 | — | |
| LLaMARole=Teacher, Size=13B2025.05 | 81 | — | |
| Structured Agent DistillationTeacher=LLaMA-13B, Student Size=7B2025.05 | 75.2 | — | |
| Structured Agent DistillationTeacher=OPT-13B, Student Size=6.7B2025.05 | 73.9 | — | |
| Token-level baselineTeacher=LLaMA-13B, Student Size=7B2025.05 | 71.5 | — | |
| GLM-4.7Intervention Mechanism=Baseline, Calibration=N/A2026.02 | 70.3 | 0 | |
| GLM-4.7Intervention Mechanism=Rollback, Calibration=Uncalibrated2026.02 | 70.3 | 0 | |
| Token-level baselineTeacher=OPT-13B, Student Size=6.7B2025.05 | 69.7 | — | |
| SeqKDTeacher=LLaMA-13B, Student Size=7B2025.05 | 69.2 | — | |
| GLM-4.7Intervention Mechanism=Rollback, Calibration=Calibrated2026.02 | 68.7 | 0 | |
| KDTeacher=LLaMA-13B, Student Size=7B2025.05 | 67.5 | — | |
| GLM-4.7Intervention Mechanism=Append, Calibration=Uncalibrated2026.02 | 67.3 | 0 | |
| Structured Agent DistillationTeacher=OPT-13B, Student Size=2.7B2025.05 | 67 | — | |
| GLM-4.7Intervention Mechanism=Append, Calibration=Calibrated2026.02 | 66.7 | 0 | |
| SeqKDTeacher=OPT-13B, Student Size=6.7B2025.05 | 66 | — | |
| KDTeacher=OPT-13B, Student Size=6.7B2025.05 | 64.3 | — | |
| MiniMax-M2.1Intervention Mechanism=Baseline, Calibration=N/A2026.02 | 64 | -25.5 | |
| Token-level baselineTeacher=OPT-13B, Student Size=2.7B2025.05 | 62.9 | — | |
| Structured Agent DistillationTeacher=OPT-13B, Student Size=1.3B2025.05 | 58.5 | — | |
| SeqKDTeacher=OPT-13B, Student Size=2.7B2025.05 | 58 | — | |
| Qwen-3-8BIntervention Mechanism=Baseline, Calibration=N/A2026.02 | 57 | -2.3 | |
| KDTeacher=OPT-13B, Student Size=2.7B2025.05 | 56.2 | — | |
| Qwen-3-8BIntervention Mechanism=Rollback, Calibration=Calibrated2026.02 | 54.7 | -2.3 | |
| Token-level baselineTeacher=OPT-13B, Student Size=1.3B2025.05 | 54.1 | — | |
| Qwen-3-8BIntervention Mechanism=Rollback, Calibration=Uncalibrated2026.02 | 53.3 | -2.3 | |
| Qwen-3-8BIntervention Mechanism=Append, Calibration=Uncalibrated2026.02 | 51.3 | -2.3 | |
| SeqKDTeacher=OPT-13B, Student Size=1.3B2025.05 | 50.3 | — | |
| Qwen-3-8BIntervention Mechanism=Append, Calibration=Calibrated2026.02 | 49.7 | -2.3 | |
| KDTeacher=OPT-13B, Student Size=1.3B2025.05 | 49 | — | |
| MiniMax-M2.1Intervention Mechanism=Rollback, Calibration=Calibrated2026.02 | 38.5 | -25.5 | |
| MiniMax-M2.1Intervention Mechanism=Rollback, Calibration=Uncalibrated2026.02 | 38 | -25.5 | |
| MiniMax-M2.1Intervention Mechanism=Append, Calibration=Calibrated2026.02 | 36.7 | -25.5 | |
| MiniMax-M2.1Intervention Mechanism=Append, Calibration=Uncalibrated2026.02 | 34.7 | -25.5 |