Instruction Induction on Instruction Induction
38.7Avg Execution ScoreItD
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| ItDLLM Backbone=Llama-2-7b-chat2024.03 | 38.7 | — | — | — | — | — | |
| ItD-IOLLM Backbone=Llama-2-7b-chat2024.03 | 32.49 | — | — | — | — | — | |
| HS&RLLM Backbone=Llama-2-7b-chat2024.03 | 28.68 | — | — | — | — | — | |
| HSLLM Backbone=Llama-2-7b-chat2024.03 | 27.83 | — | — | — | — | — | |
| SCLLM Backbone=Llama-2-7b-chat2024.03 | 23.59 | — | — | — | — | — | |
| IOLLM Backbone=Llama-2-7b-chat2024.03 | 13.23 | — | — | — | — | — | |
| Trans. CoT (DAPO)Metric=BERTScore2026.01 | 0.61 | 0.6 | 0.58 | 0.62 | 0.61 | 0.62 | |
| Trans. CoT (NRR)Metric=BERTScore2026.01 | 0.6 | 0.58 | 0.57 | 0.6 | 0.6 | 0.63 | |
| Trans. CoT (OSS)Metric=BERTScore2026.01 | 0.6 | 0.6 | 0.57 | 0.61 | 0.6 | 0.62 | |
| Trans. CoT (QwQ)Metric=BERTScore2026.01 | 0.6 | 0.6 | 0.57 | 0.61 | 0.6 | 0.62 | |
| Ens. (QwQ+DAPO/OSS)Metric=BERTScore2026.01 | 0.6 | 0.6 | 0.57 | 0.61 | 0.6 | 0.62 | |
| Ens. (QwQ+OSS/DAPO)Metric=BERTScore2026.01 | 0.6 | 0.6 | 0.57 | 0.61 | 0.6 | 0.62 | |
| Ens. (QwQ+OpenT/OSS)Metric=BERTScore2026.01 | 0.6 | 0.61 | 0.58 | 0.61 | 0.6 | 0.62 | |
| Default (Sampled) CoTMetric=BERTScore2026.01 | 0.58 | 0.56 | 0.56 | 0.6 | 0.6 | 0.6 | |
| Default (Greedy) CoTMetric=BERTScore2026.01 | 0.57 | 0.58 | 0.46 | 0.61 | 0.6 | 0.62 | |
| Trans. CoT (OpenT)Metric=BERTScore2026.01 | 0.57 | 0.6 | 0.46 | 0.59 | 0.59 | 0.61 | |
| Empty CoTMetric=BERTScore2026.01 | 0.55 | 0.53 | 0.55 | 0.56 | 0.55 | 0.57 |