Instruction Induction on BBH Induct
91.3AccuracySTRATEGY-INDUCT
Evaluation results
| Method | Links | |
|---|---|---|
| STRATEGY-INDUCTBackbone=GPT o3 mini (high), Setting=Zero-shot2026.05 | 91.3 | |
| SCoTBackbone=GPT o3 mini (high), Setting=Zero-shot2026.05 | 89.91 | |
| STRATEGY-INDUCTBackbone=GPT o3 mini (medium), Setting=Zero-shot2026.05 | 89.74 | |
| INDUCTBackbone=GPT o3 mini (high), Setting=Zero-shot2026.05 | 89.74 | |
| SCoTBackbone=GPT o3 mini (medium), Setting=Zero-shot2026.05 | 89.68 | |
| INDUCTBackbone=GPT o3 mini (medium), Setting=Zero-shot2026.05 | 89.39 | |
| INDUCTBackbone=Llama 3.1 405B, Setting=Zero-shot2026.05 | 89.28 | |
| STRATEGY-INDUCTBackbone=Llama 3.1 70B, Setting=Zero-shot2026.05 | 88.99 | |
| STRATEGY-INDUCTBackbone=Gemini 2.0 Flash Thinking, Setting=Zero-shot2026.05 | 88.99 | |
| ZCoTBackbone=GPT o3 mini (high), Setting=Zero-shot2026.05 | 88.87 | |
| STRATEGY-INDUCTBackbone=GPT o3 mini (low), Setting=Zero-shot2026.05 | 88.81 | |
| ZCoTBackbone=GPT o3 mini (medium), Setting=Zero-shot2026.05 | 88.7 | |
| STRATEGY-INDUCTBackbone=Llama 3.1 405B, Setting=Zero-shot2026.05 | 88.52 | |
| INDUCTBackbone=Llama 3.3 70B, Setting=Zero-shot2026.05 | 88.52 | |
| INDUCTBackbone=GPT o3 mini (low), Setting=Zero-shot2026.05 | 88.46 | |
| INDUCTBackbone=GPT-4o, Setting=Zero-shot2026.05 | 87.94 | |
| SCoTBackbone=GPT-4o, Setting=Zero-shot2026.05 | 87.83 | |
| STRATEGY-INDUCTBackbone=GPT-4o, Setting=Zero-shot2026.05 | 87.65 | |
| STRATEGY-INDUCTBackbone=Llama 3.3 70B, Setting=Zero-shot2026.05 | 87.3 | |
| ZCoTBackbone=Gemini 2.0 Flash Thinking, Setting=Zero-shot2026.05 | 87.25 | |
| INDUCTBackbone=Gemini 2.0 Flash Thinking, Setting=Zero-shot2026.05 | 86.9 | |
| STRATEGY-INDUCTBackbone=Gemini 2.0 Pro Exp, Setting=Zero-shot2026.05 | 86.72 | |
| SCoTBackbone=Gemini 2.0 Flash Thinking, Setting=Zero-shot2026.05 | 86.52 | |
| INDUCTBackbone=Llama 3.1 70B, Setting=Zero-shot2026.05 | 86.03 | |
| STRATEGY-INDUCTBackbone=Mistral Large 2, Setting=Zero-shot2026.05 | 85.97 | |
| ZCoTBackbone=GPT o3 mini (low), Setting=Zero-shot2026.05 | 85.91 | |
| INDUCTBackbone=Mistral Large 2, Setting=Zero-shot2026.05 | 85.57 | |
| STRATEGY-INDUCTBackbone=Gemini 2.0 Flash, Setting=Zero-shot2026.05 | 85.45 | |
| STRATEGY-INDUCTBackbone=Gemini 1.5 Pro, Setting=Zero-shot2026.05 | 85.39 | |
| ZCoTBackbone=Llama 3.1 405B, Setting=Zero-shot2026.05 | 85.04 | |
| SCoTBackbone=Gemini 2.0 Pro Exp, Setting=Zero-shot2026.05 | 85.04 | |
| SCoTBackbone=GPT o3 mini (low), Setting=Zero-shot2026.05 | 85.04 | |
| SCoTBackbone=Llama 3.1 70B, Setting=Zero-shot2026.05 | 84.52 | |
| SCoTBackbone=Llama 3.3 70B, Setting=Zero-shot2026.05 | 84.52 | |
| ZCoTBackbone=GPT-4o, Setting=Zero-shot2026.05 | 84.12 | |
| SCoTBackbone=Llama 3.1 405B, Setting=Zero-shot2026.05 | 83.94 | |
| INDUCTBackbone=Gemini 1.5 Pro, Setting=Zero-shot2026.05 | 83.71 | |
| ZCoTBackbone=Gemini 2.0 Pro Exp, Setting=Zero-shot2026.05 | 83.62 | |
| ZCoTBackbone=Llama 3.3 70B, Setting=Zero-shot2026.05 | 83.59 | |
| STRATEGY-INDUCTBackbone=GPT-4o mini, Setting=Zero-shot2026.05 | 82.32 | |
| ZCoTBackbone=Llama 3.1 70B, Setting=Zero-shot2026.05 | 82.09 | |
| ZCoTBackbone=Mistral Large 2, Setting=Zero-shot2026.05 | 82.03 | |
| SCoTBackbone=Gemini 1.5 Pro, Setting=Zero-shot2026.05 | 82.03 | |
| SCoTBackbone=Mistral Large 2, Setting=Zero-shot2026.05 | 81.86 | |
| STRATEGY-INDUCTBackbone=Gemini 1.5 Flash, Setting=Zero-shot2026.05 | 81.8 | |
| STRATEGY-INDUCTBackbone=Gemini 2.0 Flash Lite, Setting=Zero-shot2026.05 | 81.8 | |
| ZCoTBackbone=Gemini 1.5 Pro, Setting=Zero-shot2026.05 | 80.87 | |
| STRATEGY-INDUCTBackbone=Mistral Small 3, Setting=Zero-shot2026.05 | 80.41 | |
| INDUCTBackbone=Gemini 2.0 Pro Exp, Setting=Zero-shot2026.05 | 80.23 | |
| SCoTBackbone=GPT-4o mini, Setting=Zero-shot2026.05 | 80.06 | |
| SCoTBackbone=Gemini 2.0 Flash, Setting=Zero-shot2026.05 | 79.62 | |
| ZCoTBackbone=GPT-4o mini, Setting=Zero-shot2026.05 | 79.54 | |
| INDUCTBackbone=Gemini 2.0 Flash, Setting=Zero-shot2026.05 | 79.3 | |
| INDUCTBackbone=GPT-4o mini, Setting=Zero-shot2026.05 | 78.78 | |
| ZCoTBackbone=Gemini 1.5 Flash, Setting=Zero-shot2026.05 | 78.72 | |
| ZCoTBackbone=Gemini 2.0 Flash, Setting=Zero-shot2026.05 | 78.33 | |
| SCoTBackbone=Gemini 2.0 Flash Lite, Setting=Zero-shot2026.05 | 78.23 | |
| SCoTBackbone=Gemini 1.5 Flash, Setting=Zero-shot2026.05 | 77.86 | |
| INDUCTBackbone=Gemini 1.5 Flash, Setting=Zero-shot2026.05 | 77.68 | |
| INDUCTBackbone=Mistral Small 3, Setting=Zero-shot2026.05 | 76.64 | |
| ZCoTBackbone=Gemini 2.0 Flash Lite, Setting=Zero-shot2026.05 | 76.64 | |
| SCoTBackbone=Mistral Small 3, Setting=Zero-shot2026.05 | 76.52 | |
| INDUCTBackbone=Gemini 2.0 Flash Lite, Setting=Zero-shot2026.05 | 76.52 | |
| ZCoTBackbone=Mistral Small 3, Setting=Zero-shot2026.05 | 75.07 | |
| STRATEGY-INDUCTBackbone=Mistral Small 2, Setting=Zero-shot2026.05 | 70.32 | |
| ZCoTBackbone=Mistral Small 2, Setting=Zero-shot2026.05 | 70.14 | |
| SCoTBackbone=Mistral Small 2, Setting=Zero-shot2026.05 | 69.91 | |
| SCoTBackbone=Gemini 1.5 Flash 8B, Setting=Zero-shot2026.05 | 69.8 | |
| INDUCTBackbone=Mistral Small 2, Setting=Zero-shot2026.05 | 69.57 | |
| STRATEGY-INDUCTBackbone=Gemini 1.5 Flash 8B, Setting=Zero-shot2026.05 | 69.33 | |
| ZCoTBackbone=Gemini 1.5 Flash 8B, Setting=Zero-shot2026.05 | 67.42 | |
| INDUCTBackbone=Gemini 1.5 Flash 8B, Setting=Zero-shot2026.05 | 67.07 | |
| STRATEGY-INDUCTBackbone=Llama 3.1 8B, Setting=Zero-shot2026.05 | 65.33 | |
| ZCoTBackbone=Llama 3.1 8B, Setting=Zero-shot2026.05 | 62.03 | |
| INDUCTBackbone=Llama 3.1 8B, Setting=Zero-shot2026.05 | 59.48 | |
| ZCoTBackbone=Mistral Nemo 12B, Setting=Zero-shot2026.05 | 56.46 | |
| SCoTBackbone=Llama 3.1 8B, Setting=Zero-shot2026.05 | 56.29 | |
| STRATEGY-INDUCTBackbone=Mistral Nemo 12B, Setting=Zero-shot2026.05 | 55.71 | |
| SCoTBackbone=Mistral Nemo 12B, Setting=Zero-shot2026.05 | 53.97 | |
| INDUCTBackbone=Mistral Nemo 12B, Setting=Zero-shot2026.05 | 52.29 |