Instruction Induction on Evals-Induct
85.08AccuracySTRATEGY-INDUCT
Evaluation results
| Method | Links | |
|---|---|---|
| STRATEGY-INDUCTBackbone=GPT o3 mini (high), Setting=Zero-shot2026.05 | 85.08 | |
| ZCoTBackbone=GPT o3 mini (high), Setting=Zero-shot2026.05 | 85.02 | |
| SCoTBackbone=GPT o3 mini (high), Setting=Zero-shot2026.05 | 83.85 | |
| ZCoTBackbone=GPT o3 mini (medium), Setting=Zero-shot2026.05 | 83.6 | |
| STRATEGY-INDUCTBackbone=GPT o3 mini (medium), Setting=Zero-shot2026.05 | 82.18 | |
| INDUCTBackbone=GPT o3 mini (high), Setting=Zero-shot2026.05 | 81.91 | |
| SCoTBackbone=GPT o3 mini (medium), Setting=Zero-shot2026.05 | 81.14 | |
| INDUCTBackbone=GPT o3 mini (medium), Setting=Zero-shot2026.05 | 79.27 | |
| ZCoTBackbone=GPT o3 mini (low), Setting=Zero-shot2026.05 | 78.29 | |
| STRATEGY-INDUCTBackbone=GPT o3 mini (low), Setting=Zero-shot2026.05 | 77.26 | |
| INDUCTBackbone=GPT o3 mini (low), Setting=Zero-shot2026.05 | 77.02 | |
| SCoTBackbone=GPT o3 mini (low), Setting=Zero-shot2026.05 | 76.39 | |
| ZCoTBackbone=Gemini 2.0 Flash Thinking, Setting=Zero-shot2026.05 | 74.46 | |
| STRATEGY-INDUCTBackbone=Gemini 2.0 Flash Thinking, Setting=Zero-shot2026.05 | 74.16 | |
| SCoTBackbone=Gemini 2.0 Flash Thinking, Setting=Zero-shot2026.05 | 73.72 | |
| SCoTBackbone=GPT-4o, Setting=Zero-shot2026.05 | 71.99 | |
| INDUCTBackbone=GPT-4o, Setting=Zero-shot2026.05 | 71.93 | |
| STRATEGY-INDUCTBackbone=GPT-4o, Setting=Zero-shot2026.05 | 71.2 | |
| STRATEGY-INDUCTBackbone=Gemini 2.0 Pro Exp, Setting=Zero-shot2026.05 | 71.08 | |
| INDUCTBackbone=Gemini 2.0 Flash Thinking, Setting=Zero-shot2026.05 | 71.03 | |
| ZCoTBackbone=GPT-4o, Setting=Zero-shot2026.05 | 70.27 | |
| ZCoTBackbone=Gemini 2.0 Pro Exp, Setting=Zero-shot2026.05 | 69.69 | |
| SCoTBackbone=Gemini 2.0 Pro Exp, Setting=Zero-shot2026.05 | 68.88 | |
| STRATEGY-INDUCTBackbone=Gemini 1.5 Pro, Setting=Zero-shot2026.05 | 67.18 | |
| STRATEGY-INDUCTBackbone=Gemini 2.0 Flash, Setting=Zero-shot2026.05 | 66.38 | |
| SCoTBackbone=Gemini 1.5 Pro, Setting=Zero-shot2026.05 | 64.83 | |
| ZCoTBackbone=Gemini 1.5 Pro, Setting=Zero-shot2026.05 | 64.61 | |
| INDUCTBackbone=Gemini 2.0 Pro Exp, Setting=Zero-shot2026.05 | 64.47 | |
| SCoTBackbone=Gemini 2.0 Flash, Setting=Zero-shot2026.05 | 64.22 | |
| ZCoTBackbone=Mistral Large 2, Setting=Zero-shot2026.05 | 63.74 | |
| INDUCTBackbone=Mistral Large 2, Setting=Zero-shot2026.05 | 63.44 | |
| INDUCTBackbone=Gemini 1.5 Pro, Setting=Zero-shot2026.05 | 63.07 | |
| STRATEGY-INDUCTBackbone=GPT-4o mini, Setting=Zero-shot2026.05 | 62.52 | |
| STRATEGY-INDUCTBackbone=Gemini 2.0 Flash Lite, Setting=Zero-shot2026.05 | 62.49 | |
| INDUCTBackbone=Llama 3.1 405B, Setting=Zero-shot2026.05 | 62.44 | |
| INDUCTBackbone=GPT-4o mini, Setting=Zero-shot2026.05 | 62.23 | |
| STRATEGY-INDUCTBackbone=Llama 3.1 405B, Setting=Zero-shot2026.05 | 61.84 | |
| ZCoTBackbone=Llama 3.1 405B, Setting=Zero-shot2026.05 | 61.57 | |
| ZCoTBackbone=GPT-4o mini, Setting=Zero-shot2026.05 | 61.52 | |
| SCoTBackbone=Llama 3.1 405B, Setting=Zero-shot2026.05 | 61.48 | |
| ZCoTBackbone=Gemini 2.0 Flash, Setting=Zero-shot2026.05 | 61.35 | |
| INDUCTBackbone=Gemini 2.0 Flash Lite, Setting=Zero-shot2026.05 | 60.54 | |
| ZCoTBackbone=Gemini 2.0 Flash Lite, Setting=Zero-shot2026.05 | 60.48 | |
| STRATEGY-INDUCTBackbone=Llama 3.3 70B, Setting=Zero-shot2026.05 | 60.32 | |
| SCoTBackbone=GPT-4o mini, Setting=Zero-shot2026.05 | 60.22 | |
| STRATEGY-INDUCTBackbone=Mistral Large 2, Setting=Zero-shot2026.05 | 60.08 | |
| INDUCTBackbone=Gemini 2.0 Flash, Setting=Zero-shot2026.05 | 60.01 | |
| ZCoTBackbone=Llama 3.3 70B, Setting=Zero-shot2026.05 | 59.99 | |
| SCoTBackbone=Gemini 2.0 Flash Lite, Setting=Zero-shot2026.05 | 59.52 | |
| INDUCTBackbone=Llama 3.1 70B, Setting=Zero-shot2026.05 | 59.45 | |
| SCoTBackbone=Llama 3.3 70B, Setting=Zero-shot2026.05 | 59.34 | |
| SCoTBackbone=Mistral Large 2, Setting=Zero-shot2026.05 | 58.93 | |
| STRATEGY-INDUCTBackbone=Llama 3.1 70B, Setting=Zero-shot2026.05 | 58.91 | |
| INDUCTBackbone=Llama 3.3 70B, Setting=Zero-shot2026.05 | 58.49 | |
| STRATEGY-INDUCTBackbone=Mistral Small 3, Setting=Zero-shot2026.05 | 57.22 | |
| SCoTBackbone=Llama 3.1 70B, Setting=Zero-shot2026.05 | 56.99 | |
| STRATEGY-INDUCTBackbone=Gemini 1.5 Flash, Setting=Zero-shot2026.05 | 56.56 | |
| SCoTBackbone=Mistral Small 3, Setting=Zero-shot2026.05 | 56.5 | |
| ZCoTBackbone=Llama 3.1 70B, Setting=Zero-shot2026.05 | 55.91 | |
| SCoTBackbone=Gemini 1.5 Flash, Setting=Zero-shot2026.05 | 55.78 | |
| INDUCTBackbone=Mistral Small 3, Setting=Zero-shot2026.05 | 55.13 | |
| INDUCTBackbone=Gemini 1.5 Flash, Setting=Zero-shot2026.05 | 54.54 | |
| ZCoTBackbone=Mistral Small 3, Setting=Zero-shot2026.05 | 53.94 | |
| ZCoTBackbone=Gemini 1.5 Flash, Setting=Zero-shot2026.05 | 53.57 | |
| INDUCTBackbone=Mistral Small 2, Setting=Zero-shot2026.05 | 51.03 | |
| STRATEGY-INDUCTBackbone=Mistral Small 2, Setting=Zero-shot2026.05 | 48.95 | |
| ZCoTBackbone=Mistral Small 2, Setting=Zero-shot2026.05 | 48.34 | |
| SCoTBackbone=Mistral Small 2, Setting=Zero-shot2026.05 | 48.18 | |
| SCoTBackbone=Gemini 1.5 Flash 8B, Setting=Zero-shot2026.05 | 45.24 | |
| ZCoTBackbone=Gemini 1.5 Flash 8B, Setting=Zero-shot2026.05 | 44.97 | |
| STRATEGY-INDUCTBackbone=Gemini 1.5 Flash 8B, Setting=Zero-shot2026.05 | 43.2 | |
| INDUCTBackbone=Gemini 1.5 Flash 8B, Setting=Zero-shot2026.05 | 39.25 | |
| STRATEGY-INDUCTBackbone=Llama 3.1 8B, Setting=Zero-shot2026.05 | 37.36 | |
| INDUCTBackbone=Llama 3.1 8B, Setting=Zero-shot2026.05 | 36.09 | |
| INDUCTBackbone=Mistral Nemo 12B, Setting=Zero-shot2026.05 | 35.93 | |
| STRATEGY-INDUCTBackbone=Mistral Nemo 12B, Setting=Zero-shot2026.05 | 35.29 | |
| ZCoTBackbone=Llama 3.1 8B, Setting=Zero-shot2026.05 | 33.8 | |
| SCoTBackbone=Llama 3.1 8B, Setting=Zero-shot2026.05 | 32.09 | |
| ZCoTBackbone=Mistral Nemo 12B, Setting=Zero-shot2026.05 | 31.81 | |
| SCoTBackbone=Mistral Nemo 12B, Setting=Zero-shot2026.05 | 26.33 |