Logical Reasoning on BBH (Accuracy, Average Token Consumption)
70.11AccuracyIMAD (SFT+RL)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| IMAD (SFT+RL)Backbone=Qwen 2.5 7B Instruct, Evaluation Method=IMAD (SFT+RL)2026.04 | 70.11 | 489.75 | |
| DebateBackbone=Qwen 2.5 7B Instruct, Evaluation Method=Debate2026.04 | 67.58 | 5,458.03 | |
| SFTBackbone=Qwen 2.5 7B Instruct, Evaluation Method=SFT2026.04 | 67.14 | 1,063.32 | |
| DebateGPTBackbone=Qwen 2.5 7B Instruct, Evaluation Method=DebateGPT2026.04 | 63.93 | 444.23 | |
| IMAD (SFT+RL)Backbone=Mistral Nemo 12B Instruct, Evaluation Method=IMAD (SFT+RL)2026.04 | 63.73 | 366.86 | |
| SFTBackbone=Mistral Nemo 12B Instruct, Evaluation Method=SFT2026.04 | 63.66 | 1,021.95 | |
| DebateBackbone=Mistral Nemo 12B Instruct, Evaluation Method=Debate2026.04 | 62.76 | 2,059.43 | |
| SingleBackbone=Qwen 2.5 7B Instruct, Evaluation Method=Single2026.04 | 62.51 | 805.12 | |
| DebateGPTBackbone=Mistral Nemo 12B Instruct, Evaluation Method=DebateGPT2026.04 | 59.07 | 324.1 | |
| SingleBackbone=Mistral Nemo 12B Instruct, Evaluation Method=Single2026.04 | 58.7 | 352.22 | |
| IMAD (SFT+RL)Backbone=LLaMA-3.1 8B Instruct, Evaluation Method=IMAD (SFT+RL)2026.04 | 58.53 | 562.76 | |
| SingleBackbone=LLaMA-3.1 8B Instruct, Evaluation Method=Single2026.04 | 56.37 | 462.91 | |
| DebateGPTBackbone=LLaMA-3.1 8B Instruct, Evaluation Method=DebateGPT2026.04 | 55.83 | 433.4 | |
| SFTBackbone=LLaMA-3.1 8B Instruct, Evaluation Method=SFT2026.04 | 54.91 | 1,133.14 | |
| DebateBackbone=LLaMA-3.1 8B Instruct, Evaluation Method=Debate2026.04 | 51.06 | 8,887.94 |