General Reasoning on BBH
88.7ScoreDeepSeek V3.2
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| DeepSeek V3.2Model Variant=Exp Base, # Shots=3-shot, # Activated Params=37B, # Total Params=671B2026.02 | 88.7 | — | — | — | |
| Kimi-K2Model Variant=Base, # Shots=3-shot, # Activated Params=32B, # Total Params=1043B2026.02 | 88.7 | — | — | — | |
| MiMo-V2 FlashModel Variant=Base, # Shots=3-shot, # Activated Params=15B, # Total Params=309B2026.02 | 88.5 | — | — | — | |
| Step 3.5 FlashModel Variant=Base, # Shots=3-shot, # Activated Params=11B, # Total Params=196B2026.02 | 88.2 | — | — | — | |
| DeepSeek V3.1Model Variant=Base, # Shots=3-shot, # Activated Params=37B, # Total Params=671B2026.02 | 88.2 | — | — | — | |
| GLM-4.5Model Variant=Base, # Shots=3-shot, # Activated Params=32B, # Total Params=355B2026.02 | 86.2 | — | — | — | |
| System PromptBase Model=Llama-3.1-8B2026.03 | 69 | — | — | — | |
| Llama-3.1-8B-SFTBase Model=Llama-3.1-8B2026.03 | 68 | — | — | — | |
| Self-CDBase Model=Llama-3.1-8B2026.03 | 68 | — | — | — | |
| VCLBase Model=Llama-3.1-8B2026.03 | 68 | — | — | — | |
| DROBase Model=Llama-3.1-8B2026.03 | 67 | — | — | — | |
| Vector AblationBase Model=Llama-3.1-8B2026.03 | 67 | — | — | — | |
| Dare Task ArithmeticModels=Fuse2026.02 | — | — | — | 53.66 | |
| Dare Ties MergingModels=Fuse2026.02 | — | — | — | 55.59 | |
| MathCoder2-Mistral-7BModels=base12026.02 | — | — | — | 46.66 | |
| Mistral-7B-Instruct-v0.2Models=base02026.02 | — | — | — | 47.38 | |
| P(True)Confidence Method=P(True), Budget=52025.02 | — | 48 | 100 | — | |
| P(True)Confidence Method=P(True), Budget=102025.02 | — | 47 | 90 | — | |
| Response ProbabilityConfidence Method=Response Probability, Budget=52025.02 | — | 32 | 70 | — | |
| Response ProbabilityConfidence Method=Response Probability, Budget=102025.02 | — | 45 | 80 | — | |
| SCEModels=Fuse2026.02 | — | — | — | 46.67 | |
| SCF-RKLModels=Fuse2026.02 | — | — | — | 54.47 | |
| Task ArithmeticModels=Fuse2026.02 | — | — | — | 53.66 | |
| Ties MergingModels=Fuse2026.02 | — | — | — | 46.6 | |
| Verbal 1-100Confidence Method=Verbal 1-100, Budget=52025.02 | — | 22 | 50 | — | |
| Verbal 1-100Confidence Method=Verbal 1-100, Budget=102025.02 | — | 41 | 40 | — | |
| Verbal BinaryConfidence Method=Verbal Binary, Budget=52025.02 | — | 17 | 20 | — | |
| Verbal BinaryConfidence Method=Verbal Binary, Budget=102025.02 | — | 10 | 10 | — |