Complex Reasoning on BBH
90.5AccuracyDECENTMEM
Evaluation results
| Method | Links | |
|---|---|---|
| DECENTMEMBackbone=QWEN3-14B, Framework=AutoGen2026.05 | 90.5 | |
| G-MemoryBackbone=QWEN3-14B, Framework=AutoGen2026.05 | 89.2 | |
| Qwen3-32BSetting=Steer2026.06 | 88.61 | |
| Qwen3-14BSetting=Steer2026.06 | 88.38 | |
| Qwen3-4BSetting=Steer2026.06 | 87.18 | |
| LLaMA-70BSetting=Steer2026.06 | 86.42 | |
| BaseBackbone=Qwen3-4B2025.05 | 85.93 | |
| Qwen3-8BSetting=Steer2026.06 | 85.87 | |
| Qwen3-32BSetting=Base2026.06 | 85.79 | |
| Qwen3-14BSetting=Base2026.06 | 85.33 | |
| DECENTMEMBackbone=QWEN3-8B, Framework=AutoGen2026.05 | 85.07 | |
| LLaMA-70BSetting=Base2026.06 | 84.81 | |
| Qwen3-4BSetting=Base2026.06 | 84.78 | |
| Qwen3-8BSetting=Base2026.06 | 84.67 | |
| DECENTMEMBackbone=QWEN3-14B, Framework=AgentNet2026.05 | 84.3 | |
| G-MemoryBackbone=QWEN3-8B, Framework=AutoGen2026.05 | 82.09 | |
| MetaGPTBackbone=QWEN3-14B, Framework=AutoGen2026.05 | 81.68 | |
| ChatDevBackbone=QWEN3-14B, Framework=AutoGen2026.05 | 79.28 | |
| DECENTMEMBackbone=QWEN3-8B, Framework=AgentNet2026.05 | 76.19 | |
| MetaGPTBackbone=QWEN3-8B, Framework=AutoGen2026.05 | 76.12 | |
| RCEModel=Qwen-14B2026.02 | 76.1 | |
| Qwen3-1.7BSetting=Steer2026.06 | 74.55 | |
| Qwen3-1.7BSetting=Base2026.06 | 72.83 | |
| DECENTMEMBackbone=QWEN3-14B, Framework=DyLAN2026.05 | 72.43 | |
| RCEModel=Llama-3-8B2026.02 | 72.3 | |
| ChatDevBackbone=QWEN3-8B, Framework=AutoGen2026.05 | 71.14 | |
| LLaMA-8BSetting=Steer2026.06 | 70.9 | |
| RCEModel=Mistral-7B2026.02 | 70.5 | |
| LLaMA-8BSetting=Base2026.06 | 70.48 | |
| MetaGPTBackbone=QWEN3-14B, Framework=DyLAN2026.05 | 68.86 | |
| G-MemoryBackbone=QWEN3-14B, Framework=AgentNet2026.05 | 68.52 | |
| No memoryBackbone=QWEN3-14B, Framework=AutoGen2026.05 | 67.33 | |
| G-MemoryBackbone=QWEN3-14B, Framework=DyLAN2026.05 | 66.84 | |
| DECENTMEMBackbone=QWEN3-8B, Framework=DyLAN2026.05 | 65.52 | |
| MetaGPTBackbone=QWEN3-14B, Framework=AgentNet2026.05 | 65.48 | |
| ChatDevBackbone=QWEN3-14B, Framework=DyLAN2026.05 | 65.28 | |
| DisCOModel=Mistral-7B2026.02 | 64.8 | |
| Vanilla LoRABackbone=Qwen3-4B2025.05 | 64.58 | |
| DECENTMEMBackbone=QWEN3-4B, Framework=AutoGen2026.05 | 63.8 | |
| BaseModel=Qwen-14B2026.02 | 63.5 | |
| SHE-LoRABackbone=Qwen3-4B2025.05 | 63.26 | |
| G-MemoryBackbone=QWEN3-8B, Framework=AgentNet2026.05 | 62.18 | |
| GRPOModel=Mistral-7B2026.02 | 62.1 | |
| No memoryBackbone=QWEN3-8B, Framework=AutoGen2026.05 | 61.88 | |
| MetaGPTBackbone=QWEN3-8B, Framework=DyLAN2026.05 | 61.58 | |
| G-MemoryBackbone=QWEN3-4B, Framework=AutoGen2026.05 | 61.53 | |
| Qwen3 8BCompression Ratio=dense2026.02 | 60.86 | |
| No memoryBackbone=QWEN3-14B, Framework=DyLAN2026.05 | 60.52 | |
| SCModel=Mistral-7B, Samples (n)=162026.02 | 60.2 | |
| ChatDevBackbone=QWEN3-8B, Framework=DyLAN2026.05 | 59.61 | |
| ToTModel=Mistral-7B2026.02 | 59.5 | |
| DECENTMEMBackbone=QWEN3-4B, Framework=AgentNet2026.05 | 59.43 | |
| G-MemoryBackbone=QWEN3-8B, Framework=DyLAN2026.05 | 58.62 | |
| CoTModel=Mistral-7B2026.02 | 57.8 | |
| MetaGPTBackbone=QWEN3-8B, Framework=AgentNet2026.05 | 57.14 | |
| BaseModel=Llama-3-8B2026.02 | 54.7 | |
| No memoryBackbone=QWEN3-8B, Framework=DyLAN2026.05 | 54.68 | |
| MetaGPTBackbone=QWEN3-4B, Framework=AutoGen2026.05 | 54.19 | |
| ChatDevBackbone=QWEN3-4B, Framework=AutoGen2026.05 | 53.37 | |
| DECENTMEMBackbone=QWEN3-4B, Framework=DyLAN2026.05 | 52.19 | |
| BaseModel=Mistral-7B2026.02 | 51.3 | |
| ROCKETCompression Ratio=0.22026.02 | 50.33 | |
| ChatDevBackbone=QWEN3-14B, Framework=AgentNet2026.05 | 48.92 | |
| ShapLoRABackbone=LlaMA-3 8B2026.01 | 48.7 | |
| G-MemoryBackbone=QWEN3-4B, Framework=AgentNet2026.05 | 48.14 | |
| MOELoRABackbone=LlaMA-3 8B2026.01 | 47.8 | |
| AutoLoRABackbone=LlaMA-3 8B2026.01 | 47.6 | |
| ROCKETCompression Ratio=0.32026.02 | 47.54 | |
| CoSpaDiCompression Ratio=0.22026.02 | 45.25 | |
| MetaGPTBackbone=QWEN3-4B, Framework=DyLAN2026.05 | 45.23 | |
| ChatDevBackbone=QWEN3-4B, Framework=DyLAN2026.05 | 44.73 | |
| No memoryBackbone=QWEN3-4B, Framework=AutoGen2026.05 | 44.43 | |
| G-MemoryBackbone=QWEN3-4B, Framework=DyLAN2026.05 | 43.97 | |
| No memoryBackbone=QWEN3-14B, Framework=AgentNet2026.05 | 42.61 | |
| ChatDevBackbone=QWEN3-8B, Framework=AgentNet2026.05 | 42.11 | |
| No memoryBackbone=QWEN3-4B, Framework=DyLAN2026.05 | 41.17 | |
| SVD-LLMCompression Ratio=0.22026.02 | 41 | |
| MetaGPTBackbone=QWEN3-4B, Framework=AgentNet2026.05 | 40.86 | |
| InfLLM v2Training Tokens=100B, Inference Top-K=N/A, Training Top-K=42026.01 | 40.55 | |
| ROCKETCompression Ratio=0.42026.02 | 40.06 | |
| DenseTraining Tokens=100B, Inference Top-K=N/A2026.01 | 40.02 | |
| InfLLM v2Training Tokens=100B, Inference Top-K=4, Training Top-K=42026.01 | 40.02 | |
| InfLLM v2Training Tokens=100B, Inference Top-K=N/A, Training Top-K=22026.01 | 39.78 | |
| PHSATraining Tokens=100B, Inference Top-K=N/A, Training Top-K=22026.01 | 39.56 | |
| CoSpaDiCompression Ratio=0.32026.02 | 38.22 | |
| No memoryBackbone=QWEN3-8B, Framework=AgentNet2026.05 | 35.13 | |
| SVD-LLMCompression Ratio=0.32026.02 | 34.42 | |
| DenseTraining Tokens=100B, Inference Top-K=22026.01 | 33.99 | |
| ChatDevBackbone=QWEN3-4B, Framework=AgentNet2026.05 | 33.67 | |
| PHSATraining Tokens=100B, Inference Top-K=2, Training Top-K=22026.01 | 33.37 | |
| PHSATraining Tokens=100B, Inference Top-K=4, Training Top-K=42026.01 | 32.78 | |
| CoSpaDiCompression Ratio=0.42026.02 | 32.72 | |
| PHSATraining Tokens=100B, Inference Top-K=N/A, Training Top-K=42026.01 | 32.11 | |
| SVD-LLMCompression Ratio=0.42026.02 | 30.24 | |
| No memoryBackbone=QWEN3-4B, Framework=AgentNet2026.05 | 28.13 | |
| InfLLM v2Training Tokens=100B, Inference Top-K=2, Training Top-K=22026.01 | 26.26 | |
| BaseBackbone=Llama-3.2-3B2025.05 | 10.85 | |
| Vanilla LoRABackbone=Llama-3.2-3B2025.05 | 9.38 | |
| SHE-LoRABackbone=Llama-3.2-3B2025.05 | 9.26 |