Question Answering on MMLU-Pro
89.1AccuracyReasoningbank
Evaluation results
| Method | Links | |
|---|---|---|
| ReasoningbankLLM Backbone=GPT-5.22026.02 | 89.1 | |
| Reasoningbank+promptLLM Backbone=GPT-5.22026.02 | 87.9 | |
| No-MemoryLLM Backbone=GPT-5.22026.02 | 87.3 | |
| Reasoningbank+GuardLLM Backbone=GPT-5.22026.02 | 87.2 | |
| TAMELLM Backbone=GPT-5.22026.02 | 85.8 | |
| No-MemoryLLM Backbone=Qwen3-32B2026.02 | 81.6 | |
| TAMELLM Backbone=Qwen3-32B2026.02 | 79.4 | |
| ReasoningbankLLM Backbone=Qwen3-32B2026.02 | 79.1 | |
| MementoLLM Backbone=Qwen3-32B2026.02 | 77.8 | |
| Reasoningbank+promptLLM Backbone=Qwen3-32B2026.02 | 76.8 | |
| Reasoningbank+GuardLLM Backbone=Qwen3-32B2026.02 | 75 | |
| MementoLLM Backbone=GPT-5.22026.02 | 74.2 | |
| DCLLM Backbone=GPT-5.22026.02 | 70.1 | |
| DCLLM Backbone=Qwen3-32B2026.02 | 68.7 | |
| DSRL (Ours)Backbone=Qwen3-4B2026.04 | 66.49 | |
| BaseBackbone=Qwen3-4B2025.05 | 65.36 | |
| Raon-Speech2026.04 | 64.05 | |
| GCPOBackbone=Qwen3-4B2026.05 | 63.8 | |
| DSRL (Ours)Backbone=Qwen3-8B2026.04 | 63.79 | |
| DAPOBackbone=Qwen3-4B2026.05 | 62.8 | |
| DIVERBackbone=Qwen3-4B2026.05 | 62 | |
| Div-R1Backbone=Qwen3-4B2026.05 | 61.5 | |
| GRPOBackbone=Qwen3-4B2026.04 | 61.12 | |
| Fun-Audio Chat2026.04 | 61.12 | |
| Baseline with CoTBackbone=Llama 3 70B2026.07 | 60.8 | |
| GRPOBackbone=Qwen3-4B2026.05 | 60.1 | |
| VanillaBackbone=Qwen3-8B2026.04 | 59.72 | |
| GRPOBackbone=Qwen3-8B2026.04 | 59.47 | |
| DQOBackbone=Qwen3-4B2026.05 | 59.2 | |
| VanillaBackbone=Qwen3-4B2026.04 | 58.61 | |
| MALLM (Judge)Backbone=Llama 3 70B, Discussion Paradigm=Memory, Decision Protocol=Judge2026.07 | 58 | |
| BaselineBackbone=Llama 3 70B2026.07 | 57.5 | |
| MALLM (Majority Consensus)Backbone=Llama 3 70B, Discussion Paradigm=Memory, Decision Protocol=Majority Consensus2026.07 | 57.3 | |
| MALLM (Unanimity Consensus)Backbone=Llama 3 70B, Discussion Paradigm=Memory, Decision Protocol=Unanimity Consensus2026.07 | 57.3 | |
| MALLM (Average Consensus)Backbone=Llama 3 70B, Discussion Paradigm=Memory2026.07 | 57.2 | |
| MALLM (Supermajority Consensus)Backbone=Llama 3 70B, Discussion Paradigm=Memory, Decision Protocol=Supermajority Consensus2026.07 | 57 | |
| MALLM (Simple Voting)Backbone=Llama 3 70B, Discussion Paradigm=Memory, Decision Protocol=Simple Voting2026.07 | 56.5 | |
| LD-MoLEModel=Llama3.1-8B, TP=2.33 %2025.09 | 55.98 | |
| MiniCPM-o 4.52026.04 | 55.2 | |
| ReMoLEModel=Llama3.1-8B, TP=2.17 %2025.09 | 54.98 | |
| MALLM (Average Voting)Backbone=Llama 3 70B, Discussion Paradigm=Memory2026.07 | 54.6 | |
| MALLM (Ranked Voting)Backbone=Llama 3 70B, Discussion Paradigm=Memory, Decision Protocol=Ranked Voting2026.07 | 54.3 | |
| LD-MoLEModel=Qwen3-1.7B, TP=4.23 %2025.09 | 53.82 | |
| HyperCLOVA X 8B Omni2026.04 | 53.79 | |
| ReMoLEModel=Qwen3-1.7B, TP=4.12 %2025.09 | 53.4 | |
| MALLM (Cumulative Voting)Backbone=Llama 3 70B, Discussion Paradigm=Memory, Decision Protocol=Cumulative Voting2026.07 | 53 | |
| MoLA(8888)Model=Qwen3-1.7B, TP=4.12 %2025.09 | 51.12 | |
| IH-GRPOBackbone=Qwen3-8B2026.05 | 51.01 | |
| Qwen2.5-Omni2026.04 | 50.4 | |
| MoLA(2468)Model=Llama3.1-8B, TP=1.26 %2025.09 | 50.37 | |
| Online DPO with ArmoRMDPO Setting=Online, Reward Model=ArmoRM2026.02 | 50.3 | |
| MoLA(2468)Model=Qwen3-1.7B, TP=2.39 %2025.09 | 49.96 | |
| LD-MoLEModel=Llama3.2-3B, TP=3.28 %2025.09 | 49.58 | |
| Online DPO with WILDREWARDDPO Setting=Online, Reward Model=WILDREWARD2026.02 | 48.9 | |
| IH-GRPOBackbone=Qwen3-1.7B2026.05 | 48.75 | |
| Vanilla LoRABackbone=Qwen3-4B2025.05 | 48.54 | |
| Offline DPO with WILDREWARDDPO Setting=Offline, Reward Model=WILDREWARD2026.02 | 48.5 | |
| MoLA(8888)Model=Llama3.1-8B, TP=2.17 %2025.09 | 48.21 | |
| ReMoLEModel=Llama3.2-3B, TP=3.11 %2025.09 | 48.01 | |
| Llama3.1-8B-InstructDPO Setting=Original (None), Reward Model=N/A2026.02 | 48 | |
| SHE-LoRABackbone=Qwen3-4B2025.05 | 47.36 | |
| LLaMA-3-70B-Instruct2025.06 | 46.36 | |
| RCOBase Model=LLaMA-3-8B-Instruct2025.06 | 46.23 | |
| GCPOBackbone=Qwen3-1.7B2026.05 | 45.2 | |
| DIVERBackbone=Qwen3-1.7B2026.05 | 43.8 | |
| RCOBase Model=LLaMA-2-13B-Chat2025.06 | 43.01 | |
| MoLA(2468)Model=Llama3.2-3B, TP=1.80 %2025.09 | 42.31 | |
| RCOBase Model=LLaMA-2-7B-Chat2025.06 | 42.3 | |
| DQOBackbone=Qwen3-1.7B2026.05 | 41.5 | |
| DAPOBackbone=Qwen3-1.7B2026.05 | 41 | |
| RCOBase Model=Auto-J-13B2025.06 | 40.89 | |
| RCOBase Model=UltraCM-13B2025.06 | 40.74 | |
| MoLA(8888)Model=Llama3.2-3B, TP=3.11 %2025.09 | 40.31 | |
| Div-R1Backbone=Qwen3-1.7B2026.05 | 39.9 | |
| IH-GRPOBackbone=Qwen3-4B2026.05 | 39.16 | |
| GRPOBackbone=Qwen3-1.7B2026.05 | 38.5 | |
| LLaMA-3-8B-Instruct2025.06 | 37.88 | |
| LLAMA-2-70B-Chat2025.06 | 37.59 | |
| DPCOBase Model=LLaMA-2-7B-Chat2025.06 | 37.22 | |
| DPCOBase Model=LLaMA-3-8B-Instruct2025.06 | 36.69 | |
| DPCOBase Model=LLaMA-2-13B-Chat2025.06 | 36.52 | |
| BaseBackbone=Qwen3-4B2026.05 | 36.3 | |
| MALLM (Approval Voting)Backbone=Llama 3 70B, Discussion Paradigm=Memory, Decision Protocol=Approval Voting2026.07 | 36.3 | |
| DPCOBase Model=Auto-J-13B2025.06 | 36.21 | |
| DPCOBase Model=UltraCM-13B2025.06 | 35.98 | |
| Self-refinement2025.06 | 35.49 | |
| LLaMA-2-7B-Chat2025.06 | 35.29 | |
| Interactive 2 mini2026.04 | 34.95 | |
| LLaMA-2-13B-Chat2025.06 | 34.93 | |
| UltraCM-13B2025.06 | 33.73 | |
| BaseBackbone=Qwen3-1.7B2026.05 | 33.2 | |
| Initial Answer2025.06 | 33.14 | |
| Aligner2025.06 | 32.39 | |
| Fun-Audio Omni2026.04 | 31.38 | |
| Qwen3-4B(Base)Backbone=Qwen3-4B2026.05 | 30.9 | |
| BaseBackbone=Qwen3-8B2026.05 | 29.19 | |
| Auto-J-13B2025.06 | 28.65 | |
| Qwen3-1.7B(Base)Backbone=Qwen3-1.7B2026.05 | 27.5 | |
| Kimi Audio2026.04 | 16.66 | |
| SHE-LoRABackbone=Llama-3.2-3B2025.05 | 14.86 |