Arithmetic Reasoning on Arithmetics
100AccuracyOracle
Evaluation results
| Method | Links | |
|---|---|---|
| OracleModel=Qwen2.5-7B, Black-Box Friendly=true2026.04 | 100 | |
| OracleModel=Llama3.2-3B, Black-Box Friendly=true2026.04 | 99.8 | |
| OracleModel=Qwen2.5-3B, Black-Box Friendly=true2026.04 | 99.5 | |
| OracleModel=Llama3.1-8B, Black-Box Friendly=true2026.04 | 99.3 | |
| Majority VoteBackbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.03 | 98.3 | |
| RCSmedoidModel=Llama3.2-3B, Black-Box Friendly=true2026.04 | 98.3 | |
| RCSuniModel=Llama3.2-3B, Black-Box Friendly=true2026.04 | 98.2 | |
| RCSfreqModel=Llama3.2-3B, Black-Box Friendly=true2026.04 | 98.2 | |
| OracleModel=Gemma2-9B, Black-Box Friendly=true2026.04 | 98.2 | |
| SCModel=Llama3.2-3B, Black-Box Friendly=true2026.04 | 97.5 | |
| RCSprobModel=Llama3.2-3B, Black-Box Friendly=false2026.04 | 97.5 | |
| CEModel=Gemma2-9B, Black-Box Friendly=false2026.04 | 97.5 | |
| CEModel=Llama3.2-3B, Black-Box Friendly=false2026.04 | 97.3 | |
| SCModel=Gemma2-9B, Black-Box Friendly=true2026.04 | 97.3 | |
| RCSfreqModel=Gemma2-9B, Black-Box Friendly=true2026.04 | 97.3 | |
| RCSprobModel=Gemma2-9B, Black-Box Friendly=false2026.04 | 97.3 | |
| Majority VoteModel=Falcon3-7B2026.03 | 97.2 | |
| RCSmedoidModel=Gemma2-9B, Black-Box Friendly=true2026.04 | 97.2 | |
| RCSuniModel=Gemma2-9B, Black-Box Friendly=true2026.04 | 97.2 | |
| GreedyModel=Qwen2.5-7B, Black-Box Friendly=true2026.04 | 97 | |
| DARBackbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.03 | 96.7 | |
| GreedyModel=Llama3.2-3B, Black-Box Friendly=true2026.04 | 96 | |
| GreedyModel=Gemma2-9B, Black-Box Friendly=true2026.04 | 96 | |
| ANLLModel=Gemma2-9B, Black-Box Friendly=false2026.04 | 96 | |
| Society Of MindModel=Llama3.1-8B2026.03 | 95.5 | |
| DARBackbone Model=Llama3.1-8B, Number of Agents=4, Number of Rounds=22026.03 | 95.5 | |
| Uncertain PromptBackbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.03 | 95.3 | |
| Uncertain PromptModel=Falcon3-7B2026.03 | 95 | |
| DARModel=Falcon3-7B2026.03 | 95 | |
| Society Of MindBackbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.03 | 95 | |
| Uncertain PromptBackbone Model=Llama3.1-8B, Number of Agents=4, Number of Rounds=22026.03 | 95 | |
| NLLModel=Gemma2-9B, Black-Box Friendly=false2026.04 | 94.8 | |
| Vote PromptModel=Llama3.1-8B2026.03 | 94.5 | |
| DARModel=Llama3.1-8B2026.03 | 94.5 | |
| Uncertain PromptModel=Llama3.1-8B2026.03 | 93 | |
| Society Of MindBackbone Model=Llama3.1-8B, Number of Agents=4, Number of Rounds=22026.03 | 92.5 | |
| Vote PromptBackbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.03 | 92.3 | |
| Vote PromptModel=Falcon3-7B2026.03 | 91.5 | |
| Majority VoteBackbone Model=Llama3.1-8B, Number of Agents=4, Number of Rounds=22026.03 | 91.4 | |
| MAD-M^2Backbone Model=Falcon3-7B, Number of Agents=4, Number of Rounds=22026.03 | 91.3 | |
| Vote PromptBackbone Model=Llama3.1-8B, Number of Agents=4, Number of Rounds=22026.03 | 91 | |
| MAD-M2Model=Falcon3-7B2026.03 | 90.7 | |
| Majority VoteModel=Llama3.1-8B2026.03 | 90.7 | |
| Society Of MindModel=Falcon3-7B2026.03 | 90 | |
| GreedyModel=Llama3.1-8B, Black-Box Friendly=true2026.04 | 89.5 | |
| GreedyModel=Qwen2.5-3B, Black-Box Friendly=true2026.04 | 89 | |
| ANLLModel=Llama3.2-3B, Black-Box Friendly=false2026.04 | 88.5 | |
| RCSuniModel=Llama3.1-8B, Black-Box Friendly=true2026.04 | 88.5 | |
| NLLModel=Llama3.2-3B, Black-Box Friendly=false2026.04 | 86.5 | |
| RCSmedoidModel=Llama3.1-8B, Black-Box Friendly=true2026.04 | 86.5 | |
| RCSuniModel=Qwen2.5-3B, Black-Box Friendly=true2026.04 | 85.7 | |
| RCSprobModel=Llama3.1-8B, Black-Box Friendly=false2026.04 | 85.2 | |
| RCSprobModel=Qwen2.5-3B, Black-Box Friendly=false2026.04 | 84.2 | |
| RCSfreqModel=Llama3.1-8B, Black-Box Friendly=true2026.04 | 83.7 | |
| RCSfreqModel=Qwen2.5-3B, Black-Box Friendly=true2026.04 | 82.8 | |
| RCSmedoidModel=Qwen2.5-3B, Black-Box Friendly=true2026.04 | 81.7 | |
| Single-agentBackbone Model=Llama3.1-8B2026.03 | 81 | |
| CEModel=Llama3.1-8B, Black-Box Friendly=false2026.04 | 81 | |
| CEModel=Qwen2.5-3B, Black-Box Friendly=false2026.04 | 80.3 | |
| MAD-M2Model=Llama3.1-8B2026.03 | 80 | |
| DARBackbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.03 | 79.7 | |
| Single-agentModel=Llama3.1-8B2026.03 | 79 | |
| SCModel=Qwen2.5-3B, Black-Box Friendly=true2026.04 | 78.7 | |
| SCModel=Llama3.1-8B, Black-Box Friendly=true2026.04 | 78.3 | |
| MAD-M^2Backbone Model=Llama3.1-8B, Number of Agents=4, Number of Rounds=22026.03 | 78 | |
| RCSprobModel=Qwen2.5-7B, Black-Box Friendly=false2026.04 | 77.7 | |
| RCSmedoidModel=Qwen2.5-7B, Black-Box Friendly=true2026.04 | 77.5 | |
| RCSuniModel=Qwen2.5-7B, Black-Box Friendly=true2026.04 | 77.5 | |
| RCSfreqModel=Qwen2.5-7B, Black-Box Friendly=true2026.04 | 76.7 | |
| CEModel=Qwen2.5-7B, Black-Box Friendly=false2026.04 | 75.5 | |
| DARModel=Qwen2.5-1.5B2026.03 | 75 | |
| DARBackbone Model=Qwen2.5-1.5B, Number of Agents=4, Number of Rounds=22026.03 | 74.7 | |
| SCModel=Qwen2.5-7B, Black-Box Friendly=true2026.04 | 74.3 | |
| ANLLModel=Llama3.1-8B, Black-Box Friendly=false2026.04 | 73.5 | |
| NLLModel=Llama3.1-8B, Black-Box Friendly=false2026.04 | 73.3 | |
| Majority VoteBackbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.03 | 73 | |
| Society Of MindModel=Qwen2.5-1.5B2026.03 | 72 | |
| Vote PromptModel=Qwen2.5-1.5B2026.03 | 72 | |
| MAD-M^2Backbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.03 | 71.3 | |
| DARModel=Qwen2.5-3B2026.03 | 70.5 | |
| Vote PromptModel=Qwen2.5-3B2026.03 | 70 | |
| Uncertain PromptBackbone Model=Qwen2.5-1.5B, Number of Agents=4, Number of Rounds=22026.03 | 69.7 | |
| MAD-M2Model=Qwen2.5-3B2026.03 | 69 | |
| Vote PromptBackbone Model=Qwen2.5-1.5B, Number of Agents=4, Number of Rounds=22026.03 | 68 | |
| Uncertain PromptModel=Qwen2.5-1.5B2026.03 | 67.5 | |
| Majority VoteModel=Qwen2.5-3B2026.03 | 65.8 | |
| Uncertain PromptModel=Qwen2.5-3B2026.03 | 65 | |
| Society Of MindBackbone Model=Qwen2.5-1.5B, Number of Agents=4, Number of Rounds=22026.03 | 63.7 | |
| Society Of MindModel=Qwen2.5-3B2026.03 | 63 | |
| Vote PromptBackbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.03 | 61.3 | |
| Single-agentBackbone Model=Falcon3-7B2026.03 | 60.5 | |
| Majority VoteBackbone Model=Qwen2.5-1.5B, Number of Agents=4, Number of Rounds=22026.03 | 59.9 | |
| Majority VoteModel=Qwen2.5-1.5B2026.03 | 59.2 | |
| Uncertain PromptBackbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.03 | 58.7 | |
| Single-agentModel=Falcon3-7B2026.03 | 58 | |
| Single-agentBackbone Model=Qwen2.5-3B2026.03 | 55 | |
| NLLModel=Qwen2.5-7B, Black-Box Friendly=false2026.04 | 47.3 | |
| ANLLModel=Qwen2.5-7B, Black-Box Friendly=false2026.04 | 47 | |
| Society Of MindBackbone Model=Qwen2.5-3B, Number of Agents=4, Number of Rounds=22026.03 | 46.3 | |
| MAD-M2Model=Qwen2.5-1.5B2026.03 | 44 |