Mathematical Reasoning on MAWPS (test)
96.5AccuracyTATA
Evaluation Results
| Method | Links | |
|---|---|---|
| TATAModel=Qwen2.5-14B2025.02 | 96.5 | |
| hybridModel=Qwen2.5Math-7B2025.02 | 95.4 | |
| ensembleModel=Qwen2.5-3B2025.02 | 95.3 | |
| TATAModel=Qwen2.5Math-7B2025.02 | 95.2 | |
| TATAModel=Qwen2.5-3B2025.02 | 94.7 | |
| TATAModel=Qwen2.5-1.5B2025.02 | 94.2 | |
| TATAModel=Qwen2.5-7B2025.02 | 94.2 | |
| LLMBOOSTBackbone=Qwen-2.5-7B, Params=2 × 7B2025.12 | 94.1 | |
| TATAModel=Qwen2.5Math-1.5B2025.02 | 94 | |
| hybridModel=Qwen2.5-14B2025.02 | 93.8 | |
| ensembleModel=Qwen2.5Math-7B2025.02 | 93.4 | |
| LLMBOOSTBackbone=Qwen-2.5-7B, Params=3 × 7B2025.12 | 93.3 | |
| UNITEBackbone=Qwen-2.5-7B, Params=3 × 7B2025.12 | 92.8 | |
| hybridModel=Qwen2.5Math-1.5B2025.02 | 92.7 | |
| GPT-SelectModel=Qwen2.5Math-1.5B2025.02 | 92.7 | |
| T-copilotBackbone=Qwen-2.5-7B, Params=2 × 7B2025.12 | 92.7 | |
| LLMBOOSTBackbone=Qwen-2.5-3B, Params=3 × 3B2025.12 | 92.6 | |
| voteBackbone=Qwen-2.5-7B, Params=3 × 7B2025.12 | 92.4 | |
| LLMBOOSTBackbone=Qwen-2.5-3B, Params=2 × 3B2025.12 | 92.4 | |
| UNITEBackbone=Qwen-2.5-3B, Params=3 × 3B2025.12 | 92.3 | |
| ensembleModel=Qwen2.5-14B2025.02 | 92.2 | |
| UNITEBackbone=Qwen-2.5-7B, Params=2 × 7B2025.12 | 92.2 | |
| hybridModel=Qwen2.5-7B2025.02 | 92.1 | |
| voteBackbone=Qwen-2.5-3B, Params=3 × 3B2025.12 | 92.1 | |
| LLMBOOSTBackbone=Llama-3.1-8B, Params=3 × 8B2025.12 | 92 | |
| Qwen-2.5-7BBackbone=Qwen-2.5-7B, Params=1 × 7B2025.12 | 92 | |
| hybridModel=Qwen2.5-1.5B2025.02 | 91.8 | |
| GPT-SelectModel=Qwen2.5-1.5B2025.02 | 91.8 | |
| TATAModel=LLaMA-3-8B2025.02 | 91.8 | |
| ensembleModel=Qwen2.5Math-1.5B2025.02 | 91.8 | |
| voteBackbone=Qwen-2.5-7B, Params=2 × 7B2025.12 | 91.6 | |
| ensembleModel=Qwen2.5-1.5B2025.02 | 91.5 | |
| ensembleModel=Qwen2.5-7B2025.02 | 91.5 | |
| GPT-SelectModel=Qwen2.5-7B2025.02 | 91.4 | |
| UNITEBackbone=Qwen-2.5-3B, Params=2 × 3B2025.12 | 91.3 | |
| voteBackbone=Llama-3.1-8B, Params=3 × 8B2025.12 | 90.7 | |
| hybridModel=Qwen2.5-3B2025.02 | 90.2 | |
| LLMBOOSTBackbone=Llama-3.1-8B, Params=2 × 8B2025.12 | 90.1 | |
| T-copilotBackbone=Qwen-2.5-3B, Params=2 × 3B2025.12 | 90.1 | |
| Qwen-2.5-3BBackbone=Qwen-2.5-3B, Params=1 × 3B2025.12 | 89.9 | |
| T-copilotBackbone=Llama-3.1-8B, Params=2 × 8B2025.12 | 89.7 | |
| Llama-3.1-8BBackbone=Llama-3.1-8B, Params=1 × 8B2025.12 | 89.5 | |
| voteBackbone=Llama-3.1-8B, Params=2 × 8B2025.12 | 89.5 | |
| voteBackbone=Qwen-2.5-3B, Params=2 × 3B2025.12 | 89.5 | |
| GPT-SelectModel=Qwen2.5Math-7B2025.02 | 89.4 | |
| UNITEBackbone=Llama-3.1-8B, Params=2 × 8B2025.12 | 89.3 | |
| ensembleModel=LLaMA-3-8B2025.02 | 88.6 | |
| UNITEBackbone=Llama-3.1-8B, Params=3 × 8B2025.12 | 88.3 | |
| hybridModel=LLaMA-3-8B2025.02 | 88 | |
| LLMBOOSTBackbone=LLama-3.2-3B, Params=2 × 3B2025.12 | 87.4 | |
| voteBackbone=LLama-3.2-3B, Params=3 × 3B2025.12 | 87.4 | |
| GPT-SelectModel=Qwen2.5-3B2025.02 | 86.2 | |
| GPT-SelectModel=Qwen2.5-14B2025.02 | 86.2 | |
| LLMBOOSTBackbone=LLama-3.2-3B, Params=3 × 3B2025.12 | 86.2 | |
| UNITEBackbone=LLama-3.2-3B, Params=3 × 3B2025.12 | 86 | |
| TATAModel=Qwen2.5-0.5B2025.02 | 85.9 | |
| voteBackbone=LLama-3.2-3B, Params=2 × 3B2025.12 | 85.4 | |
| GPT-SelectModel=LLaMA-3-8B2025.02 | 85.3 | |
| UNITEBackbone=LLama-3.2-3B, Params=2 × 3B2025.12 | 85.2 | |
| RoAd1Model=13B, #Params.=0.02%2024.08 | 84.9 | |
| LLama-3.2-3BBackbone=LLama-3.2-3B, Params=1 × 3B2025.12 | 84.9 | |
| T-copilotBackbone=LLama-3.2-3B, Params=2 × 3B2025.12 | 84.6 | |
| RoAd4Model=13B, #Params.=0.07%2024.08 | 84.5 | |
| hybridModel=Qwen2.5-0.5B2025.02 | 84.5 | |
| RoAd2Model=13B, #Params.=0.03%2024.08 | 84 | |
| LORAModel=13B, #Params.=0.67%2024.08 | 83.6 | |
| ensembleModel=Qwen2.5-0.5B2025.02 | 83.4 | |
| AdapterPModel=7B, #Params.=3.54%2024.08 | 82.4 | |
| LoReFTModel=13B, #Params.=0.03%2024.08 | 82.4 | |
| RoAd4Model=7B, #Params.=0.08%2024.08 | 81.5 | |
| AdapterPModel=13B, #Params.=2.89%2024.08 | 81.1 | |
| GPT-SelectModel=Qwen2.5-0.5B2025.02 | 80.4 | |
| LORAModel=7B, #Params.=0.83%2024.08 | 79 | |
| RoAd2Model=7B, #Params.=0.04%2024.08 | 78.6 | |
| AdapterSModel=13B, #Params.=0.80%2024.08 | 78.6 | |
| AdapterSModel=7B, #Params.=0.99%2024.08 | 77.7 | |
| (IA)3Model=7B, #Params.=0.02%2024.08 | 76.9 | |
| RoAd1Model=7B, #Params.=0.02%2024.08 | 76.5 | |
| LoReFTModel=7B, #Params.=0.03%2024.08 | 76.2 | |
| PrefixModel=13B, #Params.=0.03%2024.08 | 66.8 | |
| PrefixModel=7B, #Params.=0.04%2024.08 | 63.4 | |
| ToolformerEvaluation Protocol=zero-shot, Tool Use=enabled2023.02 | 44 | |
| GPT-3Model Parameters=175B, Evaluation Protocol=zero-shot2023.02 | 19.8 | |
| ToolformerEvaluation Protocol=zero-shot, Tool Use=disabled2023.02 | 15 | |
| GPT-JEvaluation Protocol=zero-shot2023.02 | 9.9 | |
| GPT-J + CCEvaluation Protocol=zero-shot2023.02 | 9.3 | |
| OPTModel Parameters=66B, Evaluation Protocol=zero-shot2023.02 | 7.9 |