Question Answering on MathQA (test)
81.05AccuracyGPT-4o
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-4oModel Category=Proprietary LLM2026.01 | 81.05 | |
| Claude-3.5-HaikuModel Category=Proprietary LLM2026.01 | 79.36 | |
| Gemini-2.5-ProModel Category=Proprietary LLM2026.01 | 73.1 | |
| SGR-Llama3.3-70BMethod=Self-Graph Reasoning2026.01 | 67.17 | |
| Qwen2.5-72BModel Category=Open-source LLM2026.01 | 52.6 | |
| LLaMA3-8B (Full Precision)Model=LLaMA3-8B, Quantization Format=None, Compensation Method=None2024.10 | 40.1 | |
| GPT-5.1Model Category=Proprietary LLM2026.01 | 39.09 | |
| Qwen2.5-7BModel Category=Open-source LLM2026.01 | 38.29 | |
| LLaMA-3.3-70BModel Category=Open-source LLM2026.01 | 38.09 | |
| EoRAModel=LLaMA3-8B, Quantization Format=4-bit2024.10 | 37.21 | |
| CALDERAModel=LLaMA3-8B, Quantization Format=4-bit2024.10 | 36.7 | |
| ZeroQuant-V2Model=LLaMA3-8B, Quantization Format=4-bit2024.10 | 36.51 | |
| LRCModel=LLaMA3-8B, Quantization Format=4-bit2024.10 | 36.4 | |
| ApiQModel=LLaMA3-8B, Quantization Format=4-bit2024.10 | 36.18 | |
| OATSModel=LLaMA3-8B, Quantization Format=4-bit2024.10 | 36.01 | |
| QERAModel=LLaMA3-8B, Quantization Format=4-bit2024.10 | 35.9 | |
| SLiMModel=LLaMA3-8B, Quantization Format=4-bit2024.10 | 35.9 | |
| Act-SModel=LLaMA3-8B, Quantization Format=4-bit2024.10 | 35.84 | |
| FWSVDModel=LLaMA3-8B, Quantization Format=4-bit2024.10 | 35.64 | |
| LQERModel=LLaMA3-8B, Quantization Format=4-bit2024.10 | 35.46 | |
| LLaMA3-8B (Quantized Baseline)Model=LLaMA3-8B, Quantization Format=4-bit, Compensation Method=None2024.10 | 34.07 | |
| LLaMA-3.1-8BModel Category=Open-source LLM2026.01 | 29.39 | |
| LLaMA-3.2-3BModel Category=Open-source LLM2026.01 | 29.21 | |
| EoRAModel=LLaMA3-8B, Quantization Format=3-bit2024.10 | 29.11 | |
| LRCModel=LLaMA3-8B, Quantization Format=3-bit2024.10 | 28.64 | |
| CALDERAModel=LLaMA3-8B, Quantization Format=3-bit2024.10 | 28.1 | |
| ApiQModel=LLaMA3-8B, Quantization Format=3-bit2024.10 | 26.86 | |
| ZeroQuant-V2Model=LLaMA3-8B, Quantization Format=3-bit2024.10 | 26.43 | |
| FWSVDModel=LLaMA3-8B, Quantization Format=3-bit2024.10 | 26.3 | |
| SLiMModel=LLaMA3-8B, Quantization Format=3-bit2024.10 | 25.91 | |
| LQERModel=LLaMA3-8B, Quantization Format=3-bit2024.10 | 25.6 | |
| Act-SModel=LLaMA3-8B, Quantization Format=3-bit2024.10 | 25.42 | |
| QERAModel=LLaMA3-8B, Quantization Format=3-bit2024.10 | 25.32 | |
| OATSModel=LLaMA3-8B, Quantization Format=3-bit2024.10 | 25.3 | |
| LLaMA3-8B (Quantized Baseline)Model=LLaMA3-8B, Quantization Format=3-bit, Compensation Method=None2024.10 | 22.37 | |
| AR Transformer2026.02 | 22.24 | |
| Duo++k=32026.02 | 21.01 | |
| MDLM2026.02 | 20.7 | |
| Duo++k=22026.02 | 20.4 | |
| Duo++k=52026.02 | 20.2 | |
| Duo2026.02 | 20 |