Mathematical Reasoning on MATH 500 (Accuracy (%))
100Accuracy (%)Gemini-3-Pro
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini-3-ProCategory=MLLM, Note=direct2026.02 | 100 | |
| GPT-5.2Category=MLLM, Note=direct2026.02 | 99 | |
| Claude-Sonnet 4.5Category=MLLM, Note=direct2026.02 | 99 | |
| DS-Qwen 14BModel Size=14B, 0-shot=true, Sparsity=None2025.05 | 92.8 | |
| CATSModel Size=14B, 0-shot=true, Sparsity=50%, r=2562025.05 | 92.8 | |
| Qwen3-235B-A22BCategory=Reasoning LLM, Note=CoT2026.02 | 92.4 | |
| TwC (Ours) - Img & TxtCategory=Think with Comic, Note=G-t-R2026.02 | 92.3 | |
| E2E CapreseModel Size=14B, Sparse Algorithm=CATS, 0-shot=true, Sparsity=50%, r=2562025.05 | 92 | |
| Layer-wise CapreseModel Size=14B, Sparse Algorithm=CATS, 0-shot=true, Sparsity=50%, r=2562025.05 | 91 | |
| Layer-wise CapreseModel Size=14B, Sparse Algorithm=GRIFFIN, 0-shot=true, Sparsity=50%, r=2562025.05 | 90.8 | |
| DeepSeek-R1Category=Reasoning LLM, Note=CoT2026.02 | 90.4 | |
| DS-Qwen 7BModel Size=7B, 0-shot=true, Sparsity=None2025.05 | 90.2 | |
| TwC (Ours) - Only ImageCategory=Think with Comic, Note=direct2026.02 | 90 | |
| E2E CapreseModel Size=7B, Sparse Algorithm=CATS, 0-shot=true, Sparsity=50%, r=2562025.05 | 90 | |
| GRIFFINModel Size=14B, 0-shot=true, Sparsity=50%, r=2562025.05 | 89.8 | |
| CATSModel Size=7B, 0-shot=true, Sparsity=50%, r=2562025.05 | 89.2 | |
| E2E CapreseModel Size=14B, Sparse Algorithm=GRIFFIN, 0-shot=true, Sparsity=50%, r=2562025.05 | 89.2 | |
| Layer-wise CapreseModel Size=7B, Sparse Algorithm=CATS, 0-shot=true, Sparsity=50%, r=2562025.05 | 87 | |
| E2E CapreseModel Size=7B, Sparse Algorithm=GRIFFIN, 0-shot=true, Sparsity=50%, r=2562025.05 | 85.4 | |
| OriginalModel=R1-14B2026.03 | 85 | |
| Layer-wise CapreseModel Size=7B, Sparse Algorithm=GRIFFIN, 0-shot=true, Sparsity=50%, r=2562025.05 | 84.8 | |
| SafeReActModel=R1-14B2026.03 | 84 | |
| OriginalModel=R1-7B2026.03 | 83 | |
| Circuit-BreakerModel=R1-7B2026.03 | 82 | |
| Circuit-BreakerModel=R1-14B2026.03 | 82 | |
| s1.1-7B2026.02 | 81.4 | |
| HALLUGUARDBackbone=Qwen2.5-Math-7B2026.01 | 81 | |
| OriginalModel=R1-8B2026.03 | 81 | |
| SafeReActModel=R1-7B2026.03 | 81 | |
| GRIFFINModel Size=7B, 0-shot=true, Sparsity=50%, r=2562025.05 | 80.6 | |
| SafeReActModel=R1-8B2026.03 | 80 | |
| DS-Qwen 1.5BModel Size=1.5B, 0-shot=true, Sparsity=None2025.05 | 79.4 | |
| Circuit-BreakerModel=R1-8B2026.03 | 79 | |
| EnergyBackbone=Qwen2.5-Math-7B2026.01 | 78 | |
| OriginalModel=OT-7B2026.03 | 78 | |
| MINDBackbone=Qwen2.5-Math-7B2026.01 | 77.1 | |
| PerplexityBackbone=Qwen2.5-Math-7B2026.01 | 77.1 | |
| LN EntropyBackbone=Qwen2.5-Math-7B2026.01 | 76.2 | |
| SafeReActModel=OT-7B2026.03 | 76 | |
| RACEBackbone=Qwen2.5-Math-7B2026.01 | 75.1 | |
| SafeChainModel=R1-14B2026.03 | 75 | |
| InsideBackbone=Qwen2.5-Math-7B2026.01 | 74.9 | |
| E2E CapreseModel Size=1.5B, Sparse Algorithm=CATS, 0-shot=true, Sparsity=50%, r=2562025.05 | 74.8 | |
| SelfCheck GPTBackbone=Qwen2.5-Math-7B2026.01 | 74 | |
| Layer-wise CapreseModel Size=1.5B, Sparse Algorithm=CATS, 0-shot=true, Sparsity=50%, r=2562025.05 | 73.8 | |
| Circuit-BreakerModel=OT-7B2026.03 | 73 | |
| IO PromptBackbone=Qwen2.5-Math-7B2026.01 | 72.7 | |
| Semantic Ent.Backbone=Qwen2.5-Math-7B2026.01 | 72.5 | |
| CATSModel Size=1.5B, 0-shot=true, Sparsity=50%, r=2562025.05 | 72 | |
| Qwen3-8BLoss function=base+aux2026.02 | 71.94 | |
| FActScoreBackbone=Qwen2.5-Math-7B2026.01 | 71.6 | |
| Qwen3-8BLoss function=base2026.02 | 70.23 | |
| TWI-1-Generated PhotoCategory=Think with Image, Note=G-t-R2026.02 | 70.2 | |
| SafeChainModel=R1-7B2026.03 | 70 | |
| Qwen3-4BLoss function=base+aux2026.02 | 69.73 | |
| Qwen3-4BLoss function=base2026.02 | 69.55 | |
| SafeChainModel=OT-7B2026.03 | 69 | |
| P(true)Backbone=Qwen2.5-Math-7B2026.01 | 67.1 | |
| Sora 2Category=Think with Video, Note=V-o-T, Citation Source=Tong et al., 20252026.02 | 67 | |
| SafeChainModel=R1-8B2026.03 | 66 | |
| DGR SFTAlignment Dataset=DirectRefusal, Backbone=s1.1-7B2026.02 | 65.6 | |
| DGR SFTAlignment Dataset=STAR-1, Backbone=s1.1-7B2026.02 | 64.4 | |
| LOUVERCategory=Ours, Model=DeepSeek-R1-Distill-Llama-8B2026.05 | 62 | |
| E2E CapreseModel Size=1.5B, Sparse Algorithm=GRIFFIN, 0-shot=true, Sparsity=50%, r=2562025.05 | 60.4 | |
| Qwen3-4B + TTSVModel=Qwen3-4B, Adaptation=TTSV2025.12 | 60.2 | |
| DGR SFTAlignment Dataset=R1-ACT, Backbone=s1.1-7B2026.02 | 59 | |
| FlashAttn-2Category=Dense, Model=DeepSeek-R1-Distill-Llama-8B2026.05 | 58 | |
| LLaMA3.1-8BLoss function=base+aux2026.02 | 56.21 | |
| H2OCategory=Eviction, Model=DeepSeek-R1-Distill-Llama-8B2026.05 | 55 | |
| Vanilla SFTAlignment Dataset=STAR-1, Backbone=s1.1-7B2026.02 | 54.3 | |
| LLaMA3.1-8BLoss function=base2026.02 | 54.21 | |
| TF-TTCLPublication=-, Backbone=Llama-3.1-8B-Instruct2026.04 | 54 | |
| TF-GRPOPublication=arXiv 2025, Backbone=Llama-3.1-8B-Instruct2026.04 | 53 | |
| Qwen3-4BModel=Qwen3-4B2025.12 | 51.8 | |
| TLMPublication=ICML 2025, Backbone=Llama-3.1-8B-Instruct2026.04 | 50 | |
| EATAPublication=ICML 2022, Backbone=Llama-3.1-8B-Instruct2026.04 | 49.4 | |
| Base LLMPublication=-, Backbone=Llama-3.1-8B-Instruct, zero-shot=true2026.04 | 49.2 | |
| TentPublication=ICLR 2021, Backbone=Llama-3.1-8B-Instruct2026.04 | 49.2 | |
| COMEPublication=ICLR 2025, Backbone=Llama-3.1-8B-Instruct2026.04 | 48.8 | |
| Layer-wise CapreseModel Size=1.5B, Sparse Algorithm=GRIFFIN, 0-shot=true, Sparsity=50%, r=2562025.05 | 47.2 | |
| TwilightCategory=Adaptive, Model=DeepSeek-R1-Distill-Llama-8B, p=0.852026.05 | 44 | |
| GRIFFINModel Size=1.5B, 0-shot=true, Sparsity=50%, r=2562025.05 | 42 | |
| AlfaBackbone=LLaDA-8B-Instruct, LoRA Rank=64, Tuned Params (Usage %)=69.2M (0.85%), Steps=256, Zero-shot=true2026.03 | 33.8 | |
| DoRA†Backbone=LLaDA-8B-Instruct, LoRA Rank=128, Tuned Params (Usage %)=101.1M (1.25%), Steps=256, Zero-shot=true2026.03 | 33.4 | |
| LoRA†Backbone=LLaDA-8B-Instruct, LoRA Rank=64, Tuned Params (Usage %)=50.3M (0.62%), Steps=256, Zero-shot=true2026.03 | 33 | |
| LoRABackbone=LLaDA-8B-Instruct, LoRA Rank=128, Tuned Params (Usage %)=100.7M (1.24%), Steps=256, Zero-shot=true2026.03 | 32.6 | |
| AlfaBackbone=LLaDA-8B-Instruct, LoRA Rank=64, Tuned Params (Usage %)=69.2M (0.85%), Steps=128, Zero-shot=true2026.03 | 26.6 | |
| LoRABackbone=LLaDA-8B-Instruct, LoRA Rank=128, Tuned Params (Usage %)=100.7M (1.24%), Steps=128, Zero-shot=true2026.03 | 26.2 | |
| DoRA†Backbone=LLaDA-8B-Instruct, LoRA Rank=128, Tuned Params (Usage %)=101.1M (1.25%), Steps=128, Zero-shot=true2026.03 | 26.2 | |
| LoRA†Backbone=LLaDA-8B-Instruct, LoRA Rank=64, Tuned Params (Usage %)=50.3M (0.62%), Steps=128, Zero-shot=true2026.03 | 25 | |
| Vanilla SFTAlignment Dataset=R1-ACT, Backbone=s1.1-7B2026.02 | 20 | |
| Qwen2.5-7B-Instruct2026.02 | 19.2 | |
| DREAMLLMCategory=Think with Image, Note=G-t-R2026.02 | 12.6 | |
| Vanilla SFTAlignment Dataset=DirectRefusal, Backbone=s1.1-7B2026.02 | 10.2 |