Reasoning on BRUMO 2025
60.83AccuracyDS-Qwen 14B
Evaluation Results
| Method | Links | |
|---|---|---|
| DS-Qwen 14BModel Size=14B, 0-shot=true2025.05 | 60.83 | |
| CATSModel Size=14B, 0-shot=true2025.05 | 55 | |
| Layer-wise CapreseModel Size=14B, Sparse Algorithm=CATS, 0-shot=true2025.05 | 53 | |
| E2E CapreseModel Size=14B, Sparse Algorithm=CATS, 0-shot=true2025.05 | 49.17 | |
| E2E CapreseModel Size=7B, Sparse Algorithm=CATS, 0-shot=true2025.05 | 47.5 | |
| DS-Qwen 7BModel Size=7B, 0-shot=true2025.05 | 45.83 | |
| Layer-wise CapreseModel Size=14B, Sparse Algorithm=GRIFFIN, 0-shot=true2025.05 | 43.33 | |
| E2E CapreseModel Size=14B, Sparse Algorithm=GRIFFIN, 0-shot=true2025.05 | 40 | |
| Layer-wise CapreseModel Size=7B, Sparse Algorithm=CATS, 0-shot=true2025.05 | 39.17 | |
| CATSModel Size=7B, 0-shot=true2025.05 | 37.5 | |
| GRIFFINModel Size=14B, 0-shot=true2025.05 | 33.33 | |
| E2E CapreseModel Size=7B, Sparse Algorithm=GRIFFIN, 0-shot=true2025.05 | 29.17 | |
| Layer-wise CapreseModel Size=7B, Sparse Algorithm=GRIFFIN, 0-shot=true2025.05 | 27.5 | |
| DS-Qwen 1.5BModel Size=1.5B, 0-shot=true2025.05 | 25.83 | |
| GRIFFINModel Size=7B, 0-shot=true2025.05 | 25 | |
| E2E CapreseModel Size=1.5B, Sparse Algorithm=CATS, 0-shot=true2025.05 | 20 | |
| Layer-wise CapreseModel Size=1.5B, Sparse Algorithm=CATS, 0-shot=true2025.05 | 13.33 | |
| CATSModel Size=1.5B, 0-shot=true2025.05 | 10.83 | |
| E2E CapreseModel Size=1.5B, Sparse Algorithm=GRIFFIN, 0-shot=true2025.05 | 4.17 | |
| GRIFFINModel Size=1.5B, 0-shot=true2025.05 | 1.67 | |
| Layer-wise CapreseModel Size=1.5B, Sparse Algorithm=GRIFFIN, 0-shot=true2025.05 | 0 |