Mathematical Reasoning on AIME (Token Savings, Relative Accuracy Shift)
93.7Token SavingsGuardian
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GuardianModel Backbone=DSR, Sparsity Algorithm=H2O, Cache Budget=1282026.01 | 93.7 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Sink, Cache Budget=1282026.01 | 92.5 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=H2O, Cache Budget=2562026.01 | 91.2 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=H2O, Cache Budget=1282026.01 | 91 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Sink, Cache Budget=2562026.01 | 90.9 | -0.5 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=H2O, Cache Budget=2562026.01 | 87.9 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Sink, Cache Budget=5122026.01 | 87.8 | -0.5 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=H2O, Cache Budget=5122026.01 | 87.4 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=H2O, Cache Budget=5122026.01 | 86.4 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=H2O, Cache Budget=10242026.01 | 83.6 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Sink, Cache Budget=10242026.01 | 79.9 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=H2O, Cache Budget=10242026.01 | 79.6 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Sink, Cache Budget=2562026.01 | 72.6 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Sink, Cache Budget=1282026.01 | 71 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Sink, Cache Budget=5122026.01 | 67.6 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Sink, Cache Budget=10242026.01 | 65 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Quest, Cache Budget=5122026.01 | 53.9 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Quest, Cache Budget=10242026.01 | 48.2 | -1 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Quest, Cache Budget=2562026.01 | 44.9 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=H2O, Cache Budget=2562026.01 | 42.7 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=H2O, Cache Budget=1282026.01 | 39.5 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Sink, Cache Budget=1282026.01 | 36.8 | 0.5 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=infLLM, Cache Budget=10242026.01 | 33.2 | 0.5 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Sink, Cache Budget=2562026.01 | 31.7 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=H2O, Cache Budget=5122026.01 | 31.6 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=infLLM, Cache Budget=5122026.01 | 24.1 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Quest, Cache Budget=1282026.01 | 23.2 | 0.5 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Quest, Cache Budget=5122026.01 | 20.3 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Quest, Cache Budget=2562026.01 | 19.7 | -0.5 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Quest, Cache Budget=1282026.01 | 18.4 | -0.5 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=infLLM, Cache Budget=2562026.01 | 16.8 | 0.5 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Sink, Cache Budget=5122026.01 | 16.1 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=infLLM, Cache Budget=10242026.01 | 16 | -2 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Quest, Cache Budget=10242026.01 | 14.5 | -1 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=infLLM, Cache Budget=1282026.01 | 14.1 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=H2O, Cache Budget=10242026.01 | 11 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=infLLM, Cache Budget=5122026.01 | 10 | -0.5 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Sink, Cache Budget=10242026.01 | 6 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=infLLM, Cache Budget=1282026.01 | 4.8 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=infLLM, Cache Budget=2562026.01 | 3.9 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=infLLM, Cache Budget=10242026.01 | 2.7 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Quest, Cache Budget=10242026.01 | 1.6 | 0.5 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Quest, Cache Budget=1282026.01 | 1.4 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Quest, Cache Budget=2562026.01 | 1.1 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=infLLM, Cache Budget=5122026.01 | 1.1 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=infLLM, Cache Budget=2562026.01 | 0.8 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Quest, Cache Budget=5122026.01 | 0.6 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=infLLM, Cache Budget=1282026.01 | 0.2 | 0 |