Mathematical Reasoning on MATH-500 (Token Savings and Accuracy Shift)
84.1Token SavingsGuardian
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GuardianModel Backbone=DSR, Sparsity Algorithm=Sink, Cache Budget=1282026.01 | 84.1 | 1.5 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=H2O, Cache Budget=1282026.01 | 81.8 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Sink, Cache Budget=2562026.01 | 79.9 | 1 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=H2O, Cache Budget=2562026.01 | 78 | -0.5 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=H2O, Cache Budget=2562026.01 | 75.8 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=H2O, Cache Budget=1282026.01 | 75.5 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=H2O, Cache Budget=5122026.01 | 71 | 0.5 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Sink, Cache Budget=5122026.01 | 70.9 | -0.5 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=H2O, Cache Budget=5122026.01 | 66.8 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=H2O, Cache Budget=10242026.01 | 58.8 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=H2O, Cache Budget=10242026.01 | 53.6 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Sink, Cache Budget=10242026.01 | 51.8 | -0.5 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Sink, Cache Budget=1282026.01 | 47.9 | 1 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Sink, Cache Budget=2562026.01 | 45.8 | -0.5 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Sink, Cache Budget=5122026.01 | 35.7 | -1 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Sink, Cache Budget=10242026.01 | 29.9 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Quest, Cache Budget=5122026.01 | 26.3 | -1 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Quest, Cache Budget=2562026.01 | 23.9 | 2.5 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Quest, Cache Budget=10242026.01 | 22.3 | 2 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=H2O, Cache Budget=2562026.01 | 19.3 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=H2O, Cache Budget=1282026.01 | 16.7 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Quest, Cache Budget=1282026.01 | 15.1 | 1 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Sink, Cache Budget=1282026.01 | 14.2 | -0.5 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Sink, Cache Budget=2562026.01 | 13.7 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=H2O, Cache Budget=5122026.01 | 12.5 | 0.5 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=infLLM, Cache Budget=10242026.01 | 11.9 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Quest, Cache Budget=1282026.01 | 11.5 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Quest, Cache Budget=2562026.01 | 11.2 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=infLLM, Cache Budget=5122026.01 | 10.1 | 0.5 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=infLLM, Cache Budget=2562026.01 | 9 | -0.5 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Quest, Cache Budget=5122026.01 | 8.6 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=infLLM, Cache Budget=1282026.01 | 8.3 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Quest, Cache Budget=10242026.01 | 6 | -0.5 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Sink, Cache Budget=5122026.01 | 6 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=infLLM, Cache Budget=10242026.01 | 5.9 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=infLLM, Cache Budget=5122026.01 | 3.9 | -0.5 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=infLLM, Cache Budget=1282026.01 | 3.8 | 0.5 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=H2O, Cache Budget=10242026.01 | 3.2 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=infLLM, Cache Budget=2562026.01 | 1.7 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Sink, Cache Budget=10242026.01 | 0.8 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=infLLM, Cache Budget=10242026.01 | 0.6 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=infLLM, Cache Budget=2562026.01 | 0.4 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=infLLM, Cache Budget=5122026.01 | 0.4 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Quest, Cache Budget=1282026.01 | 0.2 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Quest, Cache Budget=10242026.01 | 0.2 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Quest, Cache Budget=5122026.01 | 0.1 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=infLLM, Cache Budget=1282026.01 | 0 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Quest, Cache Budget=2562026.01 | 0 | 0 |