Mathematical Reasoning on GSM8K (Token Savings & Accuracy Shift)
68.1Token SavingsGuardian
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GuardianModel Backbone=DSR, Sparsity Algorithm=Sink, Cache Budget=1282026.01 | 68.1 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=H2O, Cache Budget=1282026.01 | 67.8 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Sink, Cache Budget=2562026.01 | 64.9 | 0.5 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=H2O, Cache Budget=2562026.01 | 62.1 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Sink, Cache Budget=5122026.01 | 53.9 | 1 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=H2O, Cache Budget=5122026.01 | 49.3 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=H2O, Cache Budget=10242026.01 | 36.9 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Sink, Cache Budget=10242026.01 | 32.8 | -1.5 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=H2O, Cache Budget=2562026.01 | 23.4 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=H2O, Cache Budget=1282026.01 | 22.7 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=H2O, Cache Budget=5122026.01 | 18.5 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=infLLM, Cache Budget=10242026.01 | 10.9 | -1.5 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Quest, Cache Budget=2562026.01 | 10.2 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Sink, Cache Budget=2562026.01 | 10.2 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Quest, Cache Budget=10242026.01 | 9.8 | -2.5 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=H2O, Cache Budget=10242026.01 | 9.6 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Sink, Cache Budget=1282026.01 | 8.1 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Quest, Cache Budget=1282026.01 | 6.7 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=infLLM, Cache Budget=1282026.01 | 6.4 | 0 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=infLLM, Cache Budget=5122026.01 | 6.4 | -0.5 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Sink, Cache Budget=5122026.01 | 6.4 | 1 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=Quest, Cache Budget=5122026.01 | 6.1 | -1 | |
| GuardianModel Backbone=DSR, Sparsity Algorithm=infLLM, Cache Budget=2562026.01 | 5.4 | -0.5 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Quest, Cache Budget=2562026.01 | 3.6 | -0.5 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Sink, Cache Budget=10242026.01 | 3.6 | -0.5 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Quest, Cache Budget=5122026.01 | 2.4 | 0.5 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Sink, Cache Budget=2562026.01 | 1.7 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Quest, Cache Budget=1282026.01 | 1.6 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=Quest, Cache Budget=10242026.01 | 1.5 | -0.5 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=H2O, Cache Budget=5122026.01 | 1.3 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Sink, Cache Budget=1282026.01 | 1.1 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=H2O, Cache Budget=1282026.01 | 0.9 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=infLLM, Cache Budget=2562026.01 | 0.7 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=H2O, Cache Budget=2562026.01 | 0.5 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=infLLM, Cache Budget=1282026.01 | 0.4 | 0 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=infLLM, Cache Budget=5122026.01 | 0.3 | -0.5 | |
| GuardianModel Backbone=DSL, Sparsity Algorithm=infLLM, Cache Budget=10242026.01 | 0.3 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=H2O, Cache Budget=10242026.01 | 0.2 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Sink, Cache Budget=5122026.01 | 0.1 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Quest, Cache Budget=1282026.01 | 0 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=infLLM, Cache Budget=1282026.01 | 0 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Quest, Cache Budget=2562026.01 | 0 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=infLLM, Cache Budget=2562026.01 | 0 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Quest, Cache Budget=5122026.01 | 0 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=infLLM, Cache Budget=5122026.01 | 0 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Sink, Cache Budget=10242026.01 | 0 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=Quest, Cache Budget=10242026.01 | 0 | 0 | |
| GuardianModel Backbone=Qwe, Sparsity Algorithm=infLLM, Cache Budget=10242026.01 | 0 | 0 |