Mathematical Reasoning on GSM8K Var (Accuracy %)
35.44AccuracyFull-Context
Evaluation Results
| Method | Links | |
|---|---|---|
| Full-ContextModel=Qwen3-4B, Compression Ratio=-2026.01 | 35.44 | |
| Full-ContextModel=Qwen3-1.7B, Compression Ratio=-2026.01 | 9.1 | |
| SONICModel=Qwen3-4B, Compression Ratio=50%2026.01 | 8.3 | |
| H2OModel=Qwen3-4B, Compression Ratio=50%2026.01 | 8 | |
| SONICModel=Qwen3-4B, Compression Ratio=80%2026.01 | 7.69 | |
| ExpectedAttentionModel=Qwen3-4B, Compression Ratio=50%2026.01 | 7.59 | |
| SnapKVModel=Qwen3-4B, Compression Ratio=50%2026.01 | 7.39 | |
| SONICModel=Qwen3-1.7B, Compression Ratio=50%2026.01 | 7 | |
| ExpectedAttentionModel=Qwen3-1.7B, Compression Ratio=50%2026.01 | 4.5 | |
| ExpectedAttentionModel=Qwen3-4B, Compression Ratio=80%2026.01 | 4.5 | |
| SnapKVModel=Qwen3-1.7B, Compression Ratio=50%2026.01 | 4.1 | |
| SONICModel=Qwen3-1.7B, Compression Ratio=80%2026.01 | 3.14 | |
| StreamingLLMModel=Qwen3-4B, Compression Ratio=50%2026.01 | 2.79 | |
| SONICModel=Qwen3-0.6B, Compression Ratio=50%2026.01 | 2.5 | |
| ExpectedAttentionModel=Qwen3-1.7B, Compression Ratio=80%2026.01 | 2.4 | |
| StreamingLLMModel=Qwen3-1.7B, Compression Ratio=50%2026.01 | 2.2 | |
| SnapKVModel=Qwen3-4B, Compression Ratio=80%2026.01 | 2.19 | |
| ExpectedAttentionModel=Qwen3-0.6B, Compression Ratio=50%2026.01 | 2.09 | |
| Full-ContextModel=Qwen3-0.6B, Compression Ratio=-2026.01 | 2.05 | |
| StreamingLLMModel=Qwen3-0.6B, Compression Ratio=50%2026.01 | 2 | |
| H2OModel=Qwen3-0.6B, Compression Ratio=50%2026.01 | 2 | |
| StreamingLLMModel=Qwen3-1.7B, Compression Ratio=80%2026.01 | 1.7 | |
| SnapKVModel=Qwen3-1.7B, Compression Ratio=80%2026.01 | 1.7 | |
| StreamingLLMModel=Qwen3-0.6B, Compression Ratio=80%2026.01 | 1.6 | |
| SONICModel=Qwen3-0.6B, Compression Ratio=80%2026.01 | 1.4 | |
| SnapKVModel=Qwen3-0.6B, Compression Ratio=50%2026.01 | 1.3 | |
| ExpectedAttentionModel=Qwen3-0.6B, Compression Ratio=80%2026.01 | 1.2 | |
| SnapKVModel=Qwen3-0.6B, Compression Ratio=80%2026.01 | 1.2 | |
| H2OModel=Qwen3-1.7B, Compression Ratio=50%2026.01 | 0.98 | |
| StreamingLLMModel=Qwen3-4B, Compression Ratio=80%2026.01 | 0.9 | |
| H2OModel=Qwen3-0.6B, Compression Ratio=80%2026.01 | 0 | |
| H2OModel=Qwen3-1.7B, Compression Ratio=80%2026.01 | 0 | |
| H2OModel=Qwen3-4B, Compression Ratio=80%2026.01 | 0 |