List your company
ResearchFormerly WizWand

Mathematical Reasoning on GSM8K Var (Accuracy %)

35.44AccuracyFull-Context
-1.41768.151217.7227.2888Jan 29, 2026
Updated 8mo ago

Evaluation results

MethodLinks
Full-ContextModel=Qwen3-4B, Compression Ratio=-2026.0135.44
Full-ContextModel=Qwen3-1.7B, Compression Ratio=-2026.019.1
SONICModel=Qwen3-4B, Compression Ratio=50%2026.018.3
H2OModel=Qwen3-4B, Compression Ratio=50%2026.018
SONICModel=Qwen3-4B, Compression Ratio=80%2026.017.69
ExpectedAttentionModel=Qwen3-4B, Compression Ratio=50%2026.017.59
SnapKVModel=Qwen3-4B, Compression Ratio=50%2026.017.39
SONICModel=Qwen3-1.7B, Compression Ratio=50%2026.017
ExpectedAttentionModel=Qwen3-1.7B, Compression Ratio=50%2026.014.5
ExpectedAttentionModel=Qwen3-4B, Compression Ratio=80%2026.014.5
SnapKVModel=Qwen3-1.7B, Compression Ratio=50%2026.014.1
SONICModel=Qwen3-1.7B, Compression Ratio=80%2026.013.14
StreamingLLMModel=Qwen3-4B, Compression Ratio=50%2026.012.79
SONICModel=Qwen3-0.6B, Compression Ratio=50%2026.012.5
ExpectedAttentionModel=Qwen3-1.7B, Compression Ratio=80%2026.012.4
StreamingLLMModel=Qwen3-1.7B, Compression Ratio=50%2026.012.2
SnapKVModel=Qwen3-4B, Compression Ratio=80%2026.012.19
ExpectedAttentionModel=Qwen3-0.6B, Compression Ratio=50%2026.012.09
Full-ContextModel=Qwen3-0.6B, Compression Ratio=-2026.012.05
StreamingLLMModel=Qwen3-0.6B, Compression Ratio=50%2026.012
H2OModel=Qwen3-0.6B, Compression Ratio=50%2026.012
StreamingLLMModel=Qwen3-1.7B, Compression Ratio=80%2026.011.7
SnapKVModel=Qwen3-1.7B, Compression Ratio=80%2026.011.7
StreamingLLMModel=Qwen3-0.6B, Compression Ratio=80%2026.011.6
SONICModel=Qwen3-0.6B, Compression Ratio=80%2026.011.4
SnapKVModel=Qwen3-0.6B, Compression Ratio=50%2026.011.3
ExpectedAttentionModel=Qwen3-0.6B, Compression Ratio=80%2026.011.2
SnapKVModel=Qwen3-0.6B, Compression Ratio=80%2026.011.2
H2OModel=Qwen3-1.7B, Compression Ratio=50%2026.010.98
StreamingLLMModel=Qwen3-4B, Compression Ratio=80%2026.010.9
H2OModel=Qwen3-0.6B, Compression Ratio=80%2026.010
H2OModel=Qwen3-1.7B, Compression Ratio=80%2026.010
H2OModel=Qwen3-4B, Compression Ratio=80%2026.010