List your company
ResearchFormerly WizWand

Mathematical Reasoning on GSM8K (test) (Latency, Throughput)

52Latency (min)SkipKV
34.08155.04276396.96Dec 8, 2025
Updated 8mo ago

Evaluation results

MethodLinks
SkipKVBatch-size=100, Model=R1-Qwen-7B, Hardware=A100-40GB, Maximum generation length=8,192 tokens, KV-cache budget=512, Attention Mechanism=FlashAttention-22025.125225.4
SkipKVBatch-size=50, Model=R1-Qwen-7B, Hardware=A100-40GB, Maximum generation length=8,192 tokens, KV-cache budget=512, Attention Mechanism=FlashAttention-22025.125822.7
R-KVBatch-size=100, Model=R1-Qwen-7B, Hardware=A100-40GB, Maximum generation length=8,192 tokens, KV-cache budget=512, Attention Mechanism=FlashAttention-22025.126620
SkipKVBatch-size=120, Model=R1-Qwen-7B, Hardware=A100-40GB, Maximum generation length=8,192 tokens, KV-cache budget=512, Attention Mechanism=FlashAttention-22025.126620
SkipKVBatch-size=140, Model=R1-Qwen-7B, Hardware=A100-40GB, Maximum generation length=8,192 tokens, KV-cache budget=512, Attention Mechanism=FlashAttention-22025.126819.4
R-KVBatch-size=140, Model=R1-Qwen-7B, Hardware=A100-40GB, Maximum generation length=8,192 tokens, KV-cache budget=512, Attention Mechanism=FlashAttention-22025.127018.8
R-KVBatch-size=120, Model=R1-Qwen-7B, Hardware=A100-40GB, Maximum generation length=8,192 tokens, KV-cache budget=512, Attention Mechanism=FlashAttention-22025.127318.1
R-KVBatch-size=50, Model=R1-Qwen-7B, Hardware=A100-40GB, Maximum generation length=8,192 tokens, KV-cache budget=512, Attention Mechanism=FlashAttention-22025.129613.7
SkipKVBatch-size=10, Model=R1-Qwen-7B, Hardware=A100-40GB, Maximum generation length=8,192 tokens, KV-cache budget=512, Attention Mechanism=FlashAttention-22025.121369.7
SEALBatch-size=10, Model=R1-Qwen-7B, Hardware=A100-40GB, Maximum generation length=8,192 tokens, Steering factor=1, Attention Mechanism=FlashAttention-22025.121787.41
SEALBatch-size=50, Model=R1-Qwen-7B, Hardware=A100-40GB, Maximum generation length=8,192 tokens, Steering factor=1, Attention Mechanism=FlashAttention-22025.121926.87
R-KVBatch-size=10, Model=R1-Qwen-7B, Hardware=A100-40GB, Maximum generation length=8,192 tokens, KV-cache budget=512, Attention Mechanism=FlashAttention-22025.122275.81
FullKVBatch-size=10, Model=R1-Qwen-7B, Hardware=A100-40GB, Maximum generation length=8,192 tokens, Attention Mechanism=FlashAttention-22025.123244.07
FullKVBatch-size=50, Model=R1-Qwen-7B, Hardware=A100-40GB, Maximum generation length=8,192 tokens, Attention Mechanism=FlashAttention-22025.125002.64