List your company
ResearchFormerly WizWand

LLM Decoding on Long Context 64K

1,042.6Decoding Throughput (tok/s)SparDA
-13.416260.742534.9809.058Jun 3, 2026
Updated 4mo ago

Evaluation results

MethodLinks
SparDAModel=NOSA-8B, Batch size (B)=64, Offloading=Yes, GPU=NVIDIA A1002026.061,042.6
SparDAModel=NOSA-8B, Batch size (B)=128, Offloading=Yes, GPU=NVIDIA A1002026.06961.3
SparseModel=NOSA-8B, Batch size (B)=64, Offloading=Yes, GPU=NVIDIA A1002026.06911.8
SparDAModel=NOSA-8B, Batch size (B)=32, Offloading=Yes, GPU=NVIDIA A1002026.06887.8
SparseModel=NOSA-8B, Batch size (B)=128, Offloading=Yes, GPU=NVIDIA A1002026.06861.3
SparseModel=NOSA-8B, Batch size (B)=32, Offloading=Yes, GPU=NVIDIA A1002026.06704.8
SparDAModel=MiniCPM4.1-8B, Batch size (B)=64, Offloading=Yes, GPU=NVIDIA A1002026.06593.1
SparDAModel=MiniCPM4.1-8B, Batch size (B)=128, Offloading=Yes, GPU=NVIDIA A1002026.06578.4
SparDAModel=NOSA-8B, Batch size (B)=16, Offloading=Yes, GPU=NVIDIA A1002026.06577.3
SparDAModel=MiniCPM4.1-8B, Batch size (B)=32, Offloading=Yes, GPU=NVIDIA A1002026.06573.7
SparseModel=MiniCPM4.1-8B, Batch size (B)=128, Offloading=Yes, GPU=NVIDIA A1002026.06556.6
SparseModel=MiniCPM4.1-8B, Batch size (B)=64, Offloading=Yes, GPU=NVIDIA A1002026.06516.1
SparseModel=NOSA-8B, Batch size (B)=16, Offloading=Yes, GPU=NVIDIA A1002026.06479.8
SparDAModel=MiniCPM4.1-8B, Batch size (B)=16, Offloading=Yes, GPU=NVIDIA A1002026.06457.8
SparseModel=MiniCPM4.1-8B, Batch size (B)=32, Offloading=Yes, GPU=NVIDIA A1002026.06443.9
SparseModel=MiniCPM4.1-8B, Batch size (B)=16, Offloading=Yes, GPU=NVIDIA A1002026.06345.6
SparseModel=MiniCPM4.1-8B, Batch size (B)=8, Offloading=No, GPU=NVIDIA A1002026.06344.6
SparDAModel=MiniCPM4.1-8B, Batch size (B)=8, Offloading=Yes, GPU=NVIDIA A1002026.06316.3
SparseModel=NOSA-8B, Batch size (B)=8, Offloading=No, GPU=NVIDIA A1002026.06310.3
SparDAModel=NOSA-8B, Batch size (B)=8, Offloading=Yes, GPU=NVIDIA A1002026.06292.8
SparseModel=NOSA-8B, Batch size (B)=8, Offloading=Yes, GPU=NVIDIA A1002026.06284.6
SparseModel=MiniCPM4.1-8B, Batch size (B)=8, Offloading=Yes, GPU=NVIDIA A1002026.06236.7
SparseModel=MiniCPM4.1-8B, Batch size (B)=4, Offloading=No, GPU=NVIDIA A1002026.06173.2
SparDAModel=MiniCPM4.1-8B, Batch size (B)=4, Offloading=Yes, GPU=NVIDIA A1002026.06160.3
SparseModel=NOSA-8B, Batch size (B)=4, Offloading=Yes, GPU=NVIDIA A1002026.06158.5
SparseModel=NOSA-8B, Batch size (B)=4, Offloading=No, GPU=NVIDIA A1002026.06157.4
SparseModel=MiniCPM4.1-8B, Batch size (B)=4, Offloading=Yes, GPU=NVIDIA A1002026.06147.3
SparDAModel=NOSA-8B, Batch size (B)=4, Offloading=Yes, GPU=NVIDIA A1002026.06146.9
InfiniGenModel=NOSA-8B, Batch size (B)=32, Offloading=Yes, GPU=NVIDIA A1002026.06138.1
DenseModel=MiniCPM4.1-8B, Batch size (B)=8, Offloading=No, GPU=NVIDIA A1002026.06134.8
DenseModel=NOSA-8B, Batch size (B)=8, Offloading=No, GPU=NVIDIA A1002026.06133.7
InfiniGenModel=NOSA-8B, Batch size (B)=16, Offloading=Yes, GPU=NVIDIA A1002026.06133.5
DenseModel=MiniCPM4.1-8B, Batch size (B)=4, Offloading=No, GPU=NVIDIA A1002026.06109.6
DenseModel=NOSA-8B, Batch size (B)=4, Offloading=No, GPU=NVIDIA A1002026.06109.5
InfiniGenModel=NOSA-8B, Batch size (B)=8, Offloading=Yes, GPU=NVIDIA A1002026.06100.1
InfiniGenModel=MiniCPM4.1-8B, Batch size (B)=8, Offloading=Yes, GPU=NVIDIA A1002026.0667.5
InfiniGenModel=MiniCPM4.1-8B, Batch size (B)=16, Offloading=Yes, GPU=NVIDIA A1002026.0666.9
InfiniGenModel=MiniCPM4.1-8B, Batch size (B)=32, Offloading=Yes, GPU=NVIDIA A1002026.0663.3
InfiniGenModel=NOSA-8B, Batch size (B)=4, Offloading=Yes, GPU=NVIDIA A1002026.0663.3
InfiniGenModel=NOSA-8B, Batch size (B)=64, Offloading=Yes, GPU=NVIDIA A1002026.0662.5
InfiniGenModel=MiniCPM4.1-8B, Batch size (B)=4, Offloading=Yes, GPU=NVIDIA A1002026.0654
InfiniGenModel=MiniCPM4.1-8B, Batch size (B)=64, Offloading=Yes, GPU=NVIDIA A1002026.0627.2