List your company
ResearchFormerly WizWand

Commonsense Reasoning on CommonsenseQA (CSQA)

85.7AccuracyClaude-Sonnet-4.5
16.43634.41852.470.382May 19, 2025Jul 23, 2025Sep 27, 2025Dec 1, 2025Feb 5, 2026Apr 11, 2026Jun 16, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Claude-Sonnet-4.5Category=Candidate Models2026.0385.7
Qwen3-235B-A22BCategory=Candidate Models2026.0385.2
FineRouterCategory=Routers2026.0383.3
IPRCategory=Routers2026.0381.7
DeepSeek-R1Category=Candidate Models2026.0381
Llama-4-MaverickCategory=Candidate Models2026.0379.9
Claude-Haiku-4.5Category=Candidate Models2026.0379.8
kNNCategory=Routers2026.0379.8
Latent Thinking OptimizationModel=Llama-3-8B2025.0979
Qwen3-32BCategory=Candidate Models2026.0378.9
GPT-OSS-120BCategory=Candidate Models2026.0378.7
GraphRouterCategory=Routers2026.0378.7
Majority VotingModel=Llama-3-8B2025.0978.6
RouteLLMCategory=Routers2026.0377.1
RouterDCCategory=Routers2026.0376.4
Llama-3.3-70BCategory=Candidate Models2026.0376
Mistral-LargeCategory=Candidate Models2026.0375.8
Base ModelModel=Llama-3-8B2025.0974.2
DeepSeek-v3Category=Candidate Models2026.0370.7
BaselineFormat=Baseline, Bit width (b)=16.00, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.0570.2
Tensor RMS + CFormat=Tensor RMS + C, Bit width (b)=3.00, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.0567.7
Tensor RMS + SpFormat=Tensor RMS + Sp, Bit width (b)=3.05, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.0566.3
Latent Thinking OptimizationModel=Llama-2-13B2025.0965
Block AbsmaxFormat=Block Absmax, Bit width (b)=3.25, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.0563.3
Latent Thinking OptimizationModel=Llama-2-7B2025.0960.6
MLPCategory=Routers2026.0360.2
Mistral-SmallCategory=Candidate Models2026.0359.2
Channel AbsmaxFormat=Channel Absmax, Bit width (b)=3.00, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.0558.8
SnapKVBackbone=Nemotron-Nano-8B, Budget=5122025.1253
ConSA (layer-wise)Target Sparsity (ρ)=0.50, Granularity=layer-wise, Model=1.7B2026.0652.99
ConSA (head-wise, single-layer)Target Sparsity (ρ)=0.50, Granularity=head-wise, Constraint Scope=single-layer, Model=1.7B2026.0652.09
H2OBackbone=Nemotron-Nano-8B, Budget=3842025.1252
Rule (layer-wise)Target Sparsity (ρ)=0.50, Granularity=layer-wise, Model=1.7B2026.0651.43
ConSA (head-wise, all-layers)Target Sparsity (ρ)=0.50, Granularity=head-wise, Constraint Scope=all-layers, Model=1.7B2026.0651.27
FullBackbone=Nemotron-Nano-8B2025.1251
H2OBackbone=Nemotron-Nano-8B, Budget=5122025.1251
SnapKVBackbone=Nemotron-Nano-8B, Budget=3842025.1251
Majority VotingModel=Llama-2-13B2025.0950.1
Dense FATarget Sparsity (ρ)=0, Model=1.7B2026.0650.04
SnapKVBackbone=Nemotron-Nano-8B, Budget=2562025.1250
StreamingLLMBackbone=Nemotron-Nano-8B, Budget=5122025.1250
Majority VotingModel=Llama-2-7B2025.0949.3
Rule (head-wise)Target Sparsity (ρ)=0.50, Granularity=head-wise, Model=1.7B2026.0649.22
H2OBackbone=Nemotron-Nano-8B, Budget=2562025.1249
SnapKVBackbone=Nemotron-Nano-8B, Budget=1282025.1249
H2OBackbone=Nemotron-Nano-8B, Budget=1282025.1247
KnormBackbone=Nemotron-Nano-8B, Budget=5122025.1246
StreamingLLMBackbone=Nemotron-Nano-8B, Budget=3842025.1246
KnormBackbone=Nemotron-Nano-8B, Budget=3842025.1244
StreamingLLMBackbone=Nemotron-Nano-8B, Budget=2562025.1244
RKVBackbone=Nemotron-Nano-8B, Budget=3842025.1242
Tensor AbsmaxFormat=Tensor Absmax, Bit width (b)=3.00, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.0541.1
RKVBackbone=Nemotron-Nano-8B, Budget=5122025.1241
KnormBackbone=Nemotron-Nano-8B, Budget=2562025.1240
Base ModelModel=Llama-2-7B2025.0939.9
Base ModelModel=Llama-2-13B2025.0939.8
KnormBackbone=Nemotron-Nano-8B, Budget=1282025.1236
StreamingLLMBackbone=Nemotron-Nano-8B, Budget=1282025.1236
RKVBackbone=Nemotron-Nano-8B, Budget=2562025.1230
RKVBackbone=Nemotron-Nano-8B, Budget=1282025.1228
ShadowKVBackbone=Nemotron-Nano-8B2025.1220
Tensor RMSFormat=Tensor RMS, Bit width (b)=3.00, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.0519.1