Commonsense Reasoning on CommonsenseQA (CSQA)
85.7AccuracyClaude-Sonnet-4.5
Evaluation Results
| Method | Links | |
|---|---|---|
| Claude-Sonnet-4.5Category=Candidate Models2026.03 | 85.7 | |
| Qwen3-235B-A22BCategory=Candidate Models2026.03 | 85.2 | |
| FineRouterCategory=Routers2026.03 | 83.3 | |
| IPRCategory=Routers2026.03 | 81.7 | |
| DeepSeek-R1Category=Candidate Models2026.03 | 81 | |
| Llama-4-MaverickCategory=Candidate Models2026.03 | 79.9 | |
| Claude-Haiku-4.5Category=Candidate Models2026.03 | 79.8 | |
| kNNCategory=Routers2026.03 | 79.8 | |
| Latent Thinking OptimizationModel=Llama-3-8B2025.09 | 79 | |
| Qwen3-32BCategory=Candidate Models2026.03 | 78.9 | |
| GPT-OSS-120BCategory=Candidate Models2026.03 | 78.7 | |
| GraphRouterCategory=Routers2026.03 | 78.7 | |
| Majority VotingModel=Llama-3-8B2025.09 | 78.6 | |
| RouteLLMCategory=Routers2026.03 | 77.1 | |
| RouterDCCategory=Routers2026.03 | 76.4 | |
| Llama-3.3-70BCategory=Candidate Models2026.03 | 76 | |
| Mistral-LargeCategory=Candidate Models2026.03 | 75.8 | |
| Base ModelModel=Llama-3-8B2025.09 | 74.2 | |
| DeepSeek-v3Category=Candidate Models2026.03 | 70.7 | |
| BaselineFormat=Baseline, Bit width (b)=16.00, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.05 | 70.2 | |
| Tensor RMS + CFormat=Tensor RMS + C, Bit width (b)=3.00, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.05 | 67.7 | |
| Tensor RMS + SpFormat=Tensor RMS + Sp, Bit width (b)=3.05, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.05 | 66.3 | |
| Latent Thinking OptimizationModel=Llama-2-13B2025.09 | 65 | |
| Block AbsmaxFormat=Block Absmax, Bit width (b)=3.25, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.05 | 63.3 | |
| Latent Thinking OptimizationModel=Llama-2-7B2025.09 | 60.6 | |
| MLPCategory=Routers2026.03 | 60.2 | |
| Mistral-SmallCategory=Candidate Models2026.03 | 59.2 | |
| Channel AbsmaxFormat=Channel Absmax, Bit width (b)=3.00, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.05 | 58.8 | |
| SnapKVBackbone=Nemotron-Nano-8B, Budget=5122025.12 | 53 | |
| ConSA (layer-wise)Target Sparsity (ρ)=0.50, Granularity=layer-wise, Model=1.7B2026.06 | 52.99 | |
| ConSA (head-wise, single-layer)Target Sparsity (ρ)=0.50, Granularity=head-wise, Constraint Scope=single-layer, Model=1.7B2026.06 | 52.09 | |
| H2OBackbone=Nemotron-Nano-8B, Budget=3842025.12 | 52 | |
| Rule (layer-wise)Target Sparsity (ρ)=0.50, Granularity=layer-wise, Model=1.7B2026.06 | 51.43 | |
| ConSA (head-wise, all-layers)Target Sparsity (ρ)=0.50, Granularity=head-wise, Constraint Scope=all-layers, Model=1.7B2026.06 | 51.27 | |
| FullBackbone=Nemotron-Nano-8B2025.12 | 51 | |
| H2OBackbone=Nemotron-Nano-8B, Budget=5122025.12 | 51 | |
| SnapKVBackbone=Nemotron-Nano-8B, Budget=3842025.12 | 51 | |
| Majority VotingModel=Llama-2-13B2025.09 | 50.1 | |
| Dense FATarget Sparsity (ρ)=0, Model=1.7B2026.06 | 50.04 | |
| SnapKVBackbone=Nemotron-Nano-8B, Budget=2562025.12 | 50 | |
| StreamingLLMBackbone=Nemotron-Nano-8B, Budget=5122025.12 | 50 | |
| Majority VotingModel=Llama-2-7B2025.09 | 49.3 | |
| Rule (head-wise)Target Sparsity (ρ)=0.50, Granularity=head-wise, Model=1.7B2026.06 | 49.22 | |
| H2OBackbone=Nemotron-Nano-8B, Budget=2562025.12 | 49 | |
| SnapKVBackbone=Nemotron-Nano-8B, Budget=1282025.12 | 49 | |
| H2OBackbone=Nemotron-Nano-8B, Budget=1282025.12 | 47 | |
| KnormBackbone=Nemotron-Nano-8B, Budget=5122025.12 | 46 | |
| StreamingLLMBackbone=Nemotron-Nano-8B, Budget=3842025.12 | 46 | |
| KnormBackbone=Nemotron-Nano-8B, Budget=3842025.12 | 44 | |
| StreamingLLMBackbone=Nemotron-Nano-8B, Budget=2562025.12 | 44 | |
| RKVBackbone=Nemotron-Nano-8B, Budget=3842025.12 | 42 | |
| Tensor AbsmaxFormat=Tensor Absmax, Bit width (b)=3.00, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.05 | 41.1 | |
| RKVBackbone=Nemotron-Nano-8B, Budget=5122025.12 | 41 | |
| KnormBackbone=Nemotron-Nano-8B, Budget=2562025.12 | 40 | |
| Base ModelModel=Llama-2-7B2025.09 | 39.9 | |
| Base ModelModel=Llama-2-13B2025.09 | 39.8 | |
| KnormBackbone=Nemotron-Nano-8B, Budget=1282025.12 | 36 | |
| StreamingLLMBackbone=Nemotron-Nano-8B, Budget=1282025.12 | 36 | |
| RKVBackbone=Nemotron-Nano-8B, Budget=2562025.12 | 30 | |
| RKVBackbone=Nemotron-Nano-8B, Budget=1282025.12 | 28 | |
| ShadowKVBackbone=Nemotron-Nano-8B2025.12 | 20 | |
| Tensor RMSFormat=Tensor RMS, Bit width (b)=3.00, Quantization Aware Training (QAT)=true, Backbone=Llama 3.1 8B2025.05 | 19.1 |