Commonsense Reasoning on ARC-C, ARC-E, HellaSwag, LAMBADA, PIQA, WinoGrande
41.47ARC-C AccuracyRAT+
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| RAT+Dilation (D)=2, FLOPs=T/2, Training Context Length (L*)=64, Training Protocol=Adapted from pretrained dense models2026.02 | 41.47 | 73.23 | 59.55 | 50.36 | 73.34 | 56.51 | 59.08 | — | |
| RAT+Dilation (D)=4, FLOPs=T/4, Training Context Length (L*)=64, Training Protocol=Adapted from pretrained dense models2026.02 | 41.21 | 72.98 | 59.46 | 49.18 | 73.39 | 56.75 | 58.83 | — | |
| RAT+Dilation (D)=8, FLOPs=T/8, Training Context Length (L*)=64, Training Protocol=Adapted from pretrained dense models2026.02 | 40.96 | 72.98 | 59.09 | 48.52 | 73.23 | 54.7 | 58.25 | — | |
| RAT+Dilation (D)=1, FLOPs=T, Training Context Length (L*)=64, Training Protocol=Adapted from pretrained dense models2026.02 | 40.78 | 73.23 | 59.81 | 50.34 | 73.56 | 57.54 | 59.21 | — | |
| RAT+Dilation (D)=32, FLOPs=T/32, Training Context Length (L*)=64, Training Protocol=Adapted from pretrained dense models2026.02 | 40.53 | 73.19 | 58.2 | 45.74 | 72.85 | 56.83 | 57.89 | — | |
| RAT+Dilation (D)=16, FLOPs=T/16, Training Context Length (L*)=64, Training Protocol=Adapted from pretrained dense models2026.02 | 40.44 | 73.06 | 58.89 | 47.72 | 72.85 | 55.41 | 58.06 | — | |
| attentionDilation (D)=1, FLOPs=T, Training Protocol=Adapted from pretrained dense models2026.02 | 40.1 | 71.84 | 58.5 | 49.95 | 72.42 | 57.14 | 58.33 | — | |
| RAT+Dilation (D)=64, FLOPs=T/64, Training Context Length (L*)=64, Training Protocol=Adapted from pretrained dense models2026.02 | 39.76 | 72.9 | 57.93 | 44.05 | 72.91 | 57.22 | 57.46 | — | |
| RATFLOPs=T/16, Training Protocol=Trained from scratch2026.02 | 39.76 | 72.6 | 56.95 | 46.03 | 72.14 | 54.46 | 56.99 | — | |
| RAT+Dilation (D)=128, FLOPs=T/128, Training Context Length (L*)=64, Training Protocol=Adapted from pretrained dense models, Mechanism=Length generalization of recurrence2026.02 | 39.59 | 73.19 | 57.37 | 41.8 | 72.91 | 57.69 | 57.09 | — | |
| GatedDeltaNetFLOPs=288, Training Protocol=Trained from scratch2026.02 | 38.39 | 71.21 | 55.76 | 46.65 | 72.25 | 57.45 | 56.95 | — | |
| Mamba2FLOPs=256, Training Protocol=Trained from scratch2026.02 | 37.88 | 72.47 | 55.67 | 45.66 | 71.87 | 55.24 | 56.47 | — | |
| BWLAModel=Qwen3-14B, Activation Quantization=4-bit, Weight Quantization=Binarized (1-bit)2026.05 | 29.95 | 44.61 | 44.99 | 29.73 | 61.97 | 53.83 | — | 23.73 | |
| GPTQModel=LLaMA2-7B, Activation Quantization=4-bit, Weight Quantization=3-bit2026.05 | 28.07 | 26.05 | 25.43 | 0 | 50.92 | 50.91 | — | 0 | |
| BWLAModel=LLaMA3-8B, Activation Quantization=4-bit, Weight Quantization=Binarized (1-bit)2026.05 | 27.93 | 43.54 | 40.27 | 17.57 | 56.42 | 52.28 | — | 17.82 | |
| BWLAModel=LLaMA2-7B, Activation Quantization=4-bit, Weight Quantization=Binarized (1-bit)2026.05 | 27.12 | 39.21 | 39.31 | 15.3 | 55.72 | 52.09 | — | 16.4 | |
| ARB-LLMModel=Qwen3-14B, Activation Quantization=4-bit, Weight Quantization=Binarized (1-bit)2026.05 | 27.05 | 25.67 | 26.35 | 0 | 50.6 | 53.59 | — | 0 | |
| BiLLMModel=Qwen3-14B, Activation Quantization=4-bit, Weight Quantization=Binarized (1-bit)2026.05 | 26.88 | 24.07 | 25.64 | 0 | 51.03 | 51.07 | — | 0 | |
| ARB-LLMModel=LLaMA3-8B, Activation Quantization=4-bit, Weight Quantization=Binarized (1-bit)2026.05 | 26.62 | 28.45 | 25.66 | 0.27 | 50.82 | 48.38 | — | 0.08 | |
| GPTQModel=LLaMA3-8B, Activation Quantization=4-bit, Weight Quantization=3-bit2026.05 | 26.45 | 24.45 | 25.97 | 0 | 51.41 | 50.43 | — | 0 | |
| ARB-LLMModel=LLaMA2-7B, Activation Quantization=4-bit, Weight Quantization=Binarized (1-bit)2026.05 | 25.77 | 27.1 | 26.11 | 0 | 50.6 | 47.83 | — | 0 | |
| BiLLMModel=LLaMA2-7B, Activation Quantization=4-bit, Weight Quantization=Binarized (1-bit)2026.05 | 25.68 | 26.01 | 25.27 | 0 | 48.97 | 50.51 | — | 0 | |
| BiLLMModel=LLaMA3-8B, Activation Quantization=4-bit, Weight Quantization=Binarized (1-bit)2026.05 | 25.6 | 26.43 | 26.89 | 0.04 | 50.16 | 49.49 | — | 0.12 | |
| attentionDilation (D)=16, FLOPs=T/16, Training Protocol=Adapted from pretrained dense models2026.02 | 23.04 | 35.23 | 28.58 | 0.99 | 56.53 | 51.22 | 32.6 | — | |
| GPTQModel=Qwen3-14B, Activation Quantization=4-bit, Weight Quantization=3-bit2026.05 | 22.7 | 25.08 | 25.04 | 0 | 49.51 | 49.57 | — | 0 |