Language Modeling on LAMBADA standard (LS) (Accuracy)
73.67Accuracy (LAMBADA)TWLA
Evaluation Results
| Method | Links | |
|---|---|---|
| TWLABackbone=LLaMA2-70B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 73.67 | |
| TWLABackbone=LLaMA2-13B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 66.82 | |
| BaselineModel=OLMoE, Weight Bits=16, Zero-shot=true2026.04 | 65.57 | |
| TWLABackbone=Qwen3-14B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 63.95 | |
| BaselineModel=Qwen3-MoE, Weight Bits=16, Zero-shot=true2026.04 | 63.26 | |
| BaselineModel=DeepSeekV2-Lite, Weight Bits=16, Zero-shot=true2026.04 | 61.36 | |
| TWLABackbone=LLaMA2-7B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 60.94 | |
| TWLABackbone=LLaMA3-8B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 59.21 | |
| TWLABackbone=Qwen3-32B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 57.33 | |
| TWLABackbone=Qwen3-8B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.06 | 55.13 | |
| MoBiEModel=Qwen3-MoE, Weight Bits=1.34, Zero-shot=true2026.04 | 45.55 | |
| GPTQModel=Qwen3-MoE, Weight Bits=3, Zero-shot=true2026.04 | 41.7 | |
| NoWagModel=Qwen3-MoE, Weight Bits=2.11, Zero-shot=true2026.04 | 39.31 | |
| MoBiEModel=OLMoE, Weight Bits=1.46, Zero-shot=true2026.04 | 29.85 | |
| MoBiEModel=DeepSeekV2-Lite, Weight Bits=1.47, Zero-shot=true2026.04 | 28.62 | |
| ARB-LLMModel=DeepSeekV2-Lite, Weight Bits=1.11, Zero-shot=true2026.04 | 27.75 | |
| GPTQModel=OLMoE, Weight Bits=3, Zero-shot=true2026.04 | 25.88 | |
| GPTQModel=DeepSeekV2-Lite, Weight Bits=3, Zero-shot=true2026.04 | 25.88 | |
| ARB-LLMModel=OLMoE, Weight Bits=1.11, Zero-shot=true2026.04 | 24.98 | |
| NoWagModel=OLMoE, Weight Bits=2.08, Zero-shot=true2026.04 | 21.77 | |
| ARB-LLMModel=Qwen3-MoE, Weight Bits=1.11, Zero-shot=true2026.04 | 20.05 | |
| NoWagModel=DeepSeekV2-Lite, Weight Bits=2.10, Zero-shot=true2026.04 | 19.31 | |
| BiLLMModel=DeepSeekV2-Lite, Weight Bits=1.11, Zero-shot=true2026.04 | 19.1 | |
| BiLLMModel=Qwen3-MoE, Weight Bits=1.11, Zero-shot=true2026.04 | 14.56 | |
| BiLLMModel=OLMoE, Weight Bits=1.11, Zero-shot=true2026.04 | 14.13 | |
| MoEQuantModel=Qwen3-MoE, Weight Bits=2, Zero-shot=true2026.04 | 10.15 | |
| MoEQuantModel=OLMoE, Weight Bits=2, Zero-shot=true2026.04 | 10.11 | |
| MoEQuantModel=DeepSeekV2-Lite, Weight Bits=2, Zero-shot=true2026.04 | 8.2 | |
| AWQModel=Qwen3-MoE, Weight Bits=3, Zero-shot=true2026.04 | 1.22 | |
| AWQModel=OLMoE, Weight Bits=3, Zero-shot=true2026.04 | 0.55 | |
| AWQModel=DeepSeekV2-Lite, Weight Bits=3, Zero-shot=true2026.04 | 0.26 | |
| AWQModel=OLMoE, Weight Bits=2, Zero-shot=true2026.04 | 0 | |
| GPTQModel=OLMoE, Weight Bits=2, Zero-shot=true2026.04 | 0 | |
| AWQModel=DeepSeekV2-Lite, Weight Bits=2, Zero-shot=true2026.04 | 0 | |
| GPTQModel=DeepSeekV2-Lite, Weight Bits=2, Zero-shot=true2026.04 | 0 | |
| AWQModel=Qwen3-MoE, Weight Bits=2, Zero-shot=true2026.04 | 0 | |
| GPTQModel=Qwen3-MoE, Weight Bits=2, Zero-shot=true2026.04 | 0 |