List your company
ResearchFormerly WizWand

Language Modeling on LAMBADA standard (LS) (Accuracy)

73.67Accuracy (LAMBADA)TWLA
-2.946816.944136.83556.7259Apr 8, 2026Apr 18, 2026Apr 29, 2026May 10, 2026May 20, 2026May 31, 2026Jun 11, 2026
Updated 4mo ago

Evaluation results

MethodLinks
TWLABackbone=LLaMA2-70B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.0673.67
TWLABackbone=LLaMA2-13B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.0666.82
BaselineModel=OLMoE, Weight Bits=16, Zero-shot=true2026.0465.57
TWLABackbone=Qwen3-14B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.0663.95
BaselineModel=Qwen3-MoE, Weight Bits=16, Zero-shot=true2026.0463.26
BaselineModel=DeepSeekV2-Lite, Weight Bits=16, Zero-shot=true2026.0461.36
TWLABackbone=LLaMA2-7B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.0660.94
TWLABackbone=LLaMA3-8B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.0659.21
TWLABackbone=Qwen3-32B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.0657.33
TWLABackbone=Qwen3-8B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.0655.13
MoBiEModel=Qwen3-MoE, Weight Bits=1.34, Zero-shot=true2026.0445.55
GPTQModel=Qwen3-MoE, Weight Bits=3, Zero-shot=true2026.0441.7
NoWagModel=Qwen3-MoE, Weight Bits=2.11, Zero-shot=true2026.0439.31
MoBiEModel=OLMoE, Weight Bits=1.46, Zero-shot=true2026.0429.85
MoBiEModel=DeepSeekV2-Lite, Weight Bits=1.47, Zero-shot=true2026.0428.62
ARB-LLMModel=DeepSeekV2-Lite, Weight Bits=1.11, Zero-shot=true2026.0427.75
GPTQModel=OLMoE, Weight Bits=3, Zero-shot=true2026.0425.88
GPTQModel=DeepSeekV2-Lite, Weight Bits=3, Zero-shot=true2026.0425.88
ARB-LLMModel=OLMoE, Weight Bits=1.11, Zero-shot=true2026.0424.98
NoWagModel=OLMoE, Weight Bits=2.08, Zero-shot=true2026.0421.77
ARB-LLMModel=Qwen3-MoE, Weight Bits=1.11, Zero-shot=true2026.0420.05
NoWagModel=DeepSeekV2-Lite, Weight Bits=2.10, Zero-shot=true2026.0419.31
BiLLMModel=DeepSeekV2-Lite, Weight Bits=1.11, Zero-shot=true2026.0419.1
BiLLMModel=Qwen3-MoE, Weight Bits=1.11, Zero-shot=true2026.0414.56
BiLLMModel=OLMoE, Weight Bits=1.11, Zero-shot=true2026.0414.13
MoEQuantModel=Qwen3-MoE, Weight Bits=2, Zero-shot=true2026.0410.15
MoEQuantModel=OLMoE, Weight Bits=2, Zero-shot=true2026.0410.11
MoEQuantModel=DeepSeekV2-Lite, Weight Bits=2, Zero-shot=true2026.048.2
AWQModel=Qwen3-MoE, Weight Bits=3, Zero-shot=true2026.041.22
AWQModel=OLMoE, Weight Bits=3, Zero-shot=true2026.040.55
AWQModel=DeepSeekV2-Lite, Weight Bits=3, Zero-shot=true2026.040.26
AWQModel=OLMoE, Weight Bits=2, Zero-shot=true2026.040
GPTQModel=OLMoE, Weight Bits=2, Zero-shot=true2026.040
AWQModel=DeepSeekV2-Lite, Weight Bits=2, Zero-shot=true2026.040
GPTQModel=DeepSeekV2-Lite, Weight Bits=2, Zero-shot=true2026.040
AWQModel=Qwen3-MoE, Weight Bits=2, Zero-shot=true2026.040
GPTQModel=Qwen3-MoE, Weight Bits=2, Zero-shot=true2026.040