List your company
ResearchFormerly WizWand

Multiple Choice Question Answering on WinoG

68.9AccuracyMxMoE
47.37252.96158.5564.139Oct 22, 2025Dec 2, 2025Jan 12, 2026Feb 22, 2026Apr 4, 2026May 15, 2026Jun 26, 2026
Updated 4mo ago

Evaluation results

MethodLinks
MxMoEBackbone=Mixtral, Weight Bitwidth=2.252026.0468.9
BaselineBackbone=Qwen1.5-MoE, Weight Bitwidth=162026.0468.82
BaselineBackbone=Mixtral, Weight Bitwidth=162026.0468.75
MoBiEBackbone=Mixtral, Weight Bitwidth=1.602026.0468.09
GPTQBackbone=Mixtral, Weight Bitwidth=32026.0466.28
NoWagBackbone=Mixtral, Weight Bitwidth=2.082026.0465.51
MxMoEBackbone=Qwen1.5-MoE, Weight Bitwidth=2.252026.0461.25
MTP (d=1)Horizon (d)=1, Parameters=1.3B, Training tokens=100B2025.1160.54
GPTParameters=1.3B, Training tokens=100B2025.1160.46
R2LMShots=5-sh, Candidate Type=long-target2026.0660.2
GPTQBackbone=Qwen1.5-MoE, Weight Bitwidth=32026.0459.99
JTP (d=2)Horizon (d)=2, Parameters=1.3B, Training tokens=100B2025.1159.98
MoBiEBackbone=Qwen1.5-MoE, Weight Bitwidth=1.352026.0459.75
R2LM-PI (plug-in)Shots=5-sh, Candidate Type=long-target2026.0659.4
MTP (d=2)Horizon (d)=2, Parameters=1.3B, Training tokens=100B2025.1159.35
NextLat (d=1)Horizon (d)=1, Parameters=1.3B, Training tokens=100B2025.1159.27
NextLat (d=2)Horizon (d)=2, Parameters=1.3B, Training tokens=100B2025.1159.2
JTP (d=1)Horizon (d)=1, Parameters=1.3B, Training tokens=100B2025.1158.64
MoEQuantBackbone=Qwen1.5-MoE, Weight Bitwidth=22026.0458.21
NoPEModel Architecture=GLA, Parameters=1.3B, Training Tokens=26B2025.1158.2
QuantMoe-BenchBackbone=Mixtral, Weight Bitwidth=2.32026.0457.85
Bidirectional dLLMShots=5-sh, Candidate Type=long-target2026.0657.7
RoPEModel Architecture=GLA, Parameters=1.3B, Training Tokens=26B2025.1157.1
Selective RoPEModel Architecture=GLA, Parameters=1.3B, Training Tokens=26B2025.1156
NoWagBackbone=Qwen1.5-MoE, Weight Bitwidth=2.082026.0455.09
AWQBackbone=Mixtral, Weight Bitwidth=32026.0454.98
Causal dLLMShots=5-sh, Candidate Type=long-target2026.0654.3
Selective RoPEModel Architecture=Gated DeltaNet, Parameters=~400M, Training Tokens=10B, Variant=stable_selective_rope2025.1154.1
Selective RoPEModel Architecture=FoX, Parameters=370M, Training Tokens=10B, Output-norm setting=OFF2025.1153
NoPEModel Architecture=FoX, Parameters=370M, Training Tokens=10B, Output-norm setting=OFF2025.1152.8
RoPEModel Architecture=Gated DeltaNet, Parameters=~400M, Training Tokens=10B2025.1152.5
QuantMoe-BenchBackbone=Qwen1.5-MoE, Weight Bitwidth=2.32026.0452.4
NoPEModel Architecture=GLA, Parameters=370M, Training Tokens=10B2025.1152.2
NoPEModel Architecture=Gated DeltaNet, Parameters=~400M, Training Tokens=10B2025.1152.2
MDLM2025.1051.85
LDDM-M2025.1051.46
Selective RoPEModel Architecture=GLA, Parameters=370M, Training Tokens=10B2025.1151.3
AWQBackbone=Qwen1.5-MoE, Weight Bitwidth=22026.0450.91
RoPEModel Architecture=FoX, Parameters=370M, Training Tokens=10B, Output-norm setting=OFF2025.1150.8
RoPEModel Architecture=GLA, Parameters=370M, Training Tokens=10B2025.1150.7
GPTQBackbone=Qwen1.5-MoE, Weight Bitwidth=22026.0450.43
GPTQBackbone=Mixtral, Weight Bitwidth=22026.0450.33
AWQBackbone=Qwen1.5-MoE, Weight Bitwidth=32026.0450.12
AWQBackbone=Mixtral, Weight Bitwidth=22026.0450.08
Code specialistBackbone=Pythia-1B, Checkpoint=step10000, Freeze Layers=4, Seed=42, Samples=5002026.0350
MoEQuantBackbone=Mixtral, Weight Bitwidth=22026.0449.91
Base modelBackbone=Pythia-1B, Checkpoint=step10000, Freeze Layers=4, Seed=42, Samples=5002026.0349.6
MonolithicBackbone=Pythia-1B, Checkpoint=step10000, Freeze Layers=4, Seed=42, Samples=5002026.0349.4
KALAVAI MoEBackbone=Pythia-1B, Checkpoint=step10000, Freeze Layers=4, Seed=42, Samples=5002026.0349.4
Weight averageBackbone=Pythia-1B, Checkpoint=step10000, Freeze Layers=4, Seed=42, Samples=5002026.0348.6
Sci. specialistBackbone=Pythia-1B, Checkpoint=step10000, Freeze Layers=4, Seed=42, Samples=5002026.0348.2
Fict. specialistBackbone=Pythia-1B, Checkpoint=step10000, Freeze Layers=4, Seed=42, Samples=5002026.0348.2