List your company
ResearchFormerly WizWand

Pronoun Resolution on XWinograd

57.7AccuracyTransformer
51.272852.941454.6156.2786Jun 14, 2025Aug 7, 2025Oct 1, 2025Nov 25, 2025Jan 19, 2026Mar 15, 2026May 9, 2026
Updated 5mo ago

Evaluation results

MethodLinks
TransformerTraining=compute-matched, Training dataset=FineWeb-edu2026.0557.7
DMoEModel Scale=1.7B, Training Strategy=Dynamic Mixture-of-Experts, Evaluation Protocol=Zero-shot2025.0656.4
DMoEModel Scale=1.7B, Training Strategy=Dynamic Mixture-of-Experts, Evaluation Protocol=Few-shot (4-shot)2025.0656.4
BLOOM + Continued Pre-trainingModel Scale=1.7B, Training Strategy=Continued Pre-training, Evaluation Protocol=Few-shot (4-shot)2025.0655.8
Branch-Train-MixModel Scale=1.7B, Training Strategy=Branch-Train-Mix, Evaluation Protocol=Few-shot (4-shot)2025.0655.8
Branch-Train-MixModel Scale=1.7B, Training Strategy=Branch-Train-Mix, Evaluation Protocol=Zero-shot2025.0655.6
BLOOM + Continued Pre-trainingModel Scale=1.7B, Training Strategy=Continued Pre-training, Evaluation Protocol=Zero-shot2025.0655.5
BLOOMModel Scale=1.7B, Training Strategy=Base, Evaluation Protocol=Few-shot (4-shot)2025.0655.5
DMoEModel Scale=560M, Training Strategy=Dynamic Mixture-of-Experts, Evaluation Protocol=Zero-shot2025.0655.1
DMoEModel Scale=560M, Training Strategy=Dynamic Mixture-of-Experts, Evaluation Protocol=Few-shot (4-shot)2025.0655.1
BLOOMModel Scale=1.7B, Training Strategy=Base, Evaluation Protocol=Zero-shot2025.0655.1
BLOOM + Continued Pre-trainingModel Scale=560M, Training Strategy=Continued Pre-training, Evaluation Protocol=Zero-shot2025.0654.9
Branch-Train-MixModel Scale=560M, Training Strategy=Branch-Train-Mix, Evaluation Protocol=Zero-shot2025.0654.4
Branch-Train-MixModel Scale=560M, Training Strategy=Branch-Train-Mix, Evaluation Protocol=Few-shot (4-shot)2025.0654.2
BLOOM + Continued Pre-trainingModel Scale=560M, Training Strategy=Continued Pre-training, Evaluation Protocol=Few-shot (4-shot)2025.0653.8
BLOOMModel Scale=560M, Training Strategy=Base, Evaluation Protocol=Zero-shot2025.0653.7
SRMTraining=compute-matched, Training dataset=FineWeb-edu2026.0553.41
BLOOMModel Scale=560M, Training Strategy=Base, Evaluation Protocol=Few-shot (4-shot)2025.0653.3
MambaTraining=compute-matched, Training dataset=FineWeb-edu2026.0551.52