List your company

Zero-shot Evaluation on ARC-E, BoolQ, HSwag, LAMBADA, OBQA, PIQA, SocIQA, and WinoGr (test)

60.65ARC-E AccuracyXSA
50.884453.419755.95558.4903Mar 10, 2026

Evaluation results

MethodLinks
XSAModel size=2.7B, Protocol=Zero-shot2026.0360.6564.8667.462.0438.477.841.4562.7559.421.36
XSAModel size=1.3B, Protocol=Zero-shot2026.0358.8462.2962.4158.573676.6142.8459.9857.191.03
Transformer BaselineModel size=2.7B, Protocol=Zero-shot2026.0358.5960.9866.260.183776.6142.9461.9658.06—
Transformer BaselineModel size=1.3B, Protocol=Zero-shot2026.0356.1965.4760.6956.2434.675.941.458.856.16—
XSAModel size=0.7B, Protocol=Zero-shot2026.0352.6961.1956.2954.0732.273.7841.4556.253.480.26
Transformer BaselineModel size=0.7B, Protocol=Zero-shot2026.0351.2661.0755.6852.823574.0540.0255.8853.22—