List your company

Natural Language Understanding on GLUE 1.0 (dev)

95.3SST-2 (Acc)RoBERTa
89.16490.75792.3593.943Oct 14, 2019Mar 9, 2020Aug 3, 2020Dec 28, 2020May 24, 2021Oct 18, 2021Mar 14, 2022

Evaluation results

MethodLinks
RoBERTaScale=large2022.0395.3——————89
XLNetScale=large2022.0394.465.290————88.4
BERTScale=large, variant=wwm, reproduced=true2022.0393.463.187.2————87.3
PERTScale=large2022.0393.465.787.3————87.6
BERTScale=large2022.0393.260.688————86.6
BERTScale=base2022.0392.760.686.7————84.4
BERTScale=base, variant=+2022.0392.659.386————84.4
XLNetScale=base2022.0392.6——————85.8
RoBERTaScale=base2022.0392.5——————84.7
BERT baselineArchitecture=BERT-Base, Precision=FP32, Quantization Method=None2019.1092.3658.489090.387.8469.789.62—
Q8BERTArchitecture=BERT-Base, Precision=8bit, Quantization Method=Quantization-Aware Training (QAT)2019.1092.2458.4889.5690.6287.9668.7889.04—
PERTScale=base2022.039261.287.5————84.5
DQ BERTArchitecture=BERT-Base, Precision=8bit, Quantization Method=Dynamic Quantization (DQ)2019.1091.0456.7487.8889.3484.9863.3287.66—
ALBERTScale=large2022.0390.6——————83.8
ALBERTScale=base2022.0389.4——————81.9
BERT baselineArchitecture=BERT-Large, Precision=FP32, Quantization Method=None2019.10——90.8691.66——90.34—
DQ BERTArchitecture=BERT-Large, Precision=8bit, Quantization Method=Dynamic Quantization (DQ)2019.10——88.1888.38——83.04—
Q8BERTArchitecture=BERT-Large, Precision=8bit, Quantization Method=Quantization-Aware Training (QAT)2019.10——90.991.74——90.12—