Sentiment Analysis on SST-2 (dev)
96.8AccuracyUNIMO
Evaluation Results
| Method | Links | |
|---|---|---|
| UNIMOSize=large2020.12 | 96.8 | |
| RoBERTaSize=large2020.12 | 96.4 | |
| gMLP_xlargeParams (M)=9412021.05 | 95.6 | |
| XLNetSize=large2020.12 | 95.6 | |
| ALBERT xxlargeParameters=235M, Speedup=0.3x2019.09 | 95.2 | |
| UNIMOSize=base2020.12 | 95.1 | |
| gMLP_largeParams (M)=3652021.05 | 94.8 | |
| aMLP_largeAttn Size=128, Params (M)=3162021.05 | 94.8 | |
| RoBERTaSize=base2020.12 | 94.8 | |
| UniLMSize=large2020.12 | 94.5 | |
| BERT_large (ours)Attn Size=1024 (64 x 16), Params (M)=3362021.05 | 94.3 | |
| gMLP_baseParams (M)=1302021.05 | 94.2 | |
| LLM2LLM% Data=0.06, # Seed Examples=40, # Augmented=142024.03 | 94.04 | |
| BERT_base (ours)Attn Size=768 (64 x 12), Params (M)=1102021.05 | 93.8 | |
| BERT_large [2]Attn Size=1024 (64 x 16), Params (M)=3362021.05 | 93.7 | |
| aMLP_baseAttn Size=64, Params (M)=1092021.05 | 93.4 | |
| BERTSize=large2020.12 | 93.2 | |
| BERT largeParameters=334M, Speedup=1.02019.09 | 93 | |
| Baselinew-bits=32, e-bits=32, Size (MB)=415.4, Size-w/o-e (MB)=324.52019.09 | 93 | |
| LLM2LLM% Data=0.04, # Seed Examples=30, # Augmented=462024.03 | 93 | |
| Q-BERTw-bits=8, e-bits=8, Size (MB)=103.9, Size-w/o-e (MB)=81.22019.09 | 92.88 | |
| BERT baseParameters=108M, Speedup=4.7x2019.09 | 92.8 | |
| BERT_base [2]Attn Size=768 (64 x 12), Params (M)=1102021.05 | 92.7 | |
| BERTSize=base2020.12 | 92.7 | |
| Q-BERTw-bits=4, e-bits=8, Size (MB)=63.4, Size-w/o-e (MB)=40.62019.09 | 92.66 | |
| LLM2LLM% Data=0.02, # Seed Examples=20, # Augmented=442024.03 | 92.66 | |
| Q-BERTMPw-bits=2/4 MP, e-bits=8, Size (MB)=53.2, Size-w/o-e (MB)=30.52019.09 | 92.55 | |
| Q-BERTw-bits=3, e-bits=8, Size (MB)=53.2, Size-w/o-e (MB)=30.52019.09 | 92.54 | |
| ALBERT xlargeParameters=60M, Speedup=0.6x2019.09 | 92.4 | |
| Q-BERTMPw-bits=2/3 MP, e-bits=8, Size (MB)=48.1, Size-w/o-e (MB)=25.42019.09 | 92.08 | |
| ALBERT largeParameters=18M, Speedup=1.7x2019.09 | 91.7 | |
| ALBERT baseParameters=12M, Speedup=5.6x2019.09 | 90.3 | |
| DirectQw-bits=4, e-bits=8, Size (MB)=63.4, Size-w/o-e (MB)=40.62019.09 | 85.67 | |
| Q-BERTw-bits=2, e-bits=8, Size (MB)=43.1, Size-w/o-e (MB)=20.42019.09 | 84.63 | |
| DirectQw-bits=3, e-bits=8, Size (MB)=53.2, Size-w/o-e (MB)=30.52019.09 | 82.86 | |
| Baseline% Data=0.06, # Seed Examples=40, # Augmented=142024.03 | 82.8 | |
| UNIMO (w/o single-modal)Size=base, Note=Removing single-modal learning process2020.12 | 82 | |
| DirectQw-bits=2, e-bits=8, Size (MB)=43.1, Size-w/o-e (MB)=20.42019.09 | 80.62 | |
| Baseline% Data=0.04, # Seed Examples=30, # Augmented=462024.03 | 62.04 | |
| Baseline% Data=0.02, # Seed Examples=20, # Augmented=442024.03 | 52.87 | |
| Baseline% Data=0, # Seed Examples=0, # Augmented=02024.03 | 27.06 |