List your company
ResearchFormerly WizWand

Sentiment Analysis on SST-2 (dev)

96.8AccuracyUNIMO
24.270443.100261.9380.7598Sep 12, 2019Jun 13, 2020Mar 16, 2021Dec 16, 2021Sep 18, 2022Jun 20, 2023Mar 22, 2024
Updated 8mo ago

Evaluation results

MethodLinks
UNIMOSize=large2020.1296.8
RoBERTaSize=large2020.1296.4
gMLP_xlargeParams (M)=9412021.0595.6
XLNetSize=large2020.1295.6
ALBERT xxlargeParameters=235M, Speedup=0.3x2019.0995.2
UNIMOSize=base2020.1295.1
gMLP_largeParams (M)=3652021.0594.8
aMLP_largeAttn Size=128, Params (M)=3162021.0594.8
RoBERTaSize=base2020.1294.8
UniLMSize=large2020.1294.5
BERT_large (ours)Attn Size=1024 (64 x 16), Params (M)=3362021.0594.3
gMLP_baseParams (M)=1302021.0594.2
LLM2LLM% Data=0.06, # Seed Examples=40, # Augmented=142024.0394.04
BERT_base (ours)Attn Size=768 (64 x 12), Params (M)=1102021.0593.8
BERT_large [2]Attn Size=1024 (64 x 16), Params (M)=3362021.0593.7
aMLP_baseAttn Size=64, Params (M)=1092021.0593.4
BERTSize=large2020.1293.2
BERT largeParameters=334M, Speedup=1.02019.0993
Baselinew-bits=32, e-bits=32, Size (MB)=415.4, Size-w/o-e (MB)=324.52019.0993
LLM2LLM% Data=0.04, # Seed Examples=30, # Augmented=462024.0393
Q-BERTw-bits=8, e-bits=8, Size (MB)=103.9, Size-w/o-e (MB)=81.22019.0992.88
BERT baseParameters=108M, Speedup=4.7x2019.0992.8
BERT_base [2]Attn Size=768 (64 x 12), Params (M)=1102021.0592.7
BERTSize=base2020.1292.7
Q-BERTw-bits=4, e-bits=8, Size (MB)=63.4, Size-w/o-e (MB)=40.62019.0992.66
LLM2LLM% Data=0.02, # Seed Examples=20, # Augmented=442024.0392.66
Q-BERTMPw-bits=2/4 MP, e-bits=8, Size (MB)=53.2, Size-w/o-e (MB)=30.52019.0992.55
Q-BERTw-bits=3, e-bits=8, Size (MB)=53.2, Size-w/o-e (MB)=30.52019.0992.54
ALBERT xlargeParameters=60M, Speedup=0.6x2019.0992.4
Q-BERTMPw-bits=2/3 MP, e-bits=8, Size (MB)=48.1, Size-w/o-e (MB)=25.42019.0992.08
ALBERT largeParameters=18M, Speedup=1.7x2019.0991.7
ALBERT baseParameters=12M, Speedup=5.6x2019.0990.3
DirectQw-bits=4, e-bits=8, Size (MB)=63.4, Size-w/o-e (MB)=40.62019.0985.67
Q-BERTw-bits=2, e-bits=8, Size (MB)=43.1, Size-w/o-e (MB)=20.42019.0984.63
DirectQw-bits=3, e-bits=8, Size (MB)=53.2, Size-w/o-e (MB)=30.52019.0982.86
Baseline% Data=0.06, # Seed Examples=40, # Augmented=142024.0382.8
UNIMO (w/o single-modal)Size=base, Note=Removing single-modal learning process2020.1282
DirectQw-bits=2, e-bits=8, Size (MB)=43.1, Size-w/o-e (MB)=20.42019.0980.62
Baseline% Data=0.04, # Seed Examples=30, # Augmented=462024.0362.04
Baseline% Data=0.02, # Seed Examples=20, # Augmented=442024.0352.87
Baseline% Data=0, # Seed Examples=0, # Augmented=02024.0327.06