OPT
Benchmarks
TaskDatasetSOTA resultTrendResults
OPT-30B
15.7Latency (ms)
5
OPT-175B first FFN layer
0.225Latency (ms)
5
OPT-125M
46.71Latency (ms)
4
OPT-125 Personal Set
77.4Accuracy
4
OPT-125 Forget Set
65.3Accuracy
4
OPT 512 prompt + 512 generation tokens 6.7B
72.5Throughput (tokens/sec)
4
OPT 512 prompt + 32 generation tokens 6.7B
50.5Throughput (token/s)
4
OPT-125M
35.32Mean PPL
3
OPT 512 prompt + 1024 generation tokens 6.7B
62.3Throughput (token/s)
3
OPT DIPPER 6.7B
0.929ROC AUC
2
OPT-6.7B PEGASUS
0.959ROC-AUC
2
OPT-6.7B No Attack
0.999ROC-AUC
2
OPT-6.7B generated text samples (test)
3.47Fluency
2