GPT-2
Benchmarks
TaskDatasetSOTA resultTrendResults
GPT-2
0.44Delta PPL
32
GPT-2 Phase 2 (test)
89.18Accuracy
28
GPT-2 Medium (train)
29.47Training Time (h)
27
GPT-2 Pre-training (val)
2.493Validation Loss
21
GPT-2 Evaluation Set
29.91Hyper-Prior BPT
20
GPT-2 Small (124M)
0.68Validation CE Loss
12
GPT-2 Pretraining Data (train)
2.917Training Loss
12
GPT-2-Medium standardized evaluator
109.1Delta (%)
10
GPT-2
0Avg KLD (bits/token)
10
GPT-2 Transformer Block
0.2Latency (s)
10
GPT-2 124M held-out (test)
17.33Perplexity
10
GPT-2 Medium (val)
87.89Runtime
9
GPT-2 Small 128K SAE
32.16Input Attribution (%)
9
GPT-2 Small
39.32Input Success Rate
9
GPT-2 (full)
91.1Acc
9
GPT-2 (train)
0.48Pearson r
8
GPT-2
423.925VecGen
6
GPT-2 Small suite Aggregate (test)
0.71Gini Coefficient
6
GPT-2 Small 5 (test)
0.72Gini Coefficient
6
GPT-2 Small (test 4)
0.74Gini Coefficient
6
GPT-2 Small 3 (test)
0.73Gini Coefficient
6
GPT-2 Small (test)
0.71Gini Coefficient
6
GPT-2 Small (test 1)
0.33Gini Coefficient
6
GPT-2 Large
25.876Perplexity
5
GPT-2 (test)
85.75Accuracy
5