Classification on ADReSS (test)
83AccuracyGPT-2 small (paired-perplexity)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GPT-2 small (paired-perplexity)# of parameters=124M, # of masked attention heads=12, % of masked attention heads=92024.06 | 83 | 0.86 | |
| GPT-2 medium (paired-perplexity)# of parameters=355M, # of masked attention heads=92, % of masked attention heads=242024.06 | 83 | 0.85 | |
| GPT-2 large (paired-perplexity)# of parameters=774M, # of masked attention heads=388, % of masked attention heads=542024.06 | 81 | 0.8 | |
| GPT-2 XL (paired-perplexity)# of parameters=1.5B, # of masked attention heads=1080, % of masked attention heads=902024.06 | 81 | 0.82 |