Language Modeling on Broad evaluation suite unseen S1 (dev)
74.2Average Accuracyall-FA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| all-FAFA=48, SWA=0, KDA=0, GDN=02026.04 | 74.2 | 100 | 1 | |
| Idealized|All–18FA=13, SWA=32, KDA=1, GDN=22026.04 | 71.8 | 97 | 2 | |
| Reg|Lklhd–26FA=12, SWA=26, KDA=6, GDN=42026.04 | 71.1 | 96 | 2.9 | |
| Reg|Lklhd–18FA=3, SWA=25, KDA=4, GDN=162026.04 | 69.7 | 94 | 4.8 | |
| Idealized|Lklhd–6FA=0, SWA=30, KDA=5, GDN=132026.04 | 66.8 | 90 | 6.2 | |
| Idealized|All–6FA=0, SWA=30, KDA=5, GDN=132026.04 | 65.3 | 88 | 6.1 | |
| Reg|Lklhd–13FA=0, SWA=16, KDA=13, GDN=192026.04 | 60.2 | 81 | 6.9 | |
| Reg|Lklhd–10FA=0, SWA=10, KDA=5, GDN=332026.04 | 57.2 | 77 | 10.7 |