Question Answering on HotPot D2 (test)
96.1Macro F1SURGELLM-S-DistilBERT
Evaluation results
| Method | Links | |
|---|---|---|
| SURGELLM-S-DistilBERTFamily=SURGELLM-S, Parameters=66M, Training Time (s)=1192026.06 | 96.1 | |
| SURGELLM-FULL-ALBERTFamily=SURGELLM-FULL, Parameters=11M, Training Time (s)=3172026.06 | 96.1 | |
| SURGELLM-IWN-RoBERTaFamily=IWN, Parameters=125M, Training Time (s)=3322026.06 | 95.4 | |
| SURGELLM-FULL-RoBERTaFamily=SURGELLM-FULL, Parameters=125M, Training Time (s)=326, Early Stopping=true2026.06 | 95 | |
| SURGELLM-G-RoBERTaFamily=SURGELLM-G, Parameters=125M, Training Time (s)=327, Early Stopping=true2026.06 | 94.9 | |
| Baseline-RoBERTaFamily=Baseline, Parameters=125M, Training Time (s)=2332026.06 | 94.7 | |
| SURGELLM-IWN-BERTFamily=IWN, Parameters=110M, Training Time (s)=3222026.06 | 94.6 | |
| Baseline-DistilBERTFamily=Baseline, Parameters=66M, Training Time (s)=822026.06 | 94 | |
| T5-baseFamily=T5-T2T, Parameters=220M, Training Time (s)=4122026.06 | 93.9 | |
| SURGELLM-S-BERTFamily=SURGELLM-S, Parameters=110M, Training Time (s)=3172026.06 | 93.9 | |
| Baseline-BERTFamily=Baseline, Parameters=110M, Training Time (s)=2272026.06 | 93.4 |