Multi-task Performance Evaluation on Multi-task Suite D1-D4 (test)
94Avg F1SURGELLM-IWN-RoBERTa
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SURGELLM-IWN-RoBERTaFamily=IWN, Parameters=125M, Training Time (s)=3322026.06 | 94 | 0.036 | |
| SURGELLM-IWN-BERTFamily=IWN, Parameters=110M, Training Time (s)=3222026.06 | 92.7 | 0.023 | |
| SURGELLM-G-RoBERTaFamily=SURGELLM-G, Parameters=125M, Training Time (s)=327, Early Stopping=true2026.06 | 90.6 | 0.002 | |
| Baseline-RoBERTaFamily=Baseline, Parameters=125M, Training Time (s)=2332026.06 | 90.4 | — | |
| T5-baseFamily=T5-T2T, Parameters=220M, Training Time (s)=4122026.06 | 89.7 | -0.007 | |
| Baseline-BERTFamily=Baseline, Parameters=110M, Training Time (s)=2272026.06 | 89.4 | -0.01 | |
| SURGELLM-S-BERTFamily=SURGELLM-S, Parameters=110M, Training Time (s)=3172026.06 | 89.4 | -0.01 | |
| SURGELLM-FULL-RoBERTaFamily=SURGELLM-FULL, Parameters=125M, Training Time (s)=326, Early Stopping=true2026.06 | 88.9 | -0.015 | |
| Baseline-DistilBERTFamily=Baseline, Parameters=66M, Training Time (s)=822026.06 | 88.6 | -0.018 | |
| SURGELLM-FULL-ALBERTFamily=SURGELLM-FULL, Parameters=11M, Training Time (s)=3172026.06 | 88.6 | -0.018 | |
| SURGELLM-S-DistilBERTFamily=SURGELLM-S, Parameters=66M, Training Time (s)=1192026.06 | 87 | -0.034 |