Natural Language Understanding on SuperGLUE (WSC, WIC, RTE, CB, MultiRC, BoolQ, COPA, AVG)
76.9WSC ScorePoly
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| PolyModel=FLAN-T5-Large2026.05 | 76.9 | 60.6 | 86.2 | 87.5 | 0.825 | 85.1 | 90 | 82 | |
| MHRModel=FLAN-T5-Large2026.05 | 75.9 | 61.1 | 86.2 | 87.5 | 0.829 | 85 | 90 | 82.3 | |
| C-PolyModel=FLAN-T5-Large2026.05 | 75 | 67 | 88.8 | 85.7 | 0.833 | 85.6 | 90 | 83.2 | |
| PEMLModel=FLAN-T5-Large2026.05 | 75 | 80 | 90.1 | 85.6 | 0.904 | 86.4 | 82.5 | 84.3 | |
| MOE-LoRAModel=FLAN-T5-Large2026.05 | 66.3 | 57.9 | 86.4 | 87.5 | 0.834 | 85.1 | 91 | 82.3 | |
| LoRAModel=FLAN-T5-Large2026.05 | 64.4 | 59.5 | 85.9 | 85.7 | 0.827 | 81.8 | 90 | 81.8 | |
| PythiaPS (billion)=0.4, T (billion)=3002023.11 | 57.69 | 50.31 | 52.71 | 35.71 | 1.68 | 60.4 | 70 | 46.93 | |
| PEMLBackbone=GLM-10B, Setting=multitasking framework2026.05 | 55.2 | 49.1 | 69.3 | 61 | 0.688 | 68.2 | 70.7 | 63.1 | |
| C-PolyBackbone=GLM-10B, Setting=multitasking framework2026.05 | 53.4 | 48.7 | 68 | 60.3 | 0.679 | 67.3 | 70.4 | 62.2 | |
| PolyBackbone=GLM-10B, Setting=multitasking framework2026.05 | 47 | 41.7 | 62.1 | 52.1 | 0.656 | 64.6 | 65.5 | 56.9 | |
| MHRBackbone=GLM-10B, Setting=multitasking framework2026.05 | 45.5 | 42.3 | 62.7 | 50.7 | 0.657 | 64.8 | 66.3 | 56.9 | |
| BLOOMPS (billion)=0.56, T (billion)=3502023.11 | 40.38 | 50 | 52.71 | 41.07 | 1.05 | 55.14 | 61 | 43.05 | |
| HGRNPS (billion)=1, T (billion)=1002023.11 | 40.38 | 50.78 | 53.43 | 42.86 | 3.04 | 58.69 | 70 | 45.6 | |
| MOE-LoRABackbone=GLM-10B, Setting=multitasking framework2026.05 | 39.6 | 40.3 | 61.2 | 45 | 0.64 | 63.3 | 63.4 | 53.8 | |
| HGRNPS (billion)=0.15, T (billion)=1002023.11 | 38.46 | 51.1 | 56.68 | 42.86 | 1.47 | 59.91 | 65 | 45.07 | |
| HGRNPS (billion)=0.35, T (billion)=1002023.11 | 38.46 | 50.16 | 52.71 | 51.79 | 1.99 | 59.05 | 73 | 46.74 | |
| OPTPS (billion)=1.3, T (billion)=3002023.11 | 37.5 | 51.1 | 51.99 | 41.07 | 3.15 | 57.77 | 79 | 45.94 | |
| GPT-NeoPS (billion)=0.13, T (billion)=3002023.11 | 36.54 | 50 | 54.87 | 41.07 | 0.84 | 61.71 | 64 | 44.15 | |
| OPTPS (billion)=0.16, T (billion)=3002023.11 | 36.54 | 50 | 49.82 | 21.43 | 1.36 | 55.47 | 66 | 40.09 | |
| PythiaPS (billion)=0.16, T (billion)=3002023.11 | 36.54 | 50.16 | 52.71 | 41.07 | 2.52 | 55.08 | 65 | 43.3 | |
| OPTPS (billion)=0.35, T (billion)=3002023.11 | 36.54 | 50 | 51.99 | 46.43 | 1.36 | 57.74 | 72 | 45.15 | |
| GPT-NeoPS (billion)=1.3, T (billion)=3002023.11 | 36.54 | 50 | 60.29 | 44.64 | 1.99 | 61.99 | 69 | 46.35 | |
| PythiaPS (billion)=1.4, T (billion)=3002023.11 | 36.54 | 50 | 53.07 | 35.71 | 0.94 | 60.73 | 72 | 44.14 | |
| BLOOMPS (billion)=1.1, T (billion)=3502023.11 | 36.54 | 50 | 52.71 | 41.07 | 0.73 | 59.08 | 68 | 44.02 | |
| LoRABackbone=GLM-10B, Setting=multitasking framework2026.05 | 32.3 | 39.1 | 57.3 | 46.3 | 0.624 | 60.9 | 65.7 | 52 |