Natural Language Inference on CB SuperGLUE (test)
91.43AccuracyHyperPELT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HyperPELTNumber of Samples=2502022.03 | 91.43 | — | |
| HyperPELTNumber of Samples=1002022.03 | 90.71 | — | |
| Hyperformer++Number of Samples=2502022.03 | 89.63 | — | |
| HyperPELT TaskEmbedNumber of Samples=2502022.03 | 89.57 | — | |
| Prompt-tuningNumber of Samples=2502022.03 | 88.57 | — | |
| HyperPELT TaskEmbedNumber of Samples=1002022.03 | 87.86 | — | |
| Hyperformer++Number of Samples=1002022.03 | 87.41 | — | |
| Prompt-tuningNumber of Samples=1002022.03 | 87.14 | — | |
| HyperPELT TaskEmbedNumber of Samples=322022.03 | 87.14 | — | |
| HyperPELT TaskEmbedNumber of Samples=162022.03 | 86.43 | — | |
| T5-BaseNumber of Samples=1002022.03 | 85.93 | — | |
| Prompt-tuningNumber of Samples=322022.03 | 85.71 | — | |
| HyperPELT TaskEmbedNumber of Samples=42022.03 | 85.71 | — | |
| T5-BaseNumber of Samples=2502022.03 | 85.19 | — | |
| Prompt-tuningNumber of Samples=162022.03 | 84.29 | — | |
| FedAvgModel=OPT-1.3B2024.05 | 83.93 | — | |
| HyperPELTNumber of Samples=322022.03 | 83.57 | — | |
| HyperPELTNumber of Samples=162022.03 | 82.14 | — | |
| FedAvgModel=OPT-125M2024.05 | 82.14 | — | |
| Hyperformer++Number of Samples=322022.03 | 81.48 | — | |
| Prompt-tuningNumber of Samples=42022.03 | 81.43 | — | |
| T5-BaseNumber of Samples=322022.03 | 80 | — | |
| HyperPELTNumber of Samples=42022.03 | 77.86 | — | |
| T5-BaseNumber of Samples=162022.03 | 77.04 | — | |
| Hyperformer++Number of Samples=162022.03 | 76.29 | — | |
| DeComFLModel=OPT-1.3B, P=102024.05 | 75.71 | — | |
| DeComFLModel=OPT-125M, P=102024.05 | 75 | — | |
| FedZOModel=OPT-1.3B, P=102024.05 | 74.49 | — | |
| FedZOModel=OPT-125M, P=102024.05 | 74.41 | — | |
| MeZOModel=OPT-1.3B, Setting=single agent2024.05 | 74.01 | — | |
| MeZOModel=OPT-125M, Setting=single agent2024.05 | 72.49 | — | |
| Hyperformer++Number of Samples=42022.03 | 60.74 | — | |
| T5-BaseNumber of Samples=42022.03 | 57.78 | — |