Natural Language Processing on T0 MTest11 P3 (test)
61.4AccuracyFull FT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Full FTBackbone=T0, Model Scale=XL (3B)2024.06 | 61.4 | 1 | |
| HINT*Backbone=T0, Model Scale=XL (3B)2024.06 | 60.3 | — | |
| FiD-ICLBackbone=T5-LM, Model Scale=XL (3B), Number of Shots=162024.06 | 60 | 1.9 | |
| TAGIBackbone=T5-LM, Model Scale=XL (3B)2024.06 | 58.9 | 0.88 | |
| TAGIBackbone=T0, Model Scale=XL (3B)2024.06 | 58.8 | 0.88 | |
| FiD-ICLBackbone=T0, Model Scale=XL (3B), Number of Shots=162024.06 | 58.2 | 1.9 | |
| Ensemble-ICLBackbone=T0, Model Scale=XL (3B), Number of Shots=162024.06 | 57.7 | 13.2 | |
| Zero-shotBackbone=T0, Model Scale=XL (3B)2024.06 | 57.6 | 1 | |
| MetatrainBackbone=T0, Model Scale=XL (3B)2024.06 | 56.8 | 1 | |
| Full FTBackbone=T0, Model Scale=Large (800M)2024.06 | 56.6 | 1 | |
| Hyper*Backbone=T0, Model Scale=XL (3B)2024.06 | 56.2 | — | |
| FiD-ICLBackbone=T5-LM, Model Scale=Large (800M), Number of Shots=162024.06 | 55.2 | 1.9 | |
| TAGIBackbone=T5-LM, Model Scale=Large (800M)2024.06 | 54.7 | 0.88 | |
| Ensemble-ICLBackbone=T5-LM, Model Scale=Large (800M), Number of Shots=162024.06 | 54.5 | 13.2 | |
| TAGIBackbone=T0, Model Scale=Large (800M)2024.06 | 53.8 | 0.88 | |
| Ensemble-ICLBackbone=T0, Model Scale=Large (800M), Number of Shots=162024.06 | 53.7 | 13.2 | |
| FiD-ICLBackbone=T0, Model Scale=Large (800M), Number of Shots=162024.06 | 53.4 | 1.9 | |
| Concat-ICLBackbone=T0, Model Scale=Large (800M), Number of Shots=162024.06 | 53.2 | 4.1 | |
| MetatrainBackbone=T5-LM, Model Scale=XL (3B)2024.06 | 53.1 | 1 | |
| Ensemble-ICLBackbone=T5-LM, Model Scale=XL (3B), Number of Shots=162024.06 | 52.6 | 13.2 | |
| Zero-shotBackbone=T0, Model Scale=Large (800M)2024.06 | 52.4 | 1 | |
| MetatrainBackbone=T5-LM, Model Scale=Large (800M)2024.06 | 52.4 | 1 | |
| MetatrainBackbone=T0, Model Scale=Large (800M)2024.06 | 52.4 | 1 | |
| Full FTBackbone=T0, Model Scale=Base (250M)2024.06 | 51.9 | 1 | |
| FiD-ICLBackbone=T0, Model Scale=Base (250M), Number of Shots=162024.06 | 51 | 1.9 | |
| TAGIBackbone=T0, Model Scale=Base (250M)2024.06 | 50.8 | 0.88 | |
| MetatrainBackbone=T0, Model Scale=Base (250M)2024.06 | 50.1 | 1 | |
| Ensemble-ICLBackbone=T0, Model Scale=Base (250M), Number of Shots=162024.06 | 49.9 | 13.2 | |
| Zero-shotBackbone=T0, Model Scale=Base (250M)2024.06 | 49.1 | 1 | |
| Concat-ICLBackbone=T0, Model Scale=Base (250M), Number of Shots=162024.06 | 48.6 | 4.1 | |
| Concat-ICLBackbone=T5-LM, Model Scale=Large (800M), Number of Shots=162024.06 | 47.6 | 4.1 | |
| Full FTBackbone=T5-LM, Model Scale=XL (3B)2024.06 | 47.2 | 1 | |
| FiD-ICLBackbone=T5-LM, Model Scale=Base (250M), Number of Shots=162024.06 | 47 | 1.9 | |
| TAGIBackbone=T5-LM, Model Scale=Base (250M)2024.06 | 45.6 | 0.88 | |
| Full FTBackbone=T5-LM, Model Scale=Large (800M)2024.06 | 45.5 | 1 | |
| Full FTBackbone=T5-LM, Model Scale=Base (250M)2024.06 | 44.6 | 1 | |
| Ensemble-ICLBackbone=T5-LM, Model Scale=Base (250M), Number of Shots=162024.06 | 44.6 | 13.2 | |
| Concat-ICLBackbone=T5-LM, Model Scale=Base (250M), Number of Shots=162024.06 | 44.2 | 4.1 | |
| MetatrainBackbone=T5-LM, Model Scale=Base (250M)2024.06 | 44.1 | 1 | |
| Zero-shotBackbone=T5-LM, Model Scale=Base (250M)2024.06 | 43.9 | 1 | |
| Zero-shotBackbone=T5-LM, Model Scale=XL (3B)2024.06 | 42.6 | 1 | |
| Zero-shotBackbone=T5-LM, Model Scale=Large (800M)2024.06 | 41.5 | 1 |