Natural Language Processing on T0 Without SCloze dataset HyperT5 variant (test)
60.6AccuracyFiD-ICL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FiD-ICLBackbone=T5-LM, Model Scale=XL (3B), Number of Shots=162024.06 | 60.6 | 1.9 | |
| TAGIBackbone=T5-LM, Model Scale=XL (3B)2024.06 | 59.8 | 0.88 | |
| HyperTuningBackbone=T5-LM, Model Scale=XL (3B)2024.06 | 59.6 | — | |
| TAGIBackbone=T0, Model Scale=XL (3B)2024.06 | 59.2 | 0.88 | |
| FiD-ICLBackbone=T0, Model Scale=XL (3B), Number of Shots=162024.06 | 58.5 | 1.9 | |
| TAGIBackbone=T5-LM, Model Scale=Large (800M)2024.06 | 56 | 0.88 | |
| FiD-ICLBackbone=T5-LM, Model Scale=Large (800M), Number of Shots=162024.06 | 55.8 | 1.9 | |
| HyperTuningBackbone=T5-LM, Model Scale=Large (800M)2024.06 | 54.6 | — | |
| TAGIBackbone=T0, Model Scale=Large (800M)2024.06 | 54.6 | 0.88 | |
| FiD-ICLBackbone=T0, Model Scale=Large (800M), Number of Shots=162024.06 | 53.9 | 1.9 | |
| FiD-ICLBackbone=T0, Model Scale=Base (250M), Number of Shots=162024.06 | 51.7 | 1.9 | |
| TAGIBackbone=T0, Model Scale=Base (250M)2024.06 | 51.7 | 0.88 | |
| FiD-ICLBackbone=T5-LM, Model Scale=Base (250M), Number of Shots=162024.06 | 46.9 | 1.9 | |
| TAGIBackbone=T5-LM, Model Scale=Base (250M)2024.06 | 46.7 | 0.88 |