Instruction Following on Super-Natural Instructions v2 (test)
62RougeLTk-Instruct
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Tk-InstructModel Size=XXL (11B), Evaluation Protocol=Few-shot2024.06 | 62 | 1 | |
| TAGIModel Size=XXL (11B), Evaluation Protocol=Few-shot2024.06 | 58.4 | 0.39 | |
| HINTModel Size=XXL (11B), Evaluation Protocol=Few-shot2024.06 | 56.4 | 0.37 | |
| TAGIModel Size=XL (3B), Evaluation Protocol=Few-shot2024.06 | 56.3 | 0.39 | |
| Tk-InstructModel Size=XL (3B), Evaluation Protocol=Few-shot2024.06 | 54 | 1 | |
| Tk-InstructModel Size=XXL (11B), Evaluation Protocol=Zero-shot2024.06 | 53.6 | 1 | |
| HINTModel Size=XL (3B), Evaluation Protocol=Few-shot2024.06 | 53.2 | 0.37 | |
| TAGIModel Size=XXL (11B), Evaluation Protocol=Zero-shot2024.06 | 52.3 | 0.39 | |
| OPT (13B)Model Architecture=Decoder-only, Evaluation Protocol=Few-shot2024.06 | 51.5 | 0.36 | |
| HINTModel Size=XXL (11B), Evaluation Protocol=Zero-shot2024.06 | 51.1 | 0.37 | |
| HyperTuningModel Size=XL (3B), Evaluation Protocol=Few-shot2024.06 | 48.6 | 0.34 | |
| TAGIModel Size=XL (3B), Evaluation Protocol=Zero-shot2024.06 | 48.4 | 0.39 | |
| Tk-InstructModel Size=XL (3B), Evaluation Protocol=Zero-shot2024.06 | 48 | 1 | |
| HINTModel Size=XL (3B), Evaluation Protocol=Zero-shot2024.06 | 47.2 | 0.37 | |
| GPT-2 XL (1.5B)Model Architecture=Decoder-only, Evaluation Protocol=Few-shot2024.06 | 45.3 | 0.33 | |
| OPT (13B)Model Architecture=Decoder-only, Evaluation Protocol=Zero-shot2024.06 | 44.8 | 0.36 | |
| TAGIModel Size=Base (250M), Evaluation Protocol=Few-shot2024.06 | 42.5 | 0.39 | |
| Tk-InstructModel Size=Base (250M), Evaluation Protocol=Few-shot2024.06 | 42.1 | 1 | |
| HINTModel Size=Base (250M), Evaluation Protocol=Few-shot2024.06 | 41.8 | 0.37 | |
| HyperTuningModel Size=XL (3B), Evaluation Protocol=Zero-shot2024.06 | 38.9 | 0.34 | |
| GPT-2 XL (1.5B)Model Architecture=Decoder-only, Evaluation Protocol=Zero-shot2024.06 | 38.2 | 0.33 | |
| Tk-InstructModel Size=Base (250M), Evaluation Protocol=Zero-shot2024.06 | 35.3 | 1 | |
| TAGIModel Size=Base (250M), Evaluation Protocol=Zero-shot2024.06 | 35.3 | 0.39 | |
| HINTModel Size=Base (250M), Evaluation Protocol=Zero-shot2024.06 | 33.3 | 0.37 | |
| No FTModel Size=XXL (11B), Evaluation Protocol=Few-shot2024.06 | 30.5 | 1 | |
| No FTModel Size=XXL (11B), Evaluation Protocol=Zero-shot2024.06 | 26.2 | 1 | |
| HypterModel Size=XL (3B), Evaluation Protocol=Zero-shot2024.06 | 16.8 | 0.35 | |
| HypterModel Size=XXL (11B), Evaluation Protocol=Zero-shot2024.06 | 15.5 | 0.35 | |
| No FTModel Size=XL (3B), Evaluation Protocol=Zero-shot2024.06 | 14.3 | 1 | |
| HypterModel Size=XL (3B), Evaluation Protocol=Few-shot2024.06 | 14.2 | 0.35 | |
| No FTModel Size=XL (3B), Evaluation Protocol=Few-shot2024.06 | 13.6 | 1 | |
| HypterModel Size=XXL (11B), Evaluation Protocol=Few-shot2024.06 | 13.4 | 0.35 | |
| HypterModel Size=Base (250M), Evaluation Protocol=Zero-shot2024.06 | 12.1 | 0.35 | |
| HypterModel Size=Base (250M), Evaluation Protocol=Few-shot2024.06 | 10.6 | 0.35 | |
| No FTModel Size=Base (250M), Evaluation Protocol=Few-shot2024.06 | 9.4 | 1 | |
| No FTModel Size=Base (250M), Evaluation Protocol=Zero-shot2024.06 | 8.8 | 1 |