General Natural Language Processing on 18 Canonical NLP Tasks
65.9Understanding & KnowledgeNEFTUNE
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| NEFTUNEBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Alpagasus Dolly 3k, Number of training examples=2,9962024.05 | 65.9 | 57.79 | 67.28 | 11.64 | 35.43 | 44.36 | 47.07 | |
| ITBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Alpagasus Dolly 3k, Number of training examples=2,9962024.05 | 65.81 | 57.46 | 67.55 | 11.96 | 33.02 | 43.7 | 46.58 | |
| IMBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Alpagasus Dolly 3k, Number of training examples=2,9962024.05 | 65.66 | 57.47 | 73.24 | 14.57 | 37.48 | 45.29 | 48.95 | |
| NEFTUNEBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Less MMLU Chat, Number of training examples=13,5332024.05 | 65.21 | 57.43 | 63.14 | 9.45 | 35.89 | 55.32 | 47.74 | |
| NEFTUNEBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Alpagasus Alpaca 5k, Number of training examples=5,3052024.05 | 65.18 | 56.88 | 66.45 | 10.24 | 29.53 | 45.46 | 45.62 | |
| ITBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Alpagasus Alpaca 5k, Number of training examples=5,3052024.05 | 64.98 | 57.24 | 66.06 | 8.93 | 26.8 | 47.74 | 45.29 | |
| ITBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Less MMLU Chat, Number of training examples=13,5332024.05 | 64.74 | 57.42 | 62.94 | 9.53 | 33.13 | 55.35 | 47.18 | |
| IMBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Alpagasus Dolly 9k, Number of training examples=9,2292024.05 | 64.67 | 55.32 | 74.87 | 12.5 | 36.69 | 43.96 | 48 | |
| IMBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=LIMA, Number of training examples=1,0302024.05 | 64.49 | 58.21 | 75.55 | 17.06 | 38.84 | 43.45 | 49.6 | |
| IMBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Less Tydiqa, Number of training examples=13,5332024.05 | 64.28 | 56.1 | 65.7 | 17.15 | 34.86 | 54.09 | 48.7 | |
| NEFTUNEBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Alpagasus Dolly 9k, Number of training examples=9,2292024.05 | 64.2 | 56.69 | 69.51 | 8.99 | 33.91 | 42.62 | 45.99 | |
| IMBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Less BBH ICL, Number of training examples=13,5332024.05 | 64.14 | 56.72 | 71.12 | 13.56 | 39.03 | 50.34 | 49.15 | |
| ITBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Alpagasus Dolly 9k, Number of training examples=9,2292024.05 | 64.1 | 56.62 | 69.7 | 7.96 | 32.19 | 42.65 | 45.54 | |
| NEFTUNEBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Less Tydiqa, Number of training examples=13,5332024.05 | 64.03 | 55.09 | 64.02 | 13.84 | 36.65 | 51.21 | 47.47 | |
| IMBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Alpagasus Alpaca 5k, Number of training examples=5,3052024.05 | 64.01 | 56.63 | 72.47 | 11.58 | 35.52 | 44.62 | 47.47 | |
| ITBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Less Tydiqa, Number of training examples=13,5332024.05 | 64.01 | 56.81 | 64.77 | 12.06 | 36.54 | 55.09 | 48.21 | |
| IMBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Less MMLU Chat, Number of training examples=13,5332024.05 | 63.95 | 56.34 | 64.76 | 12.52 | 36.94 | 52.55 | 47.84 | |
| ITBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=LIMA, Number of training examples=1,0302024.05 | 63.92 | 58.29 | 71.96 | 16.01 | 39.27 | 43.29 | 48.79 | |
| LLAMA-2-BASEBackbone=LLAMA-2-7B-BASE2024.05 | 63.91 | 63.42 | 75.86 | 13.32 | 38.8 | 42.03 | 49.32 | |
| NEFTUNEBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Less BBH ICL, Number of training examples=13,5332024.05 | 63.88 | 58.83 | 67.97 | 13.54 | 38.63 | 51.33 | 49.03 | |
| ITBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=Less BBH ICL, Number of training examples=13,5332024.05 | 63.83 | 62.04 | 75.92 | 6.9 | 38.93 | 42.07 | 48.28 | |
| NEFTUNEBackbone=LLAMA-2-7B-BASE, Instruction Tuning Dataset=LIMA, Number of training examples=1,0302024.05 | 63.66 | 57.67 | 73.03 | 15.95 | 38.77 | 43.14 | 48.7 | |
| LLAMA-2-CHATBackbone=LLAMA-2-7B-BASE2024.05 | 55.15 | 70.28 | 15.33 | 38.92 | 51.79 | 49.15 | 49.15 |