Zero-shot Reasoning on Downstream Tasks (LMB, PIQA, HellaSwag, OPQA, ARC)
32.75LAMBADA (LMB) AccuracyFLT
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| FLTSize=Base, Loop=92026.05 | 32.75 | 65.07 | 37.05 | 33.2 | 53.87 | 28.41 | 41.72 | |
| FLTSize=Base, Loop=62026.05 | 31.15 | 64.85 | 37.03 | 35.2 | 52.81 | 27.24 | 41.38 | |
| LTiSize=Base, Loop=32026.05 | 31.12 | 64.68 | 35.79 | 33.2 | 56.06 | 28.41 | 41.54 | |
| FLTSize=Base, Loop=32026.05 | 30.64 | 63.81 | 35.72 | 31.8 | 51.09 | 26.36 | 39.9 | |
| LTiSize=Base, Loop=92026.05 | 30.56 | 64.85 | 36.24 | 32.8 | 53.78 | 27.81 | 41 | |
| LTaiSize=Base, Loop=62026.05 | 30.35 | 62.89 | 36.33 | 34 | 52.81 | 26.62 | 40.05 | |
| LTSize=Base, Loop=32026.05 | 30.07 | 64.25 | 35.43 | 34.6 | 51.64 | 27.13 | 40.52 | |
| LTiSize=Base, Loop=62026.05 | 29.92 | 64.63 | 36.33 | 31.4 | 52.65 | 26.27 | 40.2 | |
| LTaiSize=Base, Loop=32026.05 | 29.53 | 62.94 | 35.81 | 32.4 | 51.97 | 27.64 | 40.04 | |
| LTaiSize=Base, Loop=92026.05 | 28.39 | 64.41 | 35.94 | 32 | 50.96 | 28.07 | 39.96 | |
| LTSize=Base, Loop=62026.05 | 26.37 | 61.31 | 31.36 | 32.8 | 43.22 | 24.31 | 36.56 | |
| LTiSize=Small, Loop=92026.05 | 20.58 | 58.65 | 27.76 | 28 | 38.84 | 24.23 | 33.01 | |
| LTiSize=Small, Loop=62026.05 | 20.39 | 59.24 | 27.71 | 29.39 | 39.94 | 23.46 | 33.35 | |
| LTiSize=Small, Loop=32026.05 | 20.04 | 59.63 | 27.8 | 27.2 | 39.56 | 22.44 | 32.77 | |
| LTaiSize=Small, Loop=32026.05 | 19.75 | 58.86 | 27.5 | 28.2 | 39.6 | 23.54 | 32.9 | |
| LTaiSize=Small, Loop=92026.05 | 19.65 | 59.84 | 27.56 | 28.2 | 39.68 | 24.65 | 33.26 | |
| FLTSize=Small, Loop=62026.05 | 19.25 | 58.97 | 27.91 | 27.8 | 37.45 | 23.03 | 32.4 | |
| FLTSize=Small, Loop=92026.05 | 19.03 | 59.14 | 28.2 | 25.6 | 38.88 | 23.89 | 32.45 | |
| LTSize=Small, Loop=32026.05 | 18.7 | 59.46 | 27.66 | 27 | 38.97 | 23.63 | 32.57 | |
| FLTSize=Small, Loop=32026.05 | 18.39 | 59.14 | 27.48 | 25.8 | 39.56 | 22.44 | 32.13 | |
| LTSize=Small, Loop=62026.05 | 16.33 | 57.56 | 26.79 | 28.2 | 36.65 | 23.29 | 31.47 | |
| LTSize=Small, Loop=92026.05 | 12.32 | 56.9 | 26 | 26.8 | 36.48 | 22.86 | 30.22 |