Common-sense Reasoning on ARC-C, ARC-E, HellaSwag, Lambada, PIQA, and WinoGrande (test)
44.88ARC-C AccuracyRAT+
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| RAT+Pre-training Tokens=200BT, D=2, L*=64, T=<= 3002026.02 | 44.88 | 76.26 | 65.15 | 53.48 | 75.41 | 60.3 | 62.58 | |
| RAT+Pre-training Tokens=200BT, D=4, L*=64, T=<= 3002026.02 | 44.88 | 76.14 | 65.17 | 53.13 | 75.3 | 60.22 | 62.47 | |
| RAT+Pre-training Tokens=100BT, D=4, L*=64, T=<= 3002026.02 | 44.71 | 75.84 | 62.65 | 50.79 | 73.99 | 59.51 | 61.25 | |
| RAT+Pre-training Tokens=200BT, D=1, L*=64, T=<= 3002026.02 | 44.62 | 76.35 | 65.28 | 54.18 | 75.24 | 60.54 | 62.7 | |
| RAT+Pre-training Tokens=100BT, D=1, L*=64, T=<= 3002026.02 | 44.2 | 75.42 | 62.85 | 52.14 | 74.37 | 60.54 | 61.59 | |
| RAT+Pre-training Tokens=200BT, D=16, L*=64, T=<= 3002026.02 | 44.11 | 75.93 | 64.36 | 52.05 | 75.68 | 58.64 | 61.8 | |
| RAT+Pre-training Tokens=200BT, D=64, L*=64, T=<= 3002026.02 | 44.11 | 75.97 | 63.48 | 48.52 | 75.57 | 58.87 | 61.09 | |
| RAT+Pre-training Tokens=100BT, D=2, L*=64, T=<= 3002026.02 | 44.03 | 75.34 | 62.67 | 51.52 | 74.32 | 59.43 | 61.22 | |
| RAT+Pre-training Tokens=100BT, D=16, L*=64, T=<= 3002026.02 | 44.03 | 75.63 | 62.34 | 49.95 | 74.1 | 58.33 | 60.73 | |
| RAT+Pre-training Tokens=200BT, D=8, L*=64, T=<= 3002026.02 | 43.94 | 75.84 | 64.71 | 52.86 | 75.46 | 58.8 | 61.94 | |
| RAT+Pre-training Tokens=200BT, D=32, L*=64, T=<= 3002026.02 | 43.86 | 75.8 | 63.81 | 50.9 | 75.73 | 58.17 | 61.38 | |
| RAT+Pre-training Tokens=100BT, D=64, L*=64, T=<= 3002026.02 | 43.77 | 75.51 | 61.35 | 45.62 | 74.27 | 56.59 | 59.52 | |
| RAT+Pre-training Tokens=200BT, D=128, L*=64, T=<= 3002026.02 | 43.69 | 76.26 | 62.99 | 46.5 | 75.68 | 58.17 | 60.55 | |
| RAT+Pre-training Tokens=100BT, D=8, L*=64, T=<= 3002026.02 | 43.52 | 75.88 | 62.61 | 50.28 | 74.27 | 58.48 | 60.84 | |
| RAT+Pre-training Tokens=100BT, D=32, L*=64, T=<= 3002026.02 | 43.34 | 75.25 | 62.01 | 48.01 | 73.99 | 57.38 | 60 | |
| RAT+Pre-training Tokens=100BT, D=128, L*=64, T=<= 3002026.02 | 43.34 | 75.67 | 61.08 | 43.53 | 74.21 | 57.38 | 59.2 |