Multiple Choice Question Answering on HS-ZS, ARC, COPA, CSQA, PIQA, OBQA, HS, LSAT-AR, BoolQ, LangID
37.82Average AccuracyTN-gram
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| TN-gramNumber of Layers=182026.06 | 37.82 | 35.05 | 36.07 | 22.44 | 60 | 28.58 | 63 | 23.6 | 34.41 | 25.22 | 62.08 | 25.53 | |
| EngramNumber of Layers=182026.06 | 37.16 | 35.56 | 35.9 | 22.44 | 60 | 26.13 | 60.99 | 22.2 | 34.6 | 23.04 | 62.14 | 25.72 | |
| Raw GPTNumber of Layers=182026.06 | 36.85 | 34.57 | 34.05 | 22.7 | 58 | 26.78 | 62.02 | 22.8 | 33.99 | 24.35 | 60.55 | 25.5 | |
| TN-gramNumber of Layers=92026.06 | 35.44 | 30.23 | 31.1 | 19.45 | 54 | 30.71 | 57.62 | 24.4 | 29.75 | 25.22 | 61.99 | 25.35 | |
| EngramNumber of Layers=92026.06 | 34.14 | 30.14 | 31.52 | 20.48 | 55 | 21.13 | 57.51 | 23.8 | 29.87 | 22.61 | 58.17 | 25.36 | |
| Raw GPTNumber of Layers=92026.06 | 33.48 | 30 | 30.89 | 19.45 | 52 | 20.72 | 57.4 | 20.4 | 29.31 | 23.48 | 59.17 | 25.51 |