Reasoning on bAbI (test)
96.27Acc (Single Supporting Fact)ChatGPT
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| ChatGPTModel Version=GPT3.5-turbo2023.05 | 96.27 | 47.33 | 16.82 | 93.96 | 76.84 | 93.66 | 61.63 | 58.38 | |
| GPT-J (Ours)Backbone=GPT-J, Configuration=Ours2023.05 | 68.98 | 26.08 | 30.41 | 69.08 | 84.99 | 63.95 | 59.42 | 53.23 | |
| GPT-J (Base)Backbone=GPT-J, Configuration=Base2023.05 | 65.16 | 40.48 | 22.46 | 41.39 | 34.74 | 42.8 | 36.96 | 49.7 | |
| GPT-Neo (Ours)Backbone=GPT-Neo, Configuration=Ours2023.05 | 56.29 | 30.82 | 13.49 | 48.84 | 51.76 | 65.56 | 23.46 | 53.64 | |
| GPT-Neo (Base)Backbone=GPT-Neo, Configuration=Base2023.05 | 51.86 | 33.43 | 7.85 | 34.04 | 14.8 | 36.05 | 21.45 | 50.51 |