Generative Commonsense Inference on ATOMIC 2021 (test)
50.71BLEU-1CANDLE Distilled
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| CANDLE DistilledBackbone=GPT2-XL (1.5B), Training Data=CANDLE Distilled2024.01 | 50.71 | 33.85 | 25.55 | 20.43 | 32.45 | 51.91 | 69.68 | 76.86 | 78.5 | |
| GPT2-XLBackbone=GPT2-XL (1.5B), Training Data=ATOMIC2024.01 | 45.72 | 29.18 | 21.12 | 16.15 | 29.97 | 49.69 | 64.61 | 76.09 | 70.5 | |
| GPT2-XLBackbone=GPT2-XL (1.5B), Training Data=ATOMIC-10X2024.01 | 45.38 | 29.2 | 21.09 | 16.15 | 30.09 | 49.86 | 65.02 | 75.89 | 77.5 | |
| GPT2-XLBackbone=GPT2-XL (1.5B), Training Data=AbstractATOMIC2024.01 | 45.3 | 29.08 | 21 | 16.06 | 29.98 | 48.61 | 63.98 | 75.56 | 71.5 | |
| CANDLE DistilledBackbone=LLAMA2 (7B), Training Data=CANDLE Distilled2024.01 | 43.86 | 24.4 | 15.12 | 10 | 28.36 | 43.86 | 54.25 | 72.94 | 79.5 | |
| LLAMA2Backbone=LLAMA2 (7B), Training Data=AbstractATOMIC2024.01 | 42.78 | 23.64 | 14.58 | 9.471 | 27.74 | 42.55 | 53.12 | 71.51 | 71 | |
| GPT2-XLBackbone=GPT2-XL (1.5B), Training Data=ATOMIC202024.01 | 42.15 | 25.77 | 17.82 | 13.14 | 29.82 | 47.61 | 63.7 | 70.39 | 76.5 | |
| LLAMA2Backbone=LLAMA2 (7B), Training Data=ATOMIC-10X2024.01 | 42.07 | 23.08 | 14.14 | 9.198 | 28.14 | 42.75 | 53.69 | 71.93 | 76.5 | |
| LLAMA2Backbone=LLAMA2 (7B), Training Data=ATOMIC2024.01 | 42.04 | 23.01 | 14.1 | 9.125 | 27.8 | 42.9 | 53.17 | 71.52 | 68.5 | |
| LLAMA2Backbone=LLAMA2 (7B), Training Data=ATOMIC202024.01 | 41.07 | 22.46 | 13.62 | 8.619 | 27.74 | 42.42 | 53.28 | 71.77 | 74 | |
| LLAMA2Backbone=LLAMA2 (7B), Training Data=Five-shot2024.01 | 31.22 | 16.87 | 9.767 | 5.989 | 19.74 | 27.67 | 17.83 | 58.41 | 65.5 | |
| ChatGPTBackbone=ChatGPT (gpt-3.5-turbo), Mode=Five-shot2024.01 | 26.32 | 12.5 | 7.16 | 4.415 | 18.6 | 24.65 | 8.313 | 58.69 | 81 | |
| LLAMA2Backbone=LLAMA2 (7B), Training Data=Zero-shot2024.01 | 18.26 | 7.453 | 3.594 | 1.945 | 15.9 | 20.28 | 8.872 | 48.23 | 48.5 | |
| ChatGPTBackbone=ChatGPT (gpt-3.5-turbo), Mode=Zero-shot2024.01 | 11.82 | 4.258 | 1.891 | 0.926 | 13.87 | 13.73 | 4.35 | 49.28 | 78.5 | |
| ChatGPTBackbone=ChatGPT (gpt-3.5-turbo), Mode=Chain-of-thought2024.01 | 9.906 | 3.568 | 1.556 | 0.736 | 11.85 | 11.02 | 2.905 | 46.17 | 64 | |
| GPT2-XLBackbone=GPT2-XL (1.5B), Training Data=Zero-shot2024.01 | 4.35 | 1.598 | 0.732 | 0.293 | 5.702 | 5.03 | 0.792 | 37.11 | 14.5 |