Commonsense Knowledge Generation on ComFact ROCStories portion
82.04RelevanceGold
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Gold2024.02 | 82.04 | — | — | — | — | — | — | — | 94.79 | |
| GoldDescription=Reference ground truth2024.02 | 80.9 | 10.55 | 5.64 | — | — | — | — | — | — | |
| GrapherBackbone=T5-large, Clustering metric=Euclidean distance2024.02 | 68.29 | 5.08 | 2.6 | 40.58 | 50.91 | 1.4 | 23.96 | 27.21 | — | |
| DIFFUCOMET-EntityBackbone=BART-large, Clustering metric=Euclidean distance2024.02 | 66.39 | 12.89 | 5.67 | 74.38 | 70.16 | 17.01 | 47.61 | 48.4 | — | |
| DIFFUCOMET-FactBackbone=BART-large, Clustering metric=Euclidean distance2024.02 | 65.64 | 12.88 | 5.24 | 71.65 | 68.51 | 15.98 | 50.06 | 51.44 | — | |
| Beam-COMETBackbone=BART-large, Fine-tuned on=COMET-BART, Decoding strategy=Diverse Beam search, Clustering metric=Euclidean distance2024.02 | 65.03 | 15 | 5.09 | 71.64 | 68.18 | 16.97 | 47.39 | 47.19 | — | |
| Beam-BARTBackbone=BART-large, Fine-tuned on=BART-large, Decoding strategy=Diverse Beam search, Clustering metric=Euclidean distance2024.02 | 64.35 | 15 | 4.6 | 71.35 | 67.67 | 13.11 | 47.7 | 46.35 | — | |
| Greedy-COMETBackbone=BART-large, Fine-tuned on=COMET-BART, Decoding strategy=Greedy, Clustering metric=Euclidean distance2024.02 | 61.42 | 1.96 | 1.19 | 50.64 | 55.51 | 18.01 | 52.32 | 54.96 | — | |
| Sampling-COMETBackbone=BART-large, Fine-tuned on=COMET-BART, Decoding strategy=Nucleus sampling, Clustering metric=Euclidean distance2024.02 | 56.19 | 15 | 8.39 | 77.97 | 65.31 | 12.69 | 44.43 | 45.58 | — | |
| DIFFUCOMET-EntityBackbone=BART-large2024.02 | 54.18 | — | — | — | — | — | — | — | 74.15 | |
| DIFFUCOMET-FactBackbone=BART-large2024.02 | 48.27 | — | — | — | — | — | — | — | 70 | |
| Beam-COMETDecoding=Beam Search2024.02 | 42.81 | — | — | — | — | — | — | — | 74.8 | |
| Sampling-COMETDecoding=Sampling2024.02 | 30.2 | — | — | — | — | — | — | — | 49.45 |