Rule-of-Thumb Generation on MIC (test)
55.18R-1Alignedd-PM
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Alignedd-PMBackbone=Flan-T5 (Base), Decoding Strategy=Beam2023.10 | 55.18 | 35.07 | 53.86 | 93.79 | 30.83 | 9.01 | |
| BaseBackbone=Flan-T5 (Base), Decoding Strategy=Beam2023.10 | 55.07 | 34.96 | 53.74 | 93.77 | 30.68 | 9 | |
| Alignedd-PMBackbone=BART (Large), Decoding Strategy=Beam2023.10 | 55.05 | 35.18 | 53.62 | 93.86 | 30.85 | 9.4 | |
| Alignedd-PMBackbone=BART (Large), Decoding Strategy=p=0.92023.10 | 54.86 | 35.01 | 53.45 | 93.85 | 30.63 | 9.48 | |
| Aligned_softBackbone=Flan-T5 (Base), Decoding Strategy=Beam2023.10 | 54.82 | 34.65 | 53.49 | 93.75 | 30.34 | 9 | |
| Aligned_softBackbone=BART (Large), Decoding Strategy=Beam2023.10 | 54.82 | 34.85 | 53.36 | 93.82 | 30.35 | 9.36 | |
| BaseBackbone=BART (Large), Decoding Strategy=Beam2023.10 | 54.81 | 35.07 | 53.35 | 93.85 | 30.8 | 9.44 | |
| Alignedd-PMBackbone=BART (Large), Decoding Strategy=Greedy2023.10 | 54.81 | 34.86 | 53.39 | 93.83 | 30.51 | 9.48 | |
| BaseBackbone=BART (Large), Decoding Strategy=Greedy2023.10 | 54.77 | 34.85 | 53.3 | 93.84 | 30.51 | 9.54 | |
| BaseBackbone=BART (Large), Decoding Strategy=p=0.92023.10 | 54.77 | 34.96 | 53.32 | 93.84 | 30.62 | 9.56 | |
| Aligned_softBackbone=BART (Large), Decoding Strategy=p=0.92023.10 | 54.65 | 34.68 | 53.23 | 93.82 | 30.16 | 9.45 | |
| Aligned_softBackbone=BART (Large), Decoding Strategy=Greedy2023.10 | 54.54 | 34.53 | 53.1 | 93.8 | 30.01 | 9.47 | |
| Alignedd-PMBackbone=T5 (Small), Decoding Strategy=Beam2023.10 | 53.45 | 32.82 | 52.09 | 93.49 | 28.51 | 8.99 | |
| BaseBackbone=T5 (Small), Decoding Strategy=Beam2023.10 | 53.44 | 32.97 | 52.17 | 93.44 | 29.05 | 8.94 | |
| Aligned_softBackbone=T5 (Small), Decoding Strategy=Beam2023.10 | 52.14 | 31.48 | 50.79 | 93.34 | 27.05 | 8.85 | |
| Alignedd-PMBackbone=Flan-T5 (Base), Decoding Strategy=p=0.92023.10 | 41.78 | 20.69 | 40.09 | 92.07 | 18.26 | 9.32 | |
| Aligned_softBackbone=Flan-T5 (Base), Decoding Strategy=p=0.92023.10 | 41.44 | 20.33 | 39.71 | 92.02 | 17.91 | 9.29 | |
| Alignedd-PMBackbone=T5 (Small), Decoding Strategy=p=0.92023.10 | 41.41 | 20.22 | 39.75 | 91.9 | 17.75 | 9.38 | |
| BaseBackbone=Flan-T5 (Base), Decoding Strategy=p=0.92023.10 | 41.41 | 20.41 | 39.7 | 92.02 | 18.02 | 9.3 | |
| Aligned_softBackbone=T5 (Small), Decoding Strategy=p=0.92023.10 | 40.9 | 19.7 | 39.24 | 91.79 | 16.98 | 9.47 | |
| BaseBackbone=T5 (Small), Decoding Strategy=p=0.92023.10 | 40.22 | 19.23 | 38.56 | 91.59 | 16.71 | 9.85 | |
| Alignedd-PMBackbone=Flan-T5 (Base), Decoding Strategy=Greedy2023.10 | 38.34 | 17.52 | 36.53 | 91.44 | 15.49 | 9.77 | |
| Alignedd-PMBackbone=T5 (Small), Decoding Strategy=Greedy2023.10 | 38.15 | 17.22 | 36.4 | 91.29 | 15.15 | 9.74 | |
| BaseBackbone=Flan-T5 (Base), Decoding Strategy=Greedy2023.10 | 37.94 | 17.23 | 36.13 | 91.39 | 15.36 | 9.78 | |
| Aligned_softBackbone=Flan-T5 (Base), Decoding Strategy=Greedy2023.10 | 37.84 | 17.03 | 36 | 91.38 | 15.12 | 9.75 | |
| Aligned_softBackbone=T5 (Small), Decoding Strategy=Greedy2023.10 | 37.77 | 16.82 | 35.97 | 91.21 | 14.68 | 9.83 | |
| BaseBackbone=T5 (Small), Decoding Strategy=Greedy2023.10 | 37.04 | 16.3 | 35.27 | 90.94 | 14.27 | 10.47 |