Commonsense Validation and Explanation on ComVE (test)
98.5AccuracyLlama2-70B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Llama2-70BPrompt Order=explain-then-predict (EP)2024.04 | 98.5 | 69.2 | 0.238 | — | |
| Llama2-70BPrompt Order=predict-then-explain (PE)2024.04 | 97.7 | 70 | 0.172 | — | |
| Llama2-13BPrompt Order=explain-then-predict (EP)2024.04 | 75.8 | 78.4 | 0.201 | — | |
| Llama2-13BPrompt Order=predict-then-explain (PE)2024.04 | 75.6 | 82 | 0.036 | — | |
| SPARSEFIT (Att.Q+LN)Trainable Parameters=6.84%, FLOPS=2.37e14, Base Model=T5-large, Few-shot splits=602023.05 | 74.86 | — | — | 69.02 | |
| AdaLoRA (1.15%)Trainable Parameters=1.15%, FLOPS=1.48e15, Base Model=T5-large, Few-shot splits=602023.05 | 69.66 | — | — | 64.06 | |
| LoRA (Att.KQVO, Rank=4)Trainable Parameters=0.32%, FLOPS=2.75e14, Base Model=T5-large, Few-shot splits=602023.05 | 68.96 | — | — | 63.48 | |
| LoRA (Att.QV, Rank=128)Trainable Parameters=4.86%, FLOPS=2.88e14, Base Model=T5-large, Few-shot splits=602023.05 | 67.77 | — | — | 61.36 | |
| (IA)³Trainable Parameters=0.07%, FLOPS=2.74e14, Base Model=T5-large, Few-shot splits=602023.05 | 58.53 | — | — | 53.87 | |
| Llama2-7BPrompt Order=predict-then-explain (PE)2024.04 | 55.2 | 81.3 | 0.04 | — | |
| Llama2-7BPrompt Order=explain-then-predict (EP)2024.04 | 52.4 | 78.7 | 0.125 | — | |
| Random2024.04 | 50 | — | 0 | — | |
| AdaLoRA (4.48%)Trainable Parameters=4.48%, FLOPS=2.87e14, Base Model=T5-large, Few-shot splits=602023.05 | 19.43 | — | — | 16.26 |