Follow-up Questioning Consistency on MultiArith (unseen)
18.33Average Success CountUNWAVERING-FQ (+ SFT + DPO)
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| UNWAVERING-FQ (+ SFT + DPO)Type=C2023.10 | 18.33 | — | — | — | — | — | — | — | 28.7 | 63.89 | |
| Vicuna-7B + SFTType=C2023.10 | 13.33 | — | — | — | — | — | — | — | 20.34 | 65.56 | |
| Vicuna-7BModel=Vicuna-7B, Training stage=Base, Evaluation protocol=Progressive Form2023.10 | 10.78 | 3.33 | 2.78 | 83.33 | 2.78 | 83.33 | 2.78 | 83.33 | 47.36 | — | |
| Vicuna-7B + SFTModel=Vicuna-7B, Training stage=SFT, Evaluation protocol=Progressive Form2023.10 | 9.41 | 65 | 9.44 | 14.53 | 12.22 | 18.8 | 13.33 | 20.51 | 16.84 | — | |
| UNWAVERING-FQ (Vicuna-7B + SFT + DPO)Model=Vicuna-7B, Training stage=SFT + DPO, Evaluation protocol=Progressive Form2023.10 | 7.06 | 67.22 | 2.22 | 3.31 | 4.44 | 6.61 | 10 | 14.88 | 13.57 | — | |
| UNWAVERING-FQ (+ SFT + DPO)Type=L2023.10 | 6.67 | — | — | — | — | — | — | — | 10.17 | 65.56 | |
| UNWAVERING-FQ (+ SFT + DPO)Type=O2023.10 | 3.89 | — | — | — | — | — | — | — | 5.74 | 67.78 | |
| Vicuna-7BType=O2023.10 | 3.33 | — | — | — | — | — | — | — | 75.02 | 4.44 | |
| Vicuna-7B + SFTType=L2023.10 | 2.78 | — | — | — | — | — | — | — | 4.55 | 61.11 | |
| Vicuna-7B + SFTType=O2023.10 | 2.22 | — | — | — | — | — | — | — | 3.45 | 64.44 | |
| Vicuna-7BType=C2023.10 | 1.67 | — | — | — | — | — | — | — | 50 | 3.33 | |
| Vicuna-7BType=L2023.10 | 1.66 | — | — | — | — | — | — | — | 49.9 | 3.33 |