Follow-up Questioning Consistency on StrategyQA (unseen)
42.65Average Success Count (M.)Vicuna-7B + SFT
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Vicuna-7B + SFTType=O2023.10 | 42.65 | — | — | — | — | — | — | — | 83 | 51.38 | |
| Vicuna-7BType=O2023.10 | 36.68 | — | — | — | — | — | — | — | 69.61 | 52.69 | |
| Vicuna-7BType=L2023.10 | 32.9 | — | — | — | — | — | — | — | 64.76 | 50.8 | |
| Vicuna-7BType=C2023.10 | 27.07 | — | — | — | — | — | — | — | 50.13 | 54 | |
| UNWAVERING-FQ (+ SFT + DPO)Type=O2023.10 | 13.25 | — | — | — | — | — | — | — | 27.91 | 47.45 | |
| Vicuna-7B + SFTType=L2023.10 | 12.81 | — | — | — | — | — | — | — | 25.51 | 50.22 | |
| Vicuna-7BModel=Vicuna-7B, Training stage=Base, Evaluation protocol=Progressive Form2023.10 | 10.78 | 52.84 | 8.44 | 15.98 | 14.99 | 28.37 | 42.07 | 79.61 | 47.36 | — | |
| Vicuna-7B + SFTType=C2023.10 | 10.63 | — | — | — | — | — | — | — | 20.92 | 50.8 | |
| Vicuna-7B + SFTModel=Vicuna-7B, Training stage=SFT, Evaluation protocol=Progressive Form2023.10 | 9.41 | 51.09 | 4.22 | 8.26 | 14.56 | 28.49 | 16.16 | 31.62 | 16.84 | — | |
| UNWAVERING-FQ (+ SFT + DPO)Type=C2023.10 | 9.17 | — | — | — | — | — | — | — | 19.57 | 46.87 | |
| UNWAVERING-FQ (+ SFT + DPO)Type=L2023.10 | 8.59 | — | — | — | — | — | — | — | 18.1 | 47.45 | |
| UNWAVERING-FQ (Vicuna-7B + SFT + DPO)Model=Vicuna-7B, Training stage=SFT + DPO, Evaluation protocol=Progressive Form2023.10 | 7.06 | 46.29 | 3.49 | 7.55 | 11.94 | 25.79 | 15.43 | 33.33 | 13.57 | — |