Follow-up Questioning Consistency on CoinFlip (unseen)
52.2Baseline Consistency ScoreUNWAVERING-FQ (Vicuna-7B + SFT + DPO)
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| UNWAVERING-FQ (Vicuna-7B + SFT + DPO)Model=Vicuna-7B, Training stage=SFT + DPO, Evaluation protocol=Progressive Form2023.10 | 52.2 | 2 | 3.83 | 6.8 | 13.03 | 7.2 | 13.79 | 7.06 | 13.57 | |
| Vicuna-7B + SFTModel=Vicuna-7B, Training stage=SFT, Evaluation protocol=Progressive Form2023.10 | 50.4 | 1.4 | 2.78 | 6.4 | 12.7 | 7 | 13.89 | 9.41 | 16.84 | |
| Vicuna-7BModel=Vicuna-7B, Training stage=Base, Evaluation protocol=Progressive Form2023.10 | 44.4 | 0 | 0 | 0 | 0 | 23.2 | 52.25 | 10.78 | 47.36 |