Coherence judging on TW (test)
83.6Coherence (%)BART (Baseline)
Evaluation Results
| Method | Links | |
|---|---|---|
| BART (Baseline)Total examples (|X|)=10k, State supervision examples (|XA|)=0, SITUATION-SUPERVISION=None, Setting=Fine-tuning2022.12 | 83.6 | |
| SITUATIONSUPERVISION (BART)Total examples (|X|)=1k, State supervision examples (|XA|)=500, SITUATION-SUPERVISION=Aux + Latent, Setting=Fine-tuning2022.12 | 83.4 | |
| SITUATIONSUPERVISION (BART)Total examples (|X|)=1k, State supervision examples (|XA|)=1k, SITUATION-SUPERVISION=Aux Only, Setting=Fine-tuning2022.12 | 81.5 | |
| SITUATIONSUPERVISION (BART)Total examples (|X|)=1k, State supervision examples (|XA|)=500, SITUATION-SUPERVISION=Aux Only, Setting=Fine-tuning2022.12 | 80.5 | |
| BART (Baseline)Total examples (|X|)=1k, State supervision examples (|XA|)=0, SITUATION-SUPERVISION=None, Setting=Fine-tuning2022.12 | 79.4 | |
| SITUATIONSUPERVISION (GPT-3)Total examples (|X|)=25, State supervision examples (|XA|)=12, SITUATION-SUPERVISION=Aux + Latent, Setting=Prompting2022.12 | 75.6 | |
| SITUATIONSUPERVISIONTotal Examples (|X|)=25, State Supervision (|XA|)=12, Supervision Type=Aux+Latent2022.12 | 75.6 | |
| SITUATIONSUPERVISION (GPT-3)Total examples (|X|)=25, State supervision examples (|XA|)=25, SITUATION-SUPERVISION=Aux Only, Setting=Prompting2022.12 | 73.9 | |
| SITUATIONSUPERVISIONTotal Examples (|X|)=25, State Supervision (|XA|)=25, Supervision Type=Aux Only2022.12 | 73.9 | |
| SITUATIONSUPERVISION (GPT-3)Total examples (|X|)=25, State supervision examples (|XA|)=12, SITUATION-SUPERVISION=Aux Only, Setting=Prompting2022.12 | 68.5 | |
| SITUATIONSUPERVISIONTotal Examples (|X|)=25, State Supervision (|XA|)=12, Supervision Type=Aux Only2022.12 | 68.5 | |
| GPT-3 (Baseline)Total examples (|X|)=25, State supervision examples (|XA|)=0, SITUATION-SUPERVISION=None, Setting=Prompting2022.12 | 67.4 | |
| GPT-3Total Examples (|X|)=25, State Supervision (|XA|)=0, Supervision Type=None2022.12 | 67.4 |