Coherence judging on TRIP (test)
70.7AccuracySITUATIONSUPERVISION (GPT-3)
Evaluation Results
| Method | Links | |
|---|---|---|
| SITUATIONSUPERVISION (GPT-3)Total examples (|X|)=80, State supervision examples (|XA|)=80, SITUATION-SUPERVISION=Aux Only, Setting=Prompting2022.12 | 70.7 | |
| SITUATIONSUPERVISIONTotal Examples (|X|)=80, State Supervision (|XA|)=80, Supervision Type=Aux Only2022.12 | 70.7 | |
| SITUATIONSUPERVISION (GPT-3)Total examples (|X|)=80, State supervision examples (|XA|)=20, SITUATION-SUPERVISION=Aux + Latent, Setting=Prompting2022.12 | 67.1 | |
| SITUATIONSUPERVISIONTotal Examples (|X|)=80, State Supervision (|XA|)=20, Supervision Type=Aux+Latent2022.12 | 67.1 | |
| GPT-3 (Baseline)Total examples (|X|)=80, State supervision examples (|XA|)=0, SITUATION-SUPERVISION=None, Setting=Prompting2022.12 | 59.5 | |
| GPT-3Total Examples (|X|)=80, State Supervision (|XA|)=0, Supervision Type=None2022.12 | 59.5 | |
| SITUATIONSUPERVISION (GPT-3)Total examples (|X|)=80, State supervision examples (|XA|)=20, SITUATION-SUPERVISION=Aux Only, Setting=Prompting2022.12 | 58.2 | |
| SITUATIONSUPERVISIONTotal Examples (|X|)=80, State Supervision (|XA|)=20, Supervision Type=Aux Only2022.12 | 58.2 | |
| SITUATIONSUPERVISION (BART)Total examples (|X|)=1k, State supervision examples (|XA|)=500, SITUATION-SUPERVISION=Aux Only, Setting=Fine-tuning2022.12 | 43.6 | |
| SITUATIONSUPERVISION (BART)Total examples (|X|)=1k, State supervision examples (|XA|)=500, SITUATION-SUPERVISION=Aux + Latent, Setting=Fine-tuning2022.12 | 43.6 | |
| SITUATIONSUPERVISION (BART)Total examples (|X|)=1k, State supervision examples (|XA|)=1k, SITUATION-SUPERVISION=Aux Only, Setting=Fine-tuning2022.12 | 43 | |
| BART (Baseline)Total examples (|X|)=1k, State supervision examples (|XA|)=0, SITUATION-SUPERVISION=None, Setting=Fine-tuning2022.12 | 36.5 |