List your company
ResearchFormerly WizWand

Coherence judging on TW (test)

83.6Coherence (%)BART (Baseline)
66.75271.12675.579.874Dec 20, 2022
Updated 8mo ago

Evaluation results

MethodLinks
BART (Baseline)Total examples (|X|)=10k, State supervision examples (|XA|)=0, SITUATION-SUPERVISION=None, Setting=Fine-tuning2022.1283.6
SITUATIONSUPERVISION (BART)Total examples (|X|)=1k, State supervision examples (|XA|)=500, SITUATION-SUPERVISION=Aux + Latent, Setting=Fine-tuning2022.1283.4
SITUATIONSUPERVISION (BART)Total examples (|X|)=1k, State supervision examples (|XA|)=1k, SITUATION-SUPERVISION=Aux Only, Setting=Fine-tuning2022.1281.5
SITUATIONSUPERVISION (BART)Total examples (|X|)=1k, State supervision examples (|XA|)=500, SITUATION-SUPERVISION=Aux Only, Setting=Fine-tuning2022.1280.5
BART (Baseline)Total examples (|X|)=1k, State supervision examples (|XA|)=0, SITUATION-SUPERVISION=None, Setting=Fine-tuning2022.1279.4
SITUATIONSUPERVISION (GPT-3)Total examples (|X|)=25, State supervision examples (|XA|)=12, SITUATION-SUPERVISION=Aux + Latent, Setting=Prompting2022.1275.6
SITUATIONSUPERVISIONTotal Examples (|X|)=25, State Supervision (|XA|)=12, Supervision Type=Aux+Latent2022.1275.6
SITUATIONSUPERVISION (GPT-3)Total examples (|X|)=25, State supervision examples (|XA|)=25, SITUATION-SUPERVISION=Aux Only, Setting=Prompting2022.1273.9
SITUATIONSUPERVISIONTotal Examples (|X|)=25, State Supervision (|XA|)=25, Supervision Type=Aux Only2022.1273.9
SITUATIONSUPERVISION (GPT-3)Total examples (|X|)=25, State supervision examples (|XA|)=12, SITUATION-SUPERVISION=Aux Only, Setting=Prompting2022.1268.5
SITUATIONSUPERVISIONTotal Examples (|X|)=25, State Supervision (|XA|)=12, Supervision Type=Aux Only2022.1268.5
GPT-3 (Baseline)Total examples (|X|)=25, State supervision examples (|XA|)=0, SITUATION-SUPERVISION=None, Setting=Prompting2022.1267.4
GPT-3Total Examples (|X|)=25, State Supervision (|XA|)=0, Supervision Type=None2022.1267.4