Moral Reasoning on MoralExceptQA (challenge set)
64.47F1 ScoreMORALCOT
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| MORALCOTPrompt setting=Chain-of-Thought, averaged over 4 paraphrases2022.10 | 64.47 | 66.05 | 66.96 | 0.38 | 3.2 | 62.1 | 70.68 | 54.04 | |
| Delphi++Prompt setting=Averaged over 4 paraphrases2022.10 | 58.27 | 62.16 | 76.79 | 0.34 | 1.34 | 36.61 | 70.6 | 40.81 | |
| InstructGPTPrompt setting=Averaged over 4 paraphrases2022.10 | 53.94 | 64.36 | 98.52 | 0.38 | 1.59 | 42.4 | 70 | 50.48 | |
| BERT-largePrompt setting=Averaged over 4 paraphrases2022.10 | 52.49 | 56.53 | 69.61 | 0.27 | 0.71 | 42.53 | 62.46 | 45.46 | |
| GPT3Prompt setting=Averaged over 4 paraphrases2022.10 | 52.32 | 58.95 | 80.67 | 0.27 | 0.72 | 36.53 | 72.58 | 41.2 | |
| Random BaselinePrompt setting=Averaged over 4 paraphrases2022.10 | 49.37 | 48.82 | 40.08 | 0.35 | 1 | 44.88 | 57.55 | 48.36 | |
| DelphiPrompt setting=Averaged over 4 paraphrases2022.10 | 48.51 | 61.26 | 97.7 | 0.42 | 2.92 | 33.33 | 70.6 | 44.29 | |
| Always NoPrompt setting=Constant answer2022.10 | 45.99 | 60.81 | 100 | 0.258 | 0.7 | 33.33 | 70.6 | 33.33 | |
| BERT-basePrompt setting=Averaged over 4 paraphrases2022.10 | 45.28 | 48.87 | 64.16 | 0.26 | 0.82 | 40.81 | 51.65 | 43.51 | |
| ROBERTa-largePrompt setting=Averaged over 4 paraphrases2022.10 | 23.76 | 39.64 | 0.75 | 0.3 | 0.76 | 34.96 | 6.89 | 38.32 | |
| ALBERT-xxlargePrompt setting=Averaged over 4 paraphrases2022.10 | 22.07 | 39.19 | 0 | 0.46 | 1.41 | 33.33 | 6.89 | 33.33 |