Rationale Evaluation on StrategyQA (test)
0.283RORAGPT-4
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GPT-4Rationale Category=Model Generated2024.02 | 0.283 | 0.474 | 0.501 | -0.066 | 0.161 | 0.316 | 2.69 | |
| GPT-3.5Rationale Category=Model Generated2024.02 | 0.253 | 0.459 | 0.412 | -0.083 | 0.215 | 0.311 | 2.32 | |
| gold_leakyRationale Category=Synthetic Leaky2024.02 | 0.119 | 0.348 | 0.673 | 0.145 | — | 0.406 | — | |
| goldRationale Category=Synthetic Leaky2024.02 | 0.115 | 0.381 | 0.121 | -0.038 | 0.024 | 0.138 | — | |
| Llama2-7BRationale Category=Model Generated2024.02 | 0.1 | 0.232 | 0.155 | -0.035 | 0.084 | 0.121 | 1.16 | |
| Flan-T5 LargeRationale Category=Model Generated2024.02 | 0.061 | 0.132 | 0.102 | -0.013 | 0.027 | 0.071 | 0.39 | |
| vacuousRationale Category=Synthetic Leaky2024.02 | 0.043 | 0.026 | 0.505 | -0.005 | 0.131 | 0.376 | — | |
| leakyRationale Category=Synthetic Leaky2024.02 | 0.038 | 0.024 | 0.673 | 0.147 | — | 0.406 | — |