Long-form Question Answering on Long-form QA (test)
61.7Win Rate vs. Holistic RewardALARM
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| ALARM2024.03 | 61.7 | 75.2 | — | — | — | 97.6 | |
| Holistic Reward2024.03 | 60.8 | 73.6 | — | — | — | 87.8 | |
| Weighted Sum2024.03 | 59.8 | 73.8 | — | — | — | 98.5 | |
| ALARMEvaluator=gpt-3.5-turbo2024.03 | 59.06 | 49.23 | 53.96 | — | 54.08 | — | |
| Factuality RewardEvaluator=gpt-3.5-turbo2024.03 | 55.88 | — | 53.55 | 50.77 | 53.4 | — | |
| Weighted SumEvaluator=gpt-3.5-turbo2024.03 | 55.84 | 46.45 | — | 46.03 | 49.44 | — | |
| ALARMEvaluator=Factuality Rate2024.03 | 55.58 | 51.3 | 55.35 | — | 54.08 | — | |
| Factuality Reward2024.03 | 53.8 | 75.2 | — | — | — | 116.8 | |
| Factuality RewardEvaluator=Factuality Rate2024.03 | 51.68 | — | 52.76 | 48.7 | 51.05 | — | |
| ALARMEvaluator=Holistic Reward2024.03 | 50.94 | 57.27 | 51.45 | — | 53.22 | — | |
| Weighted SumEvaluator=Holistic Reward2024.03 | 49.64 | 54.95 | — | 48.55 | 51.05 | — | |
| Weighted SumEvaluator=Factuality Rate2024.03 | 49.11 | 47.24 | — | 44.65 | 47 | — | |
| Factuality RewardEvaluator=Holistic Reward2024.03 | 44.13 | — | 45.04 | 42.73 | 43.97 | — | |
| Holistic RewardEvaluator=Holistic Reward2024.03 | — | 55.86 | 50.36 | 49.06 | 51.76 | — | |
| Holistic RewardEvaluator=Factuality Rate2024.03 | — | 48.31 | 50.89 | 44.42 | 47.87 | — | |
| Holistic RewardEvaluator=gpt-3.5-turbo2024.03 | — | 44.11 | 44.16 | 40.94 | 43.07 | — |