Linguistic Suitability Evaluation on INFOLOSSQA (test)
82Givenness Good (%)GPT-4
Evaluation Results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-42024.01 | 82 | 52 | 1.5 | 46.5 | 15 | 24 | 11 | 50 | 90.5 | 99 | |
| Llama2-70B2024.01 | 75.2 | 34.5 | 5.3 | 60.2 | 26.2 | 7.8 | 17.5 | 48.5 | 79.6 | 92.2 | |
| Mistral-7B2024.01 | 74.5 | 52 | 5.9 | 42.2 | 21.6 | 9.3 | 13.2 | 55.9 | 84.3 | 96.1 | |
| NLI Pipeline2024.01 | 58.3 | 17.6 | 0 | 82.4 | 28.9 | 29.9 | 12.7 | 28.4 | 86.3 | 98 |