Question Answering on COTEMPQA 1.0 (test)
97Equal AccuracyHUMAN
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| HUMAN2024.06 | 97 | 98.3 | 97.7 | 91.1 | 93.5 | 92.3 | 82 | 87 | 84.5 | 88 | 96.2 | 92.1 | 92.8 | |
| GPT-42024.06 | 91.1 | 94.3 | 92.7 | 55.3 | 63.5 | 59.4 | 44.3 | 55.8 | 50.1 | 23.4 | 66.5 | 45 | 54.7 | |
| GPT-3.5-Turbo2024.06 | 59.4 | 66.3 | 62.8 | 40.1 | 48.5 | 44.3 | 31.5 | 42.9 | 37.2 | 0.7 | 46.1 | 23.4 | 38.9 | |
| WIZARDMATHParameter Scale=70B2024.06 | 36.7 | 46.8 | 41.8 | 23.6 | 33.7 | 28.6 | 25.5 | 37.1 | 31.3 | 0.4 | 32.9 | 16.6 | 30.1 | |
| WIZARDMATHParameter Scale=13B2024.06 | 23.9 | 29 | 26.4 | 10.9 | 15.1 | 13 | 11.7 | 17.1 | 14.4 | 0 | 13.2 | 6.6 | 14.4 | |
| WIZARDCODERParameter Scale=34B2024.06 | 19.5 | 26.3 | 22.9 | 15.2 | 22.4 | 18.8 | 15.9 | 23.9 | 19.9 | 0.9 | 25.9 | 13.4 | 19.2 | |
| CODELLAMAParameter Scale=34B2024.06 | 16.1 | 46.5 | 31.3 | 9.8 | 27 | 18.4 | 8.1 | 28.4 | 18.3 | 4.4 | 40.3 | 22.4 | 20 | |
| GPT-3.5-Turbo2024.06 | 13.8 | 14.8 | 14.3 | 11.3 | 14.3 | 12.8 | 15 | 22.9 | 18.9 | 0 | 15.5 | 7.7 | 16.3 | |
| WIZARDMATHParameter Scale=7B2024.06 | 12.4 | 16.5 | 14.4 | 9.2 | 15.2 | 12.2 | 11.6 | 20.5 | 16 | 0.4 | 22 | 11.2 | 14.8 | |
| LLAMAParameter Scale=70B2024.06 | 11.9 | 41.7 | 26.8 | 10 | 32.5 | 21.2 | 9.4 | 33.5 | 21.4 | 5.2 | 42.5 | 23.8 | 22.2 | |
| GPT-42024.06 | 11.2 | 12.3 | 11.8 | 11.5 | 14 | 12.7 | 14.8 | 18.5 | 16.7 | 0 | 13.6 | 6.8 | 14.5 | |
| LLAMAParameter Scale=13B2024.06 | 11.2 | 31.2 | 21.2 | 5.8 | 21.6 | 13.7 | 5 | 20.6 | 12.8 | 1.1 | 26.9 | 14 | 13.8 | |
| WIZARDCODERParameter Scale=7B2024.06 | 9.2 | 21.1 | 15.1 | 4.7 | 14.8 | 9.8 | 6.3 | 15.9 | 11.1 | 0.5 | 20.4 | 10.5 | 11.2 | |
| WIZARDCODERParameter Scale=13B2024.06 | 8.3 | 16.6 | 12.4 | 7 | 17.8 | 12.4 | 9.5 | 19.7 | 14.6 | 1.1 | 24.1 | 12.6 | 13.9 | |
| CODELLAMAParameter Scale=13B2024.06 | 7.6 | 28.3 | 18 | 4.1 | 17 | 10.6 | 3.3 | 19.3 | 11.3 | 3.2 | 28.6 | 15.9 | 12.4 | |
| CODELLAMAParameter Scale=7B2024.06 | 6.4 | 27.7 | 17 | 3.1 | 14.6 | 8.8 | 3.1 | 15.8 | 9.5 | 2 | 24.1 | 13 | 10.5 | |
| LLAMAParameter Scale=7B2024.06 | 4.1 | 18.9 | 11.5 | 4.7 | 19.5 | 12.1 | 4.5 | 19.5 | 12 | 0.2 | 23.8 | 12 | 12 |