Yes/No Question Answering on BoolQ (test)
79.2Accuracy4o
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| 4oModel Family=ChatGPT2024.12 | 79.2 | 4.9 | 11.3 | |
| o1-previewModel Family=ChatGPT2024.12 | 78.7 | 4.9 | 13.2 | |
| R1Model Family=DeepSeek2024.12 | 78.1 | 1.6 | 7.9 | |
| o1-miniModel Family=ChatGPT2024.12 | 74.1 | 4.2 | 15.6 | |
| CLINEmode=fine-tuned2021.07 | 73.9 | — | — | |
| RoBERTamode=fine-tuned2021.07 | 69.6 | — | — | |
| V3Model Family=DeepSeek2024.12 | 69 | 9.2 | 28.5 | |
| 3.5-turboModel Family=ChatGPT2024.12 | 62.5 | 12.1 | 34 | |
| BERTmode=fine-tuned2021.07 | 60.9 | — | — | |
| 2-7BModel Family=Llama2024.12 | 52.8 | 8.7 | 26.5 | |
| 3-8BModel Family=Llama2024.12 | 50.1 | 20.3 | 58.2 | |
| 3.1-8BModel Family=Llama2024.12 | 49.2 | 20.4 | 58.8 |