Multiple Choice Question Answering on GPQA Diamond
95AccuracyPass@100
Evaluation results
| Method | Links | |
|---|---|---|
| Pass@100Generator Model=Llama 3.1 8B Instruct, Supervision Setting=Oracle2026.04 | 95 | |
| Pass@100Generator Model=Llama 3.3 70B Instruct, Supervision Setting=Oracle2026.04 | 75.3 | |
| FUSEGenerator Model=Llama 3.3 70B Instruct, Supervision Setting=Unsupervised2026.04 | 64.4 | |
| WeaverGenerator Model=Llama 3.3 70B Instruct, Supervision Setting=Supervised2026.04 | 64.1 | |
| LogisticGenerator Model=Llama 3.3 70B Instruct, Supervision Setting=Supervised2026.04 | 63.3 | |
| Naive EnsembleGenerator Model=Llama 3.3 70B Instruct, Supervision Setting=Unsupervised2026.04 | 57.6 | |
| Naive BayesGenerator Model=Llama 3.3 70B Instruct, Supervision Setting=Supervised2026.04 | 57.1 | |
| OBV (Oracle Best Verifier)Generator Model=Llama 3.3 70B Instruct, Supervision Setting=Supervised2026.04 | 50.8 | |
| Majority VoteGenerator Model=Llama 3.3 70B Instruct, Supervision Setting=Unsupervised2026.04 | 49.5 | |
| WeaverGenerator Model=Llama 3.1 8B Instruct, Supervision Setting=Supervised2026.04 | 46.5 | |
| FUSEGenerator Model=Llama 3.1 8B Instruct, Supervision Setting=Unsupervised2026.04 | 42.7 | |
| Pass@1Generator Model=Llama 3.3 70B Instruct, Supervision Setting=Unsupervised2026.04 | 42.3 | |
| OBV (Oracle Best Verifier)Generator Model=Llama 3.1 8B Instruct, Supervision Setting=Supervised2026.04 | 40.6 | |
| Naive EnsembleGenerator Model=Llama 3.1 8B Instruct, Supervision Setting=Unsupervised2026.04 | 38.4 | |
| Naive BayesGenerator Model=Llama 3.1 8B Instruct, Supervision Setting=Supervised2026.04 | 37.5 | |
| LogisticGenerator Model=Llama 3.1 8B Instruct, Supervision Setting=Supervised2026.04 | 34.3 | |
| Majority VoteGenerator Model=Llama 3.1 8B Instruct, Supervision Setting=Unsupervised2026.04 | 32.3 | |
| Pass@1Generator Model=Llama 3.1 8B Instruct, Supervision Setting=Unsupervised2026.04 | 28.3 |