Scientific Reasoning on GPQA Diamond (pass@1, pass@5)
20.2Pass@1FA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FABackbone=Llama-3.2-3B-Instruct, Synthetic Data Strategy=GUIDEDSAMPLING - Final Answer (FA)2025.10 | 20.2 | 50.61 | |
| RSBackbone=Llama-3.2-3B-Instruct, Synthetic Data Strategy=Repeated Sampling2025.10 | 18.13 | 40.08 | |
| ToTBackbone=Llama-3.2-3B-Instruct, Synthetic Data Strategy=Tree-of-Thought2025.10 | 16.77 | 44.44 | |
| STaRBackbone=Llama-3.2-3B-Instruct, Synthetic Data Strategy=Self-Taught Reasoner2025.10 | 16.61 | 38.41 | |
| CAABackbone=Llama-3.2-3B-Instruct, Synthetic Data Strategy=GUIDEDSAMPLING - Concepts and Answer (CAA)2025.10 | 15.66 | 40.23 | |
| Base ModelBackbone=Llama-3.2-3B-Instruct2025.10 | 11.62 | 28.28 |