Scientific Reasoning on GPQA Diamond (Samples Required & Average Improvement)
8Samples RequiredBlend-ASC
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Blend-ASCSC@n=64, Model=Qwen-32B2025.11 | 8 | 5.33 | |
| Adaptive SCSC@n=64, Model=Qwen-32B2025.11 | 13 | 4.36 | |
| Fixed-AllocationSC@n=64, Model=Qwen-32B2025.11 | 15 | 4.09 | |
| Blend-ASCSC@n=128, Model=Qwen-32B2025.11 | 25 | 4.29 | |
| Blend-ASCSC@n=64, Model=Llama-3B2025.11 | 26 | 5.33 | |
| Fixed-AllocationSC@n=64, Model=Llama-3B2025.11 | 27 | 4.09 | |
| Adaptive SCSC@n=128, Model=Qwen-32B2025.11 | 29 | 2.13 | |
| Adaptive SCSC@n=64, Model=Llama-3B2025.11 | 33 | 4.36 | |
| Fixed-AllocationSC@n=128, Model=Qwen-32B2025.11 | 37 | 3.4 | |
| Fixed-AllocationSC@n=128, Model=Llama-3B2025.11 | 77 | 3.4 | |
| Blend-ASCSC@n=128, Model=Llama-3B2025.11 | 80 | 4.29 | |
| Adaptive SCSC@n=128, Model=Llama-3B2025.11 | 102 | 2.13 |