Scientific Question Answering on GPQA (accuracy)
54.44AccuracySpurious
Evaluation Results
| Method | Links | |
|---|---|---|
| SpuriousModel=Qwen/Qwen3.5-27B2026.05 | 54.44 | |
| Spurious UniversalModel=Qwen/Qwen3.5-27B2026.05 | 50.8 | |
| PromptWizardModel=Qwen/Qwen3.5-27B2026.05 | 49.59 | |
| Qwen3 8BGen. Mode=AR, TPF=1.002026.07 | 49.24 | |
| Nemotron-Labs-Diffusion-8BGen. Mode=AR, TPF=1.002026.07 | 44.44 | |
| Nemotron-Labs-Diffusion-8BGen. Mode=Quad. SS, TPF=6.382026.07 | 44.3 | |
| Nemotron-Labs-Diffusion-8BGen. Mode=Diff., TPF=2.572026.07 | 43.94 | |
| Ministral3-8B Instruct-2512Gen. Mode=AR, TPF=1.002026.07 | 42.87 | |
| Step-BackModel=Qwen/Qwen3.5-27B2026.05 | 41.11 | |
| Nemotron-Labs-Diffusion-8BGen. Mode=Linear SS, TPF=5.992026.07 | 40.4 | |
| SDAR-8B ChatGen. Mode=Diff., TPF=1.002026.07 | 40.2 | |
| Least-to-MostModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 40 | |
| Plan-and-SolveModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 37.78 | |
| Least-to-MostModel=Qwen/Qwen3.5-27B2026.05 | 37.78 | |
| Qwen2.5 7BGen. Mode=AR, TPF=1.002026.07 | 37.12 | |
| Step-BackModel=Qwen/Qwen3.5-0.8B2026.05 | 36.67 | |
| AnalogicalModel=Qwen/Qwen3.5-27B2026.05 | 36.67 | |
| Zero-Shot CoTModel=Qwen/Qwen3.5-0.8B2026.05 | 35.56 | |
| Plan-and-SolveModel=Qwen/Qwen3.5-27B2026.05 | 35.56 | |
| Self-AskModel=Qwen/Qwen3.5-27B2026.05 | 35.56 | |
| Self-AskModel=Qwen/Qwen3.5-0.8B2026.05 | 34.44 | |
| Re-ReadingModel=allenai/Olmo-3-7B-Instruct2026.05 | 34.44 | |
| AnalogicalModel=Qwen/Qwen3.5-0.8B2026.05 | 33.33 | |
| Re-ReadingModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 33.33 | |
| Self-AskModel=allenai/Olmo-3-7B-Instruct2026.05 | 33.33 | |
| PromptWizardModel=allenai/Olmo-3-7B-Instruct2026.05 | 33.33 | |
| SpuriousModel=allenai/Olmo-3-7B-Instruct2026.05 | 33.33 | |
| Re-ReadingModel=Qwen/Qwen3.5-27B2026.05 | 33.33 | |
| Zero-Shot CoTModel=Qwen/Qwen3.5-27B2026.05 | 33.33 | |
| LLaDA-8B InstructGen. Mode=Diff., TPF=1.002026.07 | 33.3 | |
| Dream-7B InstructGen. Mode=Diff., TPF=1.002026.07 | 33 | |
| SpuriousModel=Qwen/Qwen3.5-0.8B2026.05 | 32.22 | |
| SpuriousModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 31.85 | |
| Zero-Shot CoTModel=allenai/Olmo-3-7B-Instruct2026.05 | 31.11 | |
| SDAR-8B ChatGen. Mode=Diff., TPF=1.752026.07 | 30.8 | |
| Re-ReadingModel=Qwen/Qwen3.5-0.8B2026.05 | 30 | |
| AnalogicalModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 30 | |
| Spurious UniversalModel=Qwen/Qwen3.5-0.8B2026.05 | 28.93 | |
| Step-BackModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 28.89 | |
| Least-to-MostModel=Qwen/Qwen3.5-0.8B2026.05 | 27.78 | |
| AnalogicalModel=allenai/Olmo-3-7B-Instruct2026.05 | 27.78 | |
| PromptWizardModel=Qwen/Qwen3.5-0.8B2026.05 | 27.43 | |
| Plan-and-SolveModel=Qwen/Qwen3.5-0.8B2026.05 | 26.67 | |
| Least-to-MostModel=allenai/Olmo-3-7B-Instruct2026.05 | 26.67 | |
| Spurious UniversalModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 26.3 | |
| Zero-Shot CoTModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 25.56 | |
| Step-BackModel=allenai/Olmo-3-7B-Instruct2026.05 | 25.56 | |
| Plan-and-SolveModel=allenai/Olmo-3-7B-Instruct2026.05 | 25.56 | |
| Self-AskModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 24.44 | |
| Spurious UniversalModel=allenai/Olmo-3-7B-Instruct2026.05 | 24.07 | |
| PromptWizardModel=meta-llama/Llama-3.2-1B-Instruct2026.05 | 23.7 |