Science Question Answering on GPQA (Accuracy, Length, Speed, Modification)
10,500.2Average Response LengthSpeculative Thinking
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Speculative Thinkingspeculative model=1B-Instruct, target model=32B, pass@1=true2025.04 | 10,500.2 | 26 | 62.4 | 33.3 | 40.5 | |
| Speculative Thinkingspeculative model=1B-Instruct, target model=7B, pass@1=true2025.04 | 9,019.3 | 26 | 95.4 | 30.3 | 27.8 | |
| 1B-Instructtarget model=normal, pass@1=true2025.04 | 694.9 | — | 164.9 | 23.7 | — |