Q&A Reasoning on GPQA (Strict Accuracy, Token Metrics)
45.46Strict Match AccuracyCactus 0.75
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Cactus 0.75m (draft length)=20, Verifier model=Qwen 3 14B, Drafter model=Qwen 3 0.6B2026.04 | 45.46 | 6.46 | -46 | |
| Cactus 1.0m (draft length)=20, Verifier model=Qwen 3 14B, Drafter model=Qwen 3 0.6B2026.04 | 45.46 | 6.74 | -50 | |
| Cactus 0.75m (draft length)=10, Verifier model=Qwen 3 14B, Drafter model=Qwen 3 0.6B2026.04 | 44.95 | 5.14 | -50 | |
| Cactus 1.0m (draft length)=10, Verifier model=Qwen 3 14B, Drafter model=Qwen 3 0.6B2026.04 | 43.43 | 5.16 | -51 | |
| SpSm (draft length)=20, Verifier model=Qwen 3 8B, Drafter model=Qwen 3 0.6B2026.04 | 42.93 | 4.23 | — | |
| Cactus 0.75m (draft length)=10, Verifier model=Qwen 3 8B, Drafter model=Qwen 3 0.6B2026.04 | 41.42 | 5.33 | -47 | |
| TASm (draft length)=10, Verifier model=Qwen 3 8B, Drafter model=Qwen 3 0.6B2026.04 | 41.41 | 5.17 | -42 | |
| VerifierVerifier model=Qwen 3 8B2026.04 | 41.07 | — | — | |
| SpSm (draft length)=10, Verifier model=Qwen 3 8B, Drafter model=Qwen 3 0.6B2026.04 | 40.91 | 3.7 | — | |
| SpSm (draft length)=20, Verifier model=Qwen 3 14B, Drafter model=Qwen 3 0.6B2026.04 | 40.91 | 3.84 | — | |
| TASm (draft length)=20, Verifier model=Qwen 3 14B, Drafter model=Qwen 3 0.6B2026.04 | 40.4 | 6.41 | -46 | |
| VerifierVerifier model=Qwen 3 14B2026.04 | 40.07 | — | — | |
| Cactus 0.75m (draft length)=20, Verifier model=Qwen 3 8B, Drafter model=Qwen 3 0.6B2026.04 | 40.01 | 6.89 | -47 | |
| Cactus 1.0m (draft length)=20, Verifier model=Qwen 3 8B, Drafter model=Qwen 3 0.6B2026.04 | 39.9 | 7.05 | -49 | |
| Cactus 1.0m (draft length)=10, Verifier model=Qwen 3 8B, Drafter model=Qwen 3 0.6B2026.04 | 39.39 | 5.44 | -48 | |
| SpSm (draft length)=10, Verifier model=Qwen 3 14B, Drafter model=Qwen 3 0.6B2026.04 | 39.39 | 3.37 | — | |
| TASm (draft length)=20, Verifier model=Qwen 3 8B, Drafter model=Qwen 3 0.6B2026.04 | 38.89 | 6.68 | -46 | |
| TASm (draft length)=10, Verifier model=Qwen 3 14B, Drafter model=Qwen 3 0.6B2026.04 | 38.89 | 4.99 | -46 |