Scientific Reasoning on GPQA Diamond (Pass@1 accuracy)
43.43Pass@1 AccuracyTYLER Stage 2
Evaluation results
| Method | Links | |
|---|---|---|
| TYLER Stage 2Backbone=Qwen3-4B2026.06 | 43.43 | |
| MemGenBackbone=Qwen3-4B2026.06 | 40.4 | |
| GRPOBackbone=Qwen3-4B2026.06 | 39.9 | |
| TYLER Stage 1Backbone=Qwen3-4B2026.06 | 39.9 | |
| SoftCoTBackbone=Qwen3-4B2026.06 | 38.38 | |
| TYLER Stage 2Backbone=SmolLM3-3B2026.06 | 33.5 | |
| TYLER Stage 1Backbone=SmolLM3-3B2026.06 | 30.5 | |
| GRPOBackbone=SmolLM3-3B2026.06 | 28.28 | |
| MemGenBackbone=SmolLM3-3B2026.06 | 25.25 | |
| SoftCoTBackbone=SmolLM3-3B2026.06 | 24.24 | |
| SFTBackbone=SmolLM3-3B2026.06 | 21.72 | |
| Soft-ThinkingBackbone=SmolLM3-3B2026.06 | 21.72 | |
| SFTBackbone=Qwen3-4B2026.06 | 21.72 | |
| SwiReasoningBackbone=SmolLM3-3B2026.06 | 20.7 | |
| CoTBackbone=SmolLM3-3B2026.06 | 19.7 | |
| SwiReasoningBackbone=Qwen3-4B2026.06 | 17.17 | |
| CoTBackbone=Qwen3-4B2026.06 | 16.16 | |
| Soft-ThinkingBackbone=Qwen3-4B2026.06 | 15.15 |