General Reasoning on GSM8K, MATH-500, GPQA-Diamond, and TheoremQA Aggregate
65.78Average Pass@1 AccuracyTYLER Stage 2
Evaluation results
| Method | Links | |
|---|---|---|
| TYLER Stage 2Backbone=Qwen3-4B2026.06 | 65.78 | |
| TYLER Stage 1Backbone=Qwen3-4B2026.06 | 63.38 | |
| GRPOBackbone=Qwen3-4B2026.06 | 62.65 | |
| MemGenBackbone=Qwen3-4B2026.06 | 61.28 | |
| SoftCoTBackbone=Qwen3-4B2026.06 | 58.56 | |
| CoTBackbone=Qwen3-4B2026.06 | 56.56 | |
| Soft-ThinkingBackbone=Qwen3-4B2026.06 | 56.12 | |
| TYLER Stage 2Backbone=SmolLM3-3B2026.06 | 56.03 | |
| SwiReasoningBackbone=Qwen3-4B2026.06 | 55.72 | |
| SFTBackbone=Qwen3-4B2026.06 | 54.27 | |
| TYLER Stage 1Backbone=SmolLM3-3B2026.06 | 53.43 | |
| MemGenBackbone=SmolLM3-3B2026.06 | 51.73 | |
| SoftCoTBackbone=SmolLM3-3B2026.06 | 50.58 | |
| GRPOBackbone=SmolLM3-3B2026.06 | 50.43 | |
| SFTBackbone=SmolLM3-3B2026.06 | 49.16 | |
| Soft-ThinkingBackbone=SmolLM3-3B2026.06 | 46.98 | |
| SwiReasoningBackbone=SmolLM3-3B2026.06 | 45.92 | |
| CoTBackbone=SmolLM3-3B2026.06 | 41.54 |