Theorem-oriented Reasoning on TheoremQA (Pass@1 accuracy)
41.02Pass@1 AccuracyTYLER Stage 2
Evaluation Results
| Method | Links | |
|---|---|---|
| TYLER Stage 2Backbone=Qwen3-4B2026.06 | 41.02 | |
| GRPOBackbone=Qwen3-4B2026.06 | 39.36 | |
| TYLER Stage 1Backbone=Qwen3-4B2026.06 | 38.95 | |
| CoTBackbone=Qwen3-4B2026.06 | 36.95 | |
| MemGenBackbone=Qwen3-4B2026.06 | 36.87 | |
| Soft-ThinkingBackbone=Qwen3-4B2026.06 | 36.41 | |
| SFTBackbone=Qwen3-4B2026.06 | 35.61 | |
| SoftCoTBackbone=Qwen3-4B2026.06 | 34.94 | |
| SwiReasoningBackbone=Qwen3-4B2026.06 | 33.5 | |
| TYLER Stage 2Backbone=SmolLM3-3B2026.06 | 29 | |
| TYLER Stage 1Backbone=SmolLM3-3B2026.06 | 28.76 | |
| MemGenBackbone=SmolLM3-3B2026.06 | 27.3 | |
| GRPOBackbone=SmolLM3-3B2026.06 | 26.51 | |
| SFTBackbone=SmolLM3-3B2026.06 | 25.3 | |
| SoftCoTBackbone=SmolLM3-3B2026.06 | 25.03 | |
| Soft-ThinkingBackbone=SmolLM3-3B2026.06 | 21.15 | |
| CoTBackbone=SmolLM3-3B2026.06 | 19.54 | |
| SwiReasoningBackbone=SmolLM3-3B2026.06 | 19.4 |