List your company
ResearchFormerly WizWand

Theorem-oriented Reasoning on TheoremQA (Pass@1 accuracy)

41.02Pass@1 AccuracyTYLER Stage 2
18.535224.372630.2136.0474Jun 15, 2026
Updated 4mo ago

Evaluation results

MethodLinks
TYLER Stage 2Backbone=Qwen3-4B2026.0641.02
GRPOBackbone=Qwen3-4B2026.0639.36
TYLER Stage 1Backbone=Qwen3-4B2026.0638.95
CoTBackbone=Qwen3-4B2026.0636.95
MemGenBackbone=Qwen3-4B2026.0636.87
Soft-ThinkingBackbone=Qwen3-4B2026.0636.41
SFTBackbone=Qwen3-4B2026.0635.61
SoftCoTBackbone=Qwen3-4B2026.0634.94
SwiReasoningBackbone=Qwen3-4B2026.0633.5
TYLER Stage 2Backbone=SmolLM3-3B2026.0629
TYLER Stage 1Backbone=SmolLM3-3B2026.0628.76
MemGenBackbone=SmolLM3-3B2026.0627.3
GRPOBackbone=SmolLM3-3B2026.0626.51
SFTBackbone=SmolLM3-3B2026.0625.3
SoftCoTBackbone=SmolLM3-3B2026.0625.03
Soft-ThinkingBackbone=SmolLM3-3B2026.0621.15
CoTBackbone=SmolLM3-3B2026.0619.54
SwiReasoningBackbone=SmolLM3-3B2026.0619.4