Reasoning on Reasoning Suite Average
74.8AccuracyRLCM
Evaluation results
| Method | Links | |
|---|---|---|
| RLCMAggregation Strategy=Confidence-Weighted (Conf.), Token Budget=8k2026.04 | 74.8 | |
| RLCMAggregation Strategy=Majority Voting (Maj.), Token Budget=8k2026.04 | 73.3 | |
| GHG-TDABase Model=Claude 3.5 Sonnet2026.02 | 72.8 | |
| GHG-TDABase Model=GPT-4o2026.02 | 72.4 | |
| GRPOAggregation Strategy=Confidence-Weighted (Conf.), Token Budget=8k2026.04 | 72.3 | |
| GRPOAggregation Strategy=Majority Voting (Maj.), Token Budget=8k2026.04 | 71.9 | |
| AoTBase Model=Claude 3.5 Sonnet2026.02 | 71.3 | |
| AoTBase Model=GPT-4o2026.02 | 71 | |
| GoTBase Model=Claude 3.5 Sonnet2026.02 | 69.2 | |
| ToTBase Model=Claude 3.5 Sonnet2026.02 | 68.9 | |
| GoTBase Model=GPT-4o2026.02 | 68.7 | |
| ToTBase Model=GPT-4o2026.02 | 68.5 | |
| RLCRAggregation Strategy=Majority Voting (Maj.), Token Budget=8k2026.04 | 68.3 | |
| RLCRAggregation Strategy=Confidence-Weighted (Conf.), Token Budget=8k2026.04 | 67.5 | |
| CoTBase Model=Claude 3.5 Sonnet2026.02 | 64.9 | |
| CoTBase Model=GPT-4o2026.02 | 64.4 | |
| GRPOAggregation Strategy=Pass@1, Token Budget=8k2026.04 | 62.9 | |
| RLCMAggregation Strategy=Pass@1, Token Budget=8k2026.04 | 62.7 | |
| RLCRAggregation Strategy=Pass@1, Token Budget=8k2026.04 | 58.9 | |
| Pythia6.9BShots=52025.06 | 0.5729 | |
| TokAlign (LLaMA3)Shots=5, Base Model=Pythia6.9B2025.06 | 0.5727 | |
| TokAlign (LLaMA3)Shots=0, Base Model=Pythia6.9B2025.06 | 0.5652 | |
| Pythia2.8BShots=52025.06 | 0.5626 | |
| TokAlign (Qwen2)Shots=0, Base Model=Pythia6.9B2025.06 | 0.5612 | |
| Pythia6.9BShots=02025.06 | 0.5608 | |
| TokAlign (Qwen2)Shots=5, Base Model=Pythia6.9B2025.06 | 0.5602 | |
| TokAlign (Gemma)Shots=5, Base Model=Pythia6.9B2025.06 | 0.5577 | |
| TokAlign (Gemma)Shots=5, Base Model=Pythia2.8B2025.06 | 0.5553 | |
| TokAlign (Qwen2)Shots=5, Base Model=Pythia2.8B2025.06 | 0.5529 | |
| TokAlign (LLaMA3)Shots=5, Base Model=Pythia2.8B2025.06 | 0.5512 | |
| Pythia2.8BShots=02025.06 | 0.5495 | |
| TokAlign (Gemma)Shots=0, Base Model=Pythia6.9B2025.06 | 0.5489 | |
| TokAlign (LLaMA3)Shots=0, Base Model=Pythia2.8B2025.06 | 0.5427 | |
| TokAlign (Qwen2)Shots=0, Base Model=Pythia2.8B2025.06 | 0.5418 | |
| TokAlign (Gemma)Shots=0, Base Model=Pythia2.8B2025.06 | 0.5371 | |
| Pythia1BShots=02025.06 | 0.4955 | |
| Pythia1BShots=52025.06 | 0.4937 | |
| TokAlign (LLaMA3)Shots=5, Base Model=Pythia1B2025.06 | 0.4874 | |
| TokAlign (LLaMA3)Shots=0, Base Model=Pythia1B2025.06 | 0.4859 | |
| TokAlign (Gemma)Shots=0, Base Model=Pythia1B2025.06 | 0.4842 | |
| TokAlign (Qwen2)Shots=0, Base Model=Pythia1B2025.06 | 0.4791 | |
| TokAlign (Qwen2)Shots=5, Base Model=Pythia1B2025.06 | 0.4756 | |
| TokAlign (Gemma)Shots=5, Base Model=Pythia1B2025.06 | 0.475 | |
| TokAlign (LLaMA2)Shots=0, Base Model=Pythia1B2025.06 | 0.4703 | |
| TokAlign (LLaMA2)Shots=5, Base Model=Pythia1B2025.06 | 0.4643 |