Logical Reasoning on RuleBERT
67.7AccuracyBeam Search
Evaluation Results
| Method | Links | |
|---|---|---|
| Beam SearchModel=Gemma-3-4b-it, beam size=52026.04 | 67.7 | |
| Beam Search (BL)Model=Gemma-3-4b-it2026.04 | 67.7 | |
| Self-ConsistencyModel=Gemma-3-4b-it, n=52026.04 | 63.9 | |
| BranchingModel=Gemma-3-4b-it2026.04 | 63.4 | |
| BranchingModel=Gemma-3-4b-it2026.04 | 63.4 | |
| SteeringModel=Gemma-3-4b-it2026.04 | 62 | |
| TTPOModel=Gemma-3-4b-it2026.04 | 60.8 | |
| GreedyModel=Gemma-3-4b-it2026.04 | 60.3 | |
| Greedy (BL)Model=Gemma-3-4b-it2026.04 | 60.3 | |
| SteeringModel=Gemma-3-12b-it2026.04 | 59 | |
| Greedy (BL)Model=Gemma-3-12b-it2026.04 | 58.2 | |
| Beam Search (BL)Model=Gemma-3-12b-it2026.04 | 57.5 | |
| TTPOModel=Gemma-3-12b-it2026.04 | 57.3 | |
| BranchingModel=Gemma-3-12b-it2026.04 | 57 | |
| BranchingModel=Phi-4-mini-instruct (4B)2026.04 | 51.9 | |
| BranchingModel=Phi-4-mini-instruct (4B)2026.04 | 51.9 | |
| Beam SearchModel=Phi-4-mini-instruct (4B), beam size=52026.04 | 51.7 | |
| Beam Search (BL)Model=Phi-4-mini-instruct (4B)2026.04 | 51.7 | |
| SteeringModel=Phi-4-mini-instruct (4B)2026.04 | 50.8 | |
| TTPOModel=Phi-4-mini-instruct (4B)2026.04 | 50.7 | |
| GreedyModel=Phi-4-mini-instruct (4B)2026.04 | 50.3 | |
| Greedy (BL)Model=Phi-4-mini-instruct (4B)2026.04 | 50.3 | |
| Self-ConsistencyModel=Phi-4-mini-instruct (4B), n=52026.04 | 48.8 | |
| Beam Search (BL)Model=Phi-4-reasoning-plus (13B)2026.04 | 47.4 | |
| TTPOModel=Phi-4-reasoning-plus (13B)2026.04 | 43.7 | |
| BranchingModel=Phi-4-reasoning-plus (13B)2026.04 | 43.2 | |
| Greedy (BL)Model=Phi-4-reasoning-plus (13B)2026.04 | 41.6 | |
| SteeringModel=Phi-4-reasoning-plus (13B)2026.04 | 41.4 |