Logical Reasoning on LogiQA-2
83.8AccuracyGPT-5
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-5Evaluation Protocol=Closed-Source2026.01 | 83.8 | |
| Gemini2.5-ProEvaluation Protocol=Closed-Source2026.01 | 78.9 | |
| GPT-4.1Evaluation Protocol=Closed-Source2026.01 | 78.2 | |
| SE-GPTModel=GPT-42024.07 | 76.1 | |
| GPT-4oEvaluation Protocol=Closed-Source2026.01 | 72.9 | |
| Zero-shotModel=GPT-42024.07 | 70.6 | |
| Self-EXPModel=GPT-42024.07 | 69.8 | |
| Zero-shot-CoTModel=GPT-42024.07 | 68.6 | |
| ATLAS (cluster)Evaluation Protocol=In-Distribution2026.01 | 66.8 | |
| Self-ICLModel=GPT-42024.07 | 65.1 | |
| ATLAS (RL)Evaluation Protocol=Out-of-Distribution2026.01 | 62.7 | |
| Self-ICL-CoTModel=GPT-42024.07 | 60.7 | |
| Beam Search (BL)Model=Gemma-3-12b-it2026.04 | 60.1 | |
| TTPOModel=Gemma-3-12b-it2026.04 | 59.5 | |
| TTPOModel=Phi-4-mini-instruct (4B)2026.04 | 59.4 | |
| TTPOModel=Phi-4-mini-instruct (4B)2026.04 | 59.4 | |
| SteeringModel=Gemma-3-12b-it2026.04 | 59.4 | |
| Greedy (BL)Model=Gemma-3-12b-it2026.04 | 59.2 | |
| Beam SearchModel=Phi-4-mini-instruct (4B), beam size=52026.04 | 59 | |
| Beam Search (BL)Model=Phi-4-mini-instruct (4B)2026.04 | 59 | |
| Self-ConsistencyModel=Phi-4-mini-instruct (4B), n=52026.04 | 58.8 | |
| BranchingModel=Gemma-3-12b-it2026.04 | 58.5 | |
| SteeringModel=Phi-4-mini-instruct (4B)2026.04 | 58.3 | |
| GreedyModel=Phi-4-mini-instruct (4B)2026.04 | 57.7 | |
| Greedy (BL)Model=Phi-4-mini-instruct (4B)2026.04 | 57.7 | |
| BranchingModel=Phi-4-mini-instruct (4B)2026.04 | 57 | |
| BranchingModel=Gemma-3-4b-it2026.04 | 56.8 | |
| BranchingModel=Gemma-3-4b-it2026.04 | 56.8 | |
| TTPOModel=Gemma-3-4b-it2026.04 | 56 | |
| SE-GPTModel=GPT-3.52024.07 | 55.6 | |
| SteeringModel=Gemma-3-4b-it2026.04 | 55.5 | |
| Self-ConsistencyModel=Gemma-3-4b-it, n=52026.04 | 55.3 | |
| GreedyModel=Gemma-3-4b-it2026.04 | 55.2 | |
| Greedy (BL)Model=Gemma-3-4b-it2026.04 | 55.2 | |
| Beam SearchModel=Gemma-3-4b-it, beam size=52026.04 | 54 | |
| Beam Search (BL)Model=Gemma-3-4b-it2026.04 | 54 | |
| Zero-shot-CoTModel=GPT-3.52024.07 | 52.2 | |
| DiversityIteration=2, Selection Ratio=0.2, Model Scale=14b2026.04 | 52.04 | |
| Attr-DivIteration=2, Selection Ratio=0.2, Model Scale=14b2026.04 | 51.84 | |
| EVOSELECTIteration=2, Selection Ratio=0.2, Model Scale=14b2026.04 | 51.84 | |
| Zero-shotModel=GPT-3.52024.07 | 51.6 | |
| DiversityIteration=1, Selection Ratio=0.2, Model Scale=14b2026.04 | 51.59 | |
| Attr-DivIteration=1, Selection Ratio=0.2, Model Scale=14b2026.04 | 51.53 | |
| Self-EXPModel=GPT-3.52024.07 | 51.5 | |
| DiversityIteration=1, Selection Ratio=0.5, Model Scale=14b2026.04 | 51.4 | |
| RandomIteration=2, Selection Ratio=0.2, Model Scale=14b2026.04 | 51.15 | |
| TSDSIteration=1, Selection Ratio=0.2, Model Scale=14b2026.04 | 51.02 | |
| EVOSELECTIteration=1, Selection Ratio=0.5, Model Scale=14b2026.04 | 51.02 | |
| Modified Self-ICLModel=GPT-3.52024.07 | 51 | |
| EVOSELECTIteration=1, Selection Ratio=0.2, Model Scale=14b2026.04 | 50.95 | |
| TSDSIteration=2, Selection Ratio=0.2, Model Scale=14b2026.04 | 50.7 | |
| Attr-DivIteration=1, Selection Ratio=0.5, Model Scale=14b2026.04 | 50.64 | |
| RouterDCEvaluation Protocol=Out-of-Distribution2026.01 | 50.4 | |
| AttributionIteration=1, Selection Ratio=0.2, Model Scale=14b2026.04 | 50.38 | |
| AttributionIteration=1, Selection Ratio=0.5, Model Scale=14b2026.04 | 50.38 | |
| AllIteration=2, Model Scale=14b2026.04 | 50.32 | |
| AttributionIteration=2, Selection Ratio=0.2, Model Scale=14b2026.04 | 50.32 | |
| RandomIteration=1, Selection Ratio=0.5, Model Scale=14b2026.04 | 50.19 | |
| AttributionIteration=2, Selection Ratio=0.5, Model Scale=14b2026.04 | 50.13 | |
| EVOSELECTIteration=2, Selection Ratio=0.5, Model Scale=14b2026.04 | 50.13 | |
| RandomIteration=1, Selection Ratio=0.2, Model Scale=14b2026.04 | 50 | |
| DiversityIteration=2, Selection Ratio=0.5, Model Scale=14b2026.04 | 50 | |
| TSDSIteration=2, Selection Ratio=0.5, Model Scale=14b2026.04 | 50 | |
| RandomIteration=2, Selection Ratio=0.5, Model Scale=14b2026.04 | 49.87 | |
| Beam Search (BL)Model=Phi-4-reasoning-plus (13B)2026.04 | 49.8 | |
| AllIteration=1, Model Scale=14b2026.04 | 49.75 | |
| TSDSIteration=1, Selection Ratio=0.5, Model Scale=14b2026.04 | 49.43 | |
| Random RouterEvaluation Protocol=Training-free2026.01 | 49.2 | |
| BaseModel Scale=14b2026.04 | 48.98 | |
| Attr-DivIteration=2, Selection Ratio=0.5, Model Scale=14b2026.04 | 48.47 | |
| Greedy (BL)Model=Phi-4-reasoning-plus (13B)2026.04 | 47.8 | |
| RouterDCEvaluation Protocol=In-Distribution2026.01 | 47.2 | |
| BertRouterEvaluation Protocol=In-Distribution2026.01 | 47.1 | |
| SteeringModel=Phi-4-reasoning-plus (13B)2026.04 | 47 | |
| TTPOModel=Phi-4-reasoning-plus (13B)2026.04 | 45.7 | |
| ATLAS (cluster)Evaluation Protocol=Out-of-Distribution2026.01 | 45.6 | |
| ZS RouterEvaluation Protocol=Training-free, Prompting Strategy=Zero-shot2026.01 | 45.3 | |
| BranchingModel=Phi-4-reasoning-plus (13B)2026.04 | 43.6 | |
| AutoP-ICLModel=GPT-3.52024.07 | 42.2 | |
| EVOSELECTIteration=2, Selection Ratio=0.2, Model Scale=3b2026.04 | 41.98 | |
| AttributionIteration=1, Selection Ratio=0.5, Model Scale=3b2026.04 | 41.79 | |
| EVOSELECTIteration=1, Selection Ratio=0.2, Model Scale=3b2026.04 | 41.48 | |
| RandomIteration=1, Selection Ratio=0.2, Model Scale=3b2026.04 | 41.22 | |
| MLPRouterEvaluation Protocol=In-Distribution2026.01 | 41.2 | |
| EVOSELECTIteration=2, Selection Ratio=0.5, Model Scale=3b2026.04 | 41.16 | |
| EVOSELECTIteration=1, Selection Ratio=0.5, Model Scale=3b2026.04 | 41.09 | |
| RandomIteration=2, Selection Ratio=0.2, Model Scale=3b2026.04 | 41.09 | |
| AttributionIteration=2, Selection Ratio=0.5, Model Scale=3b2026.04 | 41.03 | |
| RandomIteration=1, Selection Ratio=0.5, Model Scale=3b2026.04 | 40.84 | |
| DiversityIteration=2, Selection Ratio=0.5, Model Scale=3b2026.04 | 40.84 | |
| FS RouterEvaluation Protocol=Training-free, Prompting Strategy=Few-shot2026.01 | 40.8 | |
| DiversityIteration=2, Selection Ratio=0.2, Model Scale=3b2026.04 | 40.65 | |
| BertRouterEvaluation Protocol=Out-of-Distribution2026.01 | 40.3 | |
| TSDSIteration=1, Selection Ratio=0.2, Model Scale=3b2026.04 | 40.27 | |
| TSDSIteration=1, Selection Ratio=0.5, Model Scale=3b2026.04 | 40.2 | |
| DiversityIteration=1, Selection Ratio=0.5, Model Scale=3b2026.04 | 40.01 | |
| Attr-DivIteration=2, Selection Ratio=0.2, Model Scale=3b2026.04 | 40.01 | |
| AllIteration=1, Model Scale=3b2026.04 | 39.95 | |
| RandomIteration=2, Selection Ratio=0.5, Model Scale=3b2026.04 | 39.95 | |
| TSDSIteration=2, Selection Ratio=0.5, Model Scale=3b2026.04 | 39.95 |