Analogical Reasoning on MCQ
46AccuracySM
Evaluation Results
| Method | Links | |
|---|---|---|
| SMModel (Backbone)=Qwen3-8B, Conceptual Abstraction=true, Evaluative Abstraction=true, Stage Abstraction=false, Narrative Arc Abstraction=false2026.03 | 46 | |
| SMModel (Backbone)=Llama3.1-8B, Conceptual Abstraction=true, Evaluative Abstraction=true, Stage Abstraction=false, Narrative Arc Abstraction=false2026.03 | 45 | |
| SMModel (Backbone)=Llama3.1-8B, Conceptual Abstraction=true, Evaluative Abstraction=false, Stage Abstraction=false, Narrative Arc Abstraction=false2026.03 | 42 | |
| LLM-CoTModel (Backbone)=Qwen3-8B2026.03 | 41 | |
| SMModel (Backbone)=Qwen3-8B, Conceptual Abstraction=true, Evaluative Abstraction=false, Stage Abstraction=false, Narrative Arc Abstraction=false2026.03 | 41 | |
| LLM-ZSModel (Backbone)=Qwen3-8B2026.03 | 37 | |
| SMModel (Backbone)=Qwen3-8B, Conceptual Abstraction=false, Evaluative Abstraction=false, Stage Abstraction=true, Narrative Arc Abstraction=true2026.03 | 30 | |
| SMModel (Backbone)=Qwen3-8B, Conceptual Abstraction=false, Evaluative Abstraction=false, Stage Abstraction=true, Narrative Arc Abstraction=false2026.03 | 29 | |
| LLM-CoTModel (Backbone)=Llama3.1-8B2026.03 | 28 | |
| SMModel (Backbone)=Llama3.1-8B, Conceptual Abstraction=false, Evaluative Abstraction=false, Stage Abstraction=true, Narrative Arc Abstraction=true2026.03 | 27 | |
| SMModel (Backbone)=Llama3.1-8B, Conceptual Abstraction=false, Evaluative Abstraction=false, Stage Abstraction=true, Narrative Arc Abstraction=false2026.03 | 24 | |
| LLM-ZSModel (Backbone)=Llama3.1-8B2026.03 | 23 | |
| SMModel (Backbone)=Qwen3-8B, Conceptual Abstraction=false, Evaluative Abstraction=false, Stage Abstraction=false, Narrative Arc Abstraction=false2026.03 | 17 | |
| SMModel (Backbone)=Llama3.1-8B, Conceptual Abstraction=false, Evaluative Abstraction=false, Stage Abstraction=false, Narrative Arc Abstraction=false2026.03 | 16 |