Logical Reasoning on BOARD (BOARDGAMEQA)
96.3AccuracyGemini 2.5 Pro+ASP
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemini 2.5 Pro+ASPInference Pipeline=LLM+ASP, Prompt Reference Type=Compact Reference2026.04 | 96.3 | |
| Gemini 2.5 Flash+ASPInference Pipeline=LLM+ASP, Prompt Reference Type=Compact Reference2026.04 | 94.7 | |
| Gemini 2.5 FlashInference Pipeline=Baseline2026.04 | 93 | |
| DS-R1-0528+ASPInference Pipeline=LLM+ASP, Prompt Reference Type=Compact Reference2026.04 | 91.7 | |
| Gemini 2.5 ProInference Pipeline=Baseline2026.04 | 89.8 | |
| o4-miniInference Pipeline=Baseline2026.04 | 88.2 | |
| o4-mini+ASPInference Pipeline=LLM+ASP, Prompt Reference Type=Compact Reference2026.04 | 85.2 | |
| DS-R1-0528Inference Pipeline=Baseline2026.04 | 81 | |
| SATLMLanguage Model=code-davinci-002, Decoding Method=self-consistency decoding2023.05 | 80.7 | |
| SATLMLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.05 | 79.4 | |
| DS-V3Inference Pipeline=Baseline2026.04 | 71.3 | |
| COTLanguage Model=code-davinci-002, Decoding Method=self-consistency decoding2023.05 | 62.8 | |
| COTLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.05 | 60.7 | |
| STANDARDLanguage Model=code-davinci-002, Decoding Method=greedy decoding2023.05 | 44.6 | |
| DS-V3+ASPInference Pipeline=LLM+ASP, Prompt Reference Type=Compact Reference2026.04 | 44 |