Nonmonotonic reasoning on MultiLogicNMR
100Skeptical AccuracyGemini 2.5 Pro+ASP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemini 2.5 Pro+ASPInference Pipeline=LLM+ASP, Prompt Reference Type=Compact Reference2026.04 | 100 | 93.8 | |
| Gemini 2.5 Flash+ASPInference Pipeline=LLM+ASP, Prompt Reference Type=Compact Reference2026.04 | 98.3 | 98.3 | |
| o4-mini+ASPInference Pipeline=LLM+ASP, Prompt Reference Type=Compact Reference2026.04 | 95.8 | 84 | |
| Gemini 2.5 ProInference Pipeline=Baseline2026.04 | 88.5 | 61 | |
| DS-R1-0528+ASPInference Pipeline=LLM+ASP, Prompt Reference Type=Compact Reference2026.04 | 83.8 | 73.5 | |
| DS-V3+ASPInference Pipeline=LLM+ASP, Prompt Reference Type=Compact Reference2026.04 | 74.3 | 55.3 | |
| o4-miniInference Pipeline=Baseline2026.04 | 67.5 | 69.7 | |
| DS-V3Inference Pipeline=Baseline2026.04 | 57.7 | 51 | |
| Gemini 2.5 FlashInference Pipeline=Baseline2026.04 | 42.3 | 36.2 | |
| DS-R1-0528Inference Pipeline=Baseline2026.04 | 39.3 | 46.8 |