Logical Reasoning Oversight on FOLIO
95.3AUCPipeline 1
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Pipeline 1Model=Gemma, Strategy=B2, Variant=pv32026.06 | 95.3 | 0.218 | 0.375 | |
| Pipeline 1Model=Gemma, Strategy=B3, Variant=pv22026.06 | 95.2 | 0.225 | 0.382 | |
| Pipeline 2Model=Gemma, Strategy=B2, Variant=pv22026.06 | 92.2 | 0.312 | 0.656 | |
| Pipeline 2Model=Gemma, Strategy=B1, Variant=pv22026.06 | 92.2 | 0.306 | 0.653 | |
| Pipeline 1Model=GPT-4o-mini, Strategy=B3, Variant=pv12026.06 | 87.2 | 0.465 | 0.548 | |
| Pipeline 2Model=GPT-4o-mini, Strategy=B3, Variant=pv12026.06 | 86.8 | 0.439 | 0.719 | |
| Pipeline 2Model=Qwen, Strategy=B4, Variant=pv42026.06 | 86.7 | 0.511 | 0.755 | |
| GreenModel=Gemma2026.06 | 86.4 | 0.634 | 0.817 | |
| Pipeline 2Model=GPT-4o-mini, Strategy=B3, Variant=pv22026.06 | 86.4 | 0.485 | 0.743 | |
| Pipeline 1Model=Qwen, Strategy=B1, Variant=pv32026.06 | 86.1 | 0.461 | 0.719 | |
| Pipeline 2Model=Qwen, Strategy=B1, Variant=pv32026.06 | 86.1 | 0.55 | 0.775 | |
| Pipeline 1Model=GPT-4o-mini, Strategy=B1, Variant=pv12026.06 | 86 | 0.478 | 0.683 | |
| Pipeline 1Model=Qwen, Strategy=B1, Variant=pv12026.06 | 85.4 | 0.506 | 0.753 | |
| GreenModel=GPT-4o-mini2026.06 | 80.6 | 0.743 | 0.872 | |
| BlackModel=Gemma2026.06 | 80.5 | 0.744 | 0.872 | |
| BlackModel=GPT-4o-mini2026.06 | 80.5 | 0.744 | 0.872 | |
| BlackModel=Qwen2026.06 | 80.5 | 0.744 | 0.872 | |
| GreenModel=Qwen2026.06 | 79.3 | 0.759 | 0.88 |