Logical Reasoning Oversight on MALLS
0.967AUCPipeline 1
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Pipeline 1Model=Gemma, Strategy=B3, Variant=pv32026.06 | 0.967 | 0.096 | 0.287 | |
| Pipeline 1Model=Gemma, Strategy=B1, Variant=pv32026.06 | 0.964 | 0.135 | 0.305 | |
| Pipeline 2Model=Gemma, Strategy=B1, Variant=pv42026.06 | 0.946 | 0.122 | 0.469 | |
| Pipeline 2Model=Gemma, Strategy=B3, Variant=pv42026.06 | 0.945 | 0.125 | 0.469 | |
| Pipeline 1Model=Qwen, Strategy=B4, Variant=pv42026.06 | 0.933 | 0.313 | 0.417 | |
| Pipeline 1Model=Qwen, Strategy=B1, Variant=pv32026.06 | 0.932 | 0.307 | 0.398 | |
| GreenModel=Gemma2026.06 | 0.92 | 0.375 | 0.688 | |
| Pipeline 2Model=Qwen, Strategy=B1, Variant=pv22026.06 | 0.916 | 0.313 | 0.633 | |
| Pipeline 1Model=GPT-4o-mini, Strategy=B3, Variant=pv22026.06 | 0.911 | 0.367 | 0.447 | |
| Pipeline 1Model=GPT-4o-mini, Strategy=B1, Variant=pv12026.06 | 0.91 | 0.376 | 0.462 | |
| Pipeline 2Model=Qwen, Strategy=B3, Variant=pv22026.06 | 0.91 | 0.392 | 0.696 | |
| Pipeline 2Model=GPT-4o-mini, Strategy=B3, Variant=pv22026.06 | 0.83 | 0.653 | 0.827 | |
| Pipeline 2Model=GPT-4o-mini, Strategy=B2, Variant=pv22026.06 | 0.827 | 0.663 | 0.831 | |
| GreenModel=Qwen2026.06 | 0.825 | 0.714 | 0.857 | |
| BlackModel=Gemma2026.06 | 0.82 | 0.722 | 0.861 | |
| BlackModel=GPT-4o-mini2026.06 | 0.82 | 0.722 | 0.861 | |
| BlackModel=Qwen2026.06 | 0.82 | 0.722 | 0.861 | |
| GreenModel=GPT-4o-mini2026.06 | 0.79 | 0.762 | 0.881 |