Logical Reasoning on LogiQA (TSR, ASR)
48.75Task Success Rate (TSR)Qwen-Plus
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Qwen-PlusAttack Scenario=Independent2026.05 | 48.75 | 47 | |
| Qwen-PlusAttack Scenario=Group Collusive2026.05 | 45 | 51.25 | |
| Qwen-PlusAttack Scenario=Cooperative2026.05 | 43 | 53.25 | |
| GPT-5-NanoAttack Scenario=Independent2026.05 | 18.5 | 78.5 | |
| GPT-5-NanoAttack Scenario=Group Collusive2026.05 | 11.25 | 86.75 | |
| GPT-5-NanoAttack Scenario=Cooperative2026.05 | 6.5 | 91.25 | |
| GPT-3.5-TurboAttack Scenario=Group Collusive2026.05 | 5.5 | 93.75 | |
| GPT-3.5-TurboAttack Scenario=Cooperative2026.05 | 5 | 94.75 | |
| GPT-3.5-TurboAttack Scenario=Independent2026.05 | 4.75 | 94.5 |