Logical Inference on LogiQA (TSR, ASR)
76.75Task Success Rate (TSR)G-Safeguard
Evaluation results
| Method | Links | ||
|---|---|---|---|
| G-SafeguardBackbone=Qwen-Plus, Attack Setting=Independent2026.05 | 76.75 | 16.5 | |
| STARBackbone=Qwen-Plus, Attack Setting=Independent2026.05 | 75 | 16.5 | |
| STARBackbone=Qwen-Plus, Attack Setting=Cooperative2026.05 | 73.5 | 16.75 | |
| G-SafeguardBackbone=Qwen-Plus, Attack Setting=Cooperative2026.05 | 73.25 | 19 | |
| ARGUSBackbone=Qwen-Plus, Attack Setting=Cooperative2026.05 | 64 | 29.75 | |
| ARGUSBackbone=Qwen-Plus, Attack Setting=Independent2026.05 | 62.25 | 32.5 | |
| Blind-GuardBackbone=Qwen-Plus, Attack Setting=Independent2026.05 | 57.75 | 38.25 | |
| STARBackbone=GPT-3.5-Turbo, Attack Setting=Cooperative2026.05 | 52.5 | 34.25 | |
| G-SafeguardBackbone=GPT-5-Nano, Attack Setting=Cooperative2026.05 | 52 | 38.5 | |
| Blind-GuardBackbone=Qwen-Plus, Attack Setting=Cooperative2026.05 | 51.75 | 43.75 | |
| STARBackbone=GPT-3.5-Turbo, Attack Setting=Independent2026.05 | 51 | 33.75 | |
| C&IBackbone=Qwen-Plus, Attack Setting=Independent2026.05 | 49.75 | 46.75 | |
| G-SafeguardBackbone=GPT-5-Nano, Attack Setting=Independent2026.05 | 47.25 | 44.25 | |
| G-SafeguardBackbone=GPT-3.5-Turbo, Attack Setting=Cooperative2026.05 | 46.75 | 45 | |
| G-SafeguardBackbone=GPT-3.5-Turbo, Attack Setting=Independent2026.05 | 46.25 | 44.75 | |
| C&IBackbone=Qwen-Plus, Attack Setting=Cooperative2026.05 | 46.25 | 49.5 | |
| STARBackbone=GPT-5-Nano, Attack Setting=Independent2026.05 | 45.25 | 42 | |
| STARBackbone=GPT-5-Nano, Attack Setting=Cooperative2026.05 | 41 | 44.5 | |
| Blind-GuardBackbone=GPT-5-Nano, Attack Setting=Independent2026.05 | 34.5 | 59.25 | |
| Blind-GuardBackbone=GPT-3.5-Turbo, Attack Setting=Independent2026.05 | 29.25 | 64.5 | |
| Blind-GuardBackbone=GPT-3.5-Turbo, Attack Setting=Cooperative2026.05 | 26.5 | 65.75 | |
| ARGUSBackbone=GPT-5-Nano, Attack Setting=Independent2026.05 | 25.25 | 69.5 | |
| C&IBackbone=GPT-3.5-Turbo, Attack Setting=Cooperative2026.05 | 25 | 68.5 | |
| C&IBackbone=GPT-3.5-Turbo, Attack Setting=Independent2026.05 | 24 | 71 | |
| C&IBackbone=GPT-5-Nano, Attack Setting=Independent2026.05 | 21.25 | 74.5 | |
| ARGUSBackbone=GPT-3.5-Turbo, Attack Setting=Cooperative2026.05 | 20 | 76.25 | |
| ARGUSBackbone=GPT-5-Nano, Attack Setting=Cooperative2026.05 | 16.25 | 79.75 | |
| ARGUSBackbone=GPT-3.5-Turbo, Attack Setting=Independent2026.05 | 15 | 81.25 | |
| Blind-GuardBackbone=GPT-5-Nano, Attack Setting=Cooperative2026.05 | 14.5 | 81.75 | |
| C&IBackbone=GPT-5-Nano, Attack Setting=Cooperative2026.05 | 11.5 | 86.25 |