Permission-boundary inference on AuthBench Standard Task Set
83.3Task Success Rate (TSR)Full-Access
Evaluation Results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Full-Access2026.05 | 83.3 | — | — | — | — | — | — | — | — | — | |
| Golden-Perm.2026.05 | 77.1 | — | — | — | — | — | — | — | — | — | |
| Gemini 3.1 Pro2026.05 | 75.4 | 76.7 | 90.3 | 78 | 84.4 | 91.8 | 85.3 | 47.9 | 79.5 | 49 | |
| GPT-52026.05 | 63.3 | 88.4 | 83.4 | 83.3 | 89.2 | 88.4 | 86.8 | 67 | 50.6 | 53.7 | |
| Claude Opus 4.62026.05 | 61.3 | 89.1 | 85.6 | 84.7 | 89.6 | 88.4 | 87.4 | 61.4 | 55.2 | 53 | |
| Kimi K2.52026.05 | 60 | 80.3 | 74.3 | 74.1 | 88.3 | 85.3 | 84.5 | 67.9 | 53 | 54.4 | |
| GPT-5.3-Codex2026.05 | 58.8 | 82.2 | 74.7 | 75.8 | 78.8 | 85.3 | 78.7 | 61.8 | 42.7 | 46.7 | |
| GPT-5.42026.05 | 52.6 | 78.8 | 74.9 | 73.7 | 87.6 | 84.2 | 83.2 | 68.9 | 47.9 | 53.3 | |
| Qwen3-Coder2026.05 | 52.1 | 73.4 | 65.1 | 64.1 | 83.6 | 73 | 73.1 | 65.2 | 45.2 | 49.1 | |
| Qwen3.5-397B2026.05 | 45.6 | 78.1 | 75.7 | 71.8 | 88.7 | 85.2 | 83.6 | 74.7 | 55.1 | 59.6 | |
| MiniMax M2.72026.05 | 42.1 | 75.5 | 67.6 | 66.7 | 85.6 | 79 | 77.6 | 71.8 | 49.8 | 54.6 |