A1 shortcut attack defense on Real-LLM 8 targets × 3 phrasings full stack late 2025/2026 (val)
0Attack-Induced ISRMockAgent (worst-case bound)
Evaluation results
| Method | Links | |
|---|---|---|
| MockAgent (worst-case bound)ContractGuard level=L3, Temperature=02026.06 | 0 | |
| Claude Opus 4.8ContractGuard level=L3, Temperature=02026.06 | 0 | |
| Claude Sonnet 4.6ContractGuard level=L3, Temperature=02026.06 | 0 | |
| Claude Haiku 4.5ContractGuard level=L3, Temperature=02026.06 | 0 | |
| GPT-OSS-120BContractGuard level=L3, Temperature=02026.06 | 0 | |
| Amazon Nova PremierContractGuard level=L3, Temperature=02026.06 | 0 | |
| Amazon Nova 2 LiteContractGuard level=L3, Temperature=02026.06 | 0 | |
| MockAgent (worst-case bound)ContractGuard level=L0, Temperature=02026.06 | 100 | |
| Claude Opus 4.8ContractGuard level=L0, Temperature=02026.06 | 100 | |
| Claude Sonnet 4.6ContractGuard level=L0, Temperature=02026.06 | 100 | |
| Claude Haiku 4.5ContractGuard level=L0, Temperature=02026.06 | 100 | |
| GPT-OSS-120BContractGuard level=L0, Temperature=02026.06 | 100 | |
| Amazon Nova PremierContractGuard level=L0, Temperature=02026.06 | 100 | |
| Amazon Nova 2 LiteContractGuard level=L0, Temperature=02026.06 | 100 |