Safety certification of tool-integrated LLMs on OpenAPI pool
72PU ScoreLLMCert-T
Evaluation results
| Method | Links | |
|---|---|---|
| LLMCert-TSpecification=Intent Shifting, Selector=Llama-3.1 8B, Generator=Gemma-3 4B, M=300, N=102025.10 | 72 | |
| LLMCert-TSpecification=Abstention Trigger, Selector=Llama-3.1 8B, Generator=Gemma-3 4B, M=300, N=102025.10 | 67 | |
| LLMCert-TSpecification=Privilege Escalation, Selector=Llama-3.1 8B, Generator=Gemma-3 4B, M=300, N=102025.10 | 50 | |
| LLMCert-TSpecification=Top-N Saturation, Selector=Llama-3.1 8B, Generator=Gemma-3 4B, M=300, N=102025.10 | 22 | |
| LLMCert-TSpecification=Distractor Selection, Selector=Llama-3.1 8B, Generator=Gemma-3 4B, M=300, N=102025.10 | 19 |