Exfiltration on ACIARENA Code
51.11UALLM Debate
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LLM DebateBase LLM=GPT-4o-mini2026.04 | 51.11 | 80.22 | |
| Self ConsistencyBase LLM=GPT-4o-mini2026.04 | 47.78 | 80 | |
| AgentVerseBase LLM=GPT-4o-mini2026.04 | 47.33 | 80.45 | |
| AutoGen + ACI-SENTINELMAS Platform=AutoGen, Defense=ACI-SENTINEL2026.04 | 45.11 | 0.22 | |
| AutoGenBase LLM=GPT-4o-mini2026.04 | 44.89 | 77.55 | |
| AgentVerse + ACI-SENTINELMAS Platform=AgentVerse, Defense=ACI-SENTINEL2026.04 | 43.11 | 9.11 | |
| MetaGPT + SANDWICHMAS Platform=MetaGPT, Defense=SANDWICH2026.04 | 41.78 | 71.56 | |
| AgentVerse + SANDWICHMAS Platform=AgentVerse, Defense=SANDWICH2026.04 | 40.89 | 58 | |
| AgentVerse + BERT DETECTORMAS Platform=AgentVerse, Defense=BERT DETECTOR2026.04 | 40.67 | 61.11 | |
| AgentVerseMAS Platform=AgentVerse, Defense=None2026.04 | 40 | 61.56 | |
| AutoGen + SANDWICHMAS Platform=AutoGen, Defense=SANDWICH2026.04 | 39.56 | 60 | |
| MetaGPTBase LLM=GPT-4o-mini2026.04 | 38.89 | 80.22 | |
| AgentVerse + AGRAILMAS Platform=AgentVerse, Defense=AGRAIL2026.04 | 37.78 | 67.33 | |
| MetaGPTMAS Platform=MetaGPT, Defense=None2026.04 | 34.67 | 60.44 | |
| AgentVerse + G-SAFEGUARDMAS Platform=AgentVerse, Defense=G-SAFEGUARD2026.04 | 33.33 | 56.22 | |
| AutoGenMAS Platform=AutoGen, Defense=None2026.04 | 32.22 | 54 | |
| MetaGPT + ACI-SENTINELMAS Platform=MetaGPT, Defense=ACI-SENTINEL2026.04 | 28.22 | 9.78 | |
| MetaGPT + G-SAFEGUARDMAS Platform=MetaGPT, Defense=G-SAFEGUARD2026.04 | 28 | 42.89 | |
| MetaGPT + DELIMITERMAS Platform=MetaGPT, Defense=DELIMITER2026.04 | 24.89 | 59.78 | |
| AgentVerse + DELIMITERMAS Platform=AgentVerse, Defense=DELIMITER2026.04 | 24.22 | 59.56 | |
| AutoGen + BERT DETECTORMAS Platform=AutoGen, Defense=BERT DETECTOR2026.04 | 22.67 | 36.67 | |
| AutoGen + G-SAFEGUARDMAS Platform=AutoGen, Defense=G-SAFEGUARD2026.04 | 21.33 | 34 | |
| AutoGen + DELIMITERMAS Platform=AutoGen, Defense=DELIMITER2026.04 | 20.22 | 44.22 | |
| MetaGPT + BERT DETECTORMAS Platform=MetaGPT, Defense=BERT DETECTOR2026.04 | 20.22 | 60.67 | |
| CAMELBase LLM=GPT-4o-mini2026.04 | 16.22 | 26 | |
| AutoGen + AGRAILMAS Platform=AutoGen, Defense=AGRAIL2026.04 | 14 | 29.33 | |
| MetaGPT + AGRAILMAS Platform=MetaGPT, Defense=AGRAIL2026.04 | 5.56 | 19.78 |