Classification on JAILBREAK (test)
98.8AccuracyACT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ACTTree Depth (d)=4, Optimization Steps (k)=20, LLM Backbone=GPT-4.1 Mini2025.09 | 98.8 | 98.8 | |
| Rule FitLLM Backbone=Avg2025.09 | 96.8 | 96.8 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=GPT-4.1 Mini2025.09 | 95.2 | 95.5 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=GPT-4.1 Mini2025.09 | 94.8 | 94.8 | |
| TEXTGRADLLM Backbone=GPT-4.1 Mini2025.09 | 94.8 | 95.1 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=GPT-4.1 Mini2025.09 | 93.6 | 93.3 | |
| ACTTree Depth (d)=4, Optimization Steps (k)=20, LLM Backbone=Avg2025.09 | 93.4 | 93.3 | |
| TF-IDF + CARTTree Depth (d)=4, LLM Backbone=Avg2025.09 | 92.8 | 92.4 | |
| ACTTree Depth (d)=4, Optimization Steps (k)=20, LLM Backbone=GPT-4.1 Nano2025.09 | 92 | 91.5 | |
| ACTTree Depth (d)=4, Optimization Steps (k)=20, LLM Backbone=Qwen3 4b2025.09 | 92 | 92.7 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=GPT-4.1 Nano2025.09 | 91.6 | 91.6 | |
| TEXTGRADLLM Backbone=Gemma3 4b2025.09 | 91.6 | 91.3 | |
| TEXTGRADLLM Backbone=Avg2025.09 | 91.4 | 91.7 | |
| ACTTree Depth (d)=4, Optimization Steps (k)=20, LLM Backbone=Gemma3 4b2025.09 | 90.8 | 90 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=Avg2025.09 | 90.4 | 89.9 | |
| TEXTGRADLLM Backbone=GPT-4.1 Nano2025.09 | 90.4 | 90.3 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=Gemma3 4b2025.09 | 89.6 | 89.3 | |
| TEXTGRADLLM Backbone=Qwen3 4b2025.09 | 88.8 | 90 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=Avg2025.09 | 86.4 | 86.5 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=GPT-4.1 Nano2025.09 | 85.5 | 83.5 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=Qwen3 4b2025.09 | 85.5 | 83.9 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=GPT-4.1 Nano2025.09 | 85.5 | 85.1 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=Avg2025.09 | 84.5 | 84.2 | |
| Con. tag.LLM Backbone=Gemma3 4b2025.09 | 82.7 | 82.2 | |
| Con. tag.LLM Backbone=GPT-4.1 Mini2025.09 | 82.7 | 79.8 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=Qwen3 4b2025.09 | 82.7 | 82.6 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=Gemma3 4b2025.09 | 82.3 | 82.7 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=Qwen3 4b2025.09 | 81.1 | 77.7 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=Gemma3 4b2025.09 | 77.9 | 82.2 | |
| Con. tag.LLM Backbone=Qwen3 4b2025.09 | 76.3 | 70.1 | |
| Con. tag.LLM Backbone=Avg2025.09 | 75.1 | 66 | |
| Con. tag.LLM Backbone=GPT-4.1 Nano2025.09 | 58.6 | 31.8 |