Classification on SPAM (test)
99.6AccuracyACT
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| ACTModel=Nano, d=4, k=102025.09 | 99.6 | 99.6 | |
| ACTModel=Nano, d=2, k=102025.09 | 99.5 | 99.5 | |
| ACTModel=Nano, d=3, k=102025.09 | 99.5 | 99.5 | |
| ACTModel=Nano, d=5, k=102025.09 | 99.5 | 99.5 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=GPT-4.1 Mini2025.09 | 99.5 | 99.5 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=GPT-4.1 Nano2025.09 | 99.2 | 99.2 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=Avg2025.09 | 99 | 99 | |
| ACTModel=Gemma, d=5, k=102025.09 | 98.8 | 98.8 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=Qwen3 4b2025.09 | 98.8 | 98.8 | |
| ACTTree Depth (d)=2, Optimization Steps (k)=5, LLM Backbone=GPT-4.1 Nano2025.09 | 98.7 | 98.6 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=Gemma3 4b2025.09 | 98.5 | 98.5 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=GPT-4.1 Nano2025.09 | 98.3 | 98.3 | |
| TextGradModel=Nano2025.09 | 98.2 | 98.2 | |
| DSPyModel=Nano2025.09 | 98.2 | 98.2 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=GPT-4.1 Mini2025.09 | 98.2 | 98.2 | |
| ACTTree Depth (d)=2, Optimization Steps (k)=5, LLM Backbone=GPT-4.1 Mini2025.09 | 98.2 | 98.2 | |
| ACTModel=Gemma, d=4, k=102025.09 | 97.9 | 98 | |
| ACTModel=Gemma, d=3, k=102025.09 | 97.6 | 97.6 | |
| DSPyModel=Gemma2025.09 | 97.4 | 97.5 | |
| ACTTree Depth (d)=2, Optimization Steps (k)=5, LLM Backbone=Avg2025.09 | 97.4 | 97.3 | |
| TEXTGRADLLM Backbone=GPT-4.1 Nano2025.09 | 97.3 | 97.3 | |
| TextGradModel=Gemma2025.09 | 97.2 | 97.2 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=Qwen3 4b2025.09 | 97.2 | 97.2 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=Avg2025.09 | 97.2 | 97.2 | |
| Rule FitLLM Backbone=Avg2025.09 | 97 | 96.9 | |
| TEXTGRADLLM Backbone=GPT-4.1 Mini2025.09 | 96.7 | 96.6 | |
| ACTTree Depth (d)=2, Optimization Steps (k)=5, LLM Backbone=Qwen3 4b2025.09 | 96.7 | 96.6 | |
| ACTModel=Gemma, d=2, k=102025.09 | 96.3 | 96.4 | |
| TEXTGRADLLM Backbone=Avg2025.09 | 96.3 | 96.2 | |
| TEXTGRADLLM Backbone=Qwen3 4b2025.09 | 96 | 96 | |
| ACTTree Depth (d)=2, Optimization Steps (k)=5, LLM Backbone=Gemma3 4b2025.09 | 95.8 | 95.8 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=GPT-4.1 Mini2025.09 | 95.7 | 95.5 | |
| CoTModel=Nano2025.09 | 95.6 | 95.5 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=GPT-4.1 Nano2025.09 | 95.5 | 95.3 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=Gemma3 4b2025.09 | 95 | 95.2 | |
| TEXTGRADLLM Backbone=Gemma3 4b2025.09 | 95 | 94.7 | |
| TF-IDF + CARTd=102025.09 | 94.9 | 95.2 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=Qwen3 4b2025.09 | 93.2 | 93.6 | |
| TF-IDF + CARTTree Depth (d)=5, LLM Backbone=Avg2025.09 | 92.7 | 93.1 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=Avg2025.09 | 89.4 | 90.8 | |
| Con. tag.LLM Backbone=Gemma3 4b2025.09 | 83.2 | 85.5 | |
| Con. tag.LLM Backbone=GPT-4.1 Mini2025.09 | 82.3 | 85 | |
| CoTModel=Gemma2025.09 | 75 | 80.1 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=Gemma3 4b2025.09 | 73.3 | 78.6 | |
| Con. tag.LLM Backbone=Avg2025.09 | 73 | 64 | |
| Con. tag.LLM Backbone=GPT-4.1 Nano2025.09 | 64.2 | 45.8 | |
| Con. tag.LLM Backbone=Qwen3 4b2025.09 | 62.3 | 39.6 |