Text Classification on IMDB (test) (LLM Performance)
96.8Average AccuracyDSPy
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=GPT-4.1 Mini2025.09 | 96.8 | — | — | — | — | — | — | — | — | 96.8 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=20, LLM Backbone=GPT-4.1 Mini2025.09 | 96.3 | — | — | — | — | — | — | — | — | 96.2 | |
| TEXTGRADLLM Backbone=GPT-4.1 Mini2025.09 | 95.8 | — | — | — | — | — | — | — | — | 95.8 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=GPT-4.1 Mini2025.09 | 95.7 | — | — | — | — | — | — | — | — | 95.6 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=GPT-4.1 Mini2025.09 | 95.7 | — | — | — | — | — | — | — | — | 95.7 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=GPT-4.1 Nano2025.09 | 95.5 | — | — | — | — | — | — | — | — | 95.5 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=20, LLM Backbone=Qwen3 4b2025.09 | 95.5 | — | — | — | — | — | — | — | — | 95.5 | |
| DSPysearch strategy=BFSR, number of demos=82025.09 | 95.2 | 93.7 | 93.6 | 95.5 | 95.5 | 96.8 | 96.8 | 94.8 | 94.8 | 95.2 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=Avg2025.09 | 95.2 | — | — | — | — | — | — | — | — | 95.2 | |
| TEXTGRADLLM Backbone=GPT-4.1 Nano2025.09 | 95.2 | — | — | — | — | — | — | — | — | 95.1 | |
| ACTtree depth (d)=3, optimization steps (k)=202025.09 | 95.1 | 93.7 | 93.6 | 94.7 | 94.7 | 96.3 | 96.2 | 95.5 | 95.5 | 95 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=20, LLM Backbone=Avg2025.09 | 95.1 | — | — | — | — | — | — | — | — | 95 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=Qwen3 4b2025.09 | 94.8 | — | — | — | — | — | — | — | — | 94.8 | |
| TEXTGRAD2025.09 | 94.7 | 93.2 | 92.9 | 95.2 | 95.1 | 95.8 | 95.8 | 94.7 | 94.5 | 94.6 | |
| TEXTGRADLLM Backbone=Qwen3 4b2025.09 | 94.7 | — | — | — | — | — | — | — | — | 94.5 | |
| TEXTGRADLLM Backbone=Avg2025.09 | 94.7 | — | — | — | — | — | — | — | — | 94.6 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=20, LLM Backbone=GPT-4.1 Nano2025.09 | 94.7 | — | — | — | — | — | — | — | — | 94.7 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=GPT-4.1 Nano2025.09 | 94.2 | — | — | — | — | — | — | — | — | 94.1 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=Qwen3 4b2025.09 | 94.2 | — | — | — | — | — | — | — | — | 94 | |
| CoTshot configuration=0-shot2025.09 | 94.1 | 92.3 | 91.9 | 94.2 | 94.1 | 95.7 | 95.6 | 94.2 | 94 | 93.9 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=Avg2025.09 | 94.1 | — | — | — | — | — | — | — | — | 93.9 | |
| ACTtree depth (d)=3, optimization steps (k)=102025.09 | 93.9 | 93.2 | 93.2 | 92.8 | 92.8 | 95.7 | 95.7 | 93.8 | 93.9 | 93.9 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=Avg2025.09 | 93.9 | — | — | — | — | — | — | — | — | 93.9 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=Qwen3 4b2025.09 | 93.8 | — | — | — | — | — | — | — | — | 93.9 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=Gemma3 4b2025.09 | 93.7 | — | — | — | — | — | — | — | — | 93.6 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=20, LLM Backbone=Gemma3 4b2025.09 | 93.7 | — | — | — | — | — | — | — | — | 93.6 | |
| TEXTGRADLLM Backbone=Gemma3 4b2025.09 | 93.2 | — | — | — | — | — | — | — | — | 92.9 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=Gemma3 4b2025.09 | 93.2 | — | — | — | — | — | — | — | — | 93.2 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=GPT-4.1 Nano2025.09 | 92.8 | — | — | — | — | — | — | — | — | 92.8 | |
| RoBERTa2025.09 | 92.7 | — | — | — | — | — | — | — | — | 92.7 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=Gemma3 4b2025.09 | 92.3 | — | — | — | — | — | — | — | — | 91.9 | |
| BERT2025.09 | 90.2 | — | — | — | — | — | — | — | — | 90.1 | |
| TF-IDF + XGBoosttree depth (d)=2, number of trees (nt)=2002025.09 | 76.2 | — | — | — | — | — | — | — | — | 76.7 | |
| Rule Fit2025.09 | 75.2 | — | — | — | — | — | — | — | — | 73.7 | |
| Rule FitLLM Backbone=Avg2025.09 | 75.2 | — | — | — | — | — | — | — | — | 73.7 | |
| Con. tag.LLM Backbone=Gemma3 4b2025.09 | 74.8 | — | — | — | — | — | — | — | — | 76.9 | |
| TF-IDF + CARTtree depth (d)=32025.09 | 66.8 | — | — | — | — | — | — | — | — | 72.8 | |
| TF-IDF + CARTTree Depth (d)=3, LLM Backbone=Avg2025.09 | 66.8 | — | — | — | — | — | — | — | — | 72.8 | |
| Con. tag.2025.09 | 58.1 | 74.8 | 76.9 | 56.2 | 31.7 | 50 | 66.7 | 51.3 | 63.8 | 59.8 | |
| Con. tag.LLM Backbone=Avg2025.09 | 58.1 | — | — | — | — | — | — | — | — | 59.8 | |
| Con. tag.LLM Backbone=GPT-4.1 Nano2025.09 | 56.2 | — | — | — | — | — | — | — | — | 31.7 | |
| Con. tag.LLM Backbone=Qwen3 4b2025.09 | 51.3 | — | — | — | — | — | — | — | — | 63.8 | |
| Con. tag.LLM Backbone=GPT-4.1 Mini2025.09 | 50 | — | — | — | — | — | — | — | — | 66.7 |