Classification on DIAGNO (test)
85.5AccuracyTF-IDF + CART
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| TF-IDF + CARTd=72025.09 | 85.5 | 76.7 | |
| ACTTree Depth (d)=4, Optimization Steps (k)=20, LLM Backbone=GPT-4.1 Mini2025.09 | 82.3 | 81.8 | |
| Rule FitLLM Backbone=Avg2025.09 | 80.2 | 79.3 | |
| TF-IDF + CARTTree Depth (d)=3, LLM Backbone=Avg2025.09 | 78.8 | 78.9 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=GPT-4.1 Mini2025.09 | 77.3 | 75.2 | |
| Con. tag.LLM Backbone=GPT-4.1 Mini2025.09 | 75.3 | 78.6 | |
| ACTModel=Nano, d=5, k=102025.09 | 73.7 | 54.2 | |
| ACTModel=Nano, d=8, k=102025.09 | 73.4 | 56.1 | |
| ACTModel=Gemma, d=5, k=102025.09 | 72.8 | 52.7 | |
| ACTTree Depth (d)=4, Optimization Steps (k)=20, LLM Backbone=Avg2025.09 | 72.8 | 74.2 | |
| ACTModel=Nano, d=4, k=102025.09 | 72.7 | 54.1 | |
| ACTModel=Gemma, d=4, k=102025.09 | 72.6 | 41.5 | |
| ACTModel=Gemma, d=8, k=102025.09 | 71.9 | 59.4 | |
| DSPyModel=Nano2025.09 | 70.7 | 45.9 | |
| ACTTree Depth (d)=4, Optimization Steps (k)=20, LLM Backbone=GPT-4.1 Nano2025.09 | 70.3 | 70.8 | |
| ACTTree Depth (d)=4, Optimization Steps (k)=20, LLM Backbone=Qwen3 4b2025.09 | 70.3 | 73.5 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=GPT-4.1 Nano2025.09 | 68.7 | 61.2 | |
| CoTModel=Nano2025.09 | 68.6 | 48.1 | |
| TextGradModel=Nano2025.09 | 68.6 | 48.1 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=Avg2025.09 | 68.6 | 67.5 | |
| ACTTree Depth (d)=4, Optimization Steps (k)=20, LLM Backbone=Gemma3 4b2025.09 | 68.3 | 70.8 | |
| TextGradModel=Gemma2025.09 | 67 | 54.2 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=GPT-4.1 Nano2025.09 | 66.8 | 67 | |
| DSPyModel=Gemma2025.09 | 66.5 | 57.9 | |
| TEXTGRADLLM Backbone=GPT-4.1 Mini2025.09 | 65.8 | 55.5 | |
| Con. tag.LLM Backbone=Gemma3 4b2025.09 | 65.5 | 63.9 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=Gemma3 4b2025.09 | 65.3 | 66.5 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=Avg2025.09 | 65.1 | 60.9 | |
| ACTTree Depth (d)=3, Optimization Steps (k)=10, LLM Backbone=Qwen3 4b2025.09 | 64.8 | 61.3 | |
| Con. tag.LLM Backbone=Avg2025.09 | 64.7 | 66.3 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=GPT-4.1 Mini2025.09 | 64.5 | 59.4 | |
| TEXTGRADLLM Backbone=GPT-4.1 Nano2025.09 | 64.5 | 53 | |
| TEXTGRADLLM Backbone=Avg2025.09 | 64.4 | 56.3 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=Gemma3 4b2025.09 | 64 | 68.2 | |
| TEXTGRADLLM Backbone=Qwen3 4b2025.09 | 64 | 60 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=GPT-4.1 Nano2025.09 | 63.3 | 54.7 | |
| DSPyEvaluation Protocol=BFSR, 8 demos, LLM Backbone=Qwen3 4b2025.09 | 63.3 | 54.9 | |
| TEXTGRADLLM Backbone=Gemma3 4b2025.09 | 63.3 | 56.7 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=Qwen3 4b2025.09 | 63.2 | 41.7 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=Avg2025.09 | 62.3 | 54.3 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=Gemma3 4b2025.09 | 61.5 | 69.5 | |
| CoTEvaluation Protocol=0-shot, LLM Backbone=GPT-4.1 Mini2025.09 | 61.2 | 51.4 | |
| Con. tag.LLM Backbone=GPT-4.1 Nano2025.09 | 59.3 | 58.1 | |
| Con. tag.LLM Backbone=Qwen3 4b2025.09 | 58.8 | 64.7 | |
| CoTModel=Gemma2025.09 | 50.1 | 55 |