Classification (8 classes) on Twitter customer-support requests
96.88Accuracy (8 classes)DSPy GEPA
Evaluation results
| Method | Links | |
|---|---|---|
| DSPy GEPATrain Tokens (Base Model)=1636241 / 10261, Train Tokens (Reflection Model)=107506 / 94565, Inference Tokens (Base Model)=184569 / 8792026.04 | 96.88 | |
| AIRTrain Tokens (Embed Model)=7282 / -, Train Tokens (Base Model)=54704 / 1392, Train Tokens (Reflection Model)=10301 / 10981, Inference Tokens (Base Model)=55392 / 1742026.04 | 95.31 | |
| Fine-tuningTrain Tokens (Base Model)=54153 / 1362, Inference Tokens (Base Model)=6752 / 1792026.04 | 90.63 | |
| TextGradTrain Tokens (Base Model)=501625 / 394, Train Tokens (Reflection Model)=876468 / 348127, Inference Tokens (Base Model)=475808 / 2152026.04 | 46.87 | |
| DSPy MIPROv2Train Tokens (Base Model)=1676737 / 23626, Inference Tokens (Base Model)=75257 / 8862026.04 | 31.25 | |
| DSPy BootstrapFewShot (#16)Train Tokens (Base Model)=48874 / 562, Inference Tokens (Base Model)=80032 / 9342026.04 | 18.75 | |
| KNNInference Tokens (Base Model)=9008 / 1832026.04 | 9.37 | |
| Initial promptInference Tokens (Base Model)=6752 / 1802026.04 | 0 |