Event Logical Reasoning on BizFinBench v2
73.34AccuracyFine-tuning
Evaluation results
| Method | Links | |
|---|---|---|
| Fine-tuningTrain Tokens (input / output) Base Model=247908 / 7560, Inference Tokens (input / output) Base Model=28477 / 8562026.04 | 73.34 | |
| KNNInference Tokens (input / output) Base Model=41249 / 8952026.04 | 51.67 | |
| AIRTrain Tokens (input / output) Embed Model=65923 / -, Train Tokens (input / output) Base Model=453192 / 3008, Train Tokens (input / output) Reflection Model=58716 / 65583, Inference Tokens (input / output) Base Model=292237 / 10362026.04 | 51.67 | |
| DSPy GEPATrain Tokens (input / output) Base Model=1998619 / 23064, Train Tokens (input / output) Reflection Model=37111 / 26706, Inference Tokens (input / output) Base Model=170138 / 13962026.04 | 46.67 | |
| Initial promptInference Tokens (input / output) Base Model=28477 / 10362026.04 | 45 | |
| DSPy MIPROv2Train Tokens (input / output) Base Model=1616492 / 38339, Inference Tokens (input / output) Base Model=72278 / 14562026.04 | 43.33 | |
| TextGradTrain Tokens (input / output) Base Model=728850 / 78210, Train Tokens (input / output) Reflection Model=7589 / 112655, Inference Tokens (input / output) Base Model=51157 / 9762026.04 | 43.33 | |
| DSPy BootstrapFewShot (#3)Train Tokens (input / output) Base Model=17001 / 344, Inference Tokens (input / output) Base Model=72338 / 14562026.04 | 35 |