Financial Time Series Reasoning on FinTSR-Bench (test B)
3DrawdownPhi-3.5-mini
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Phi-3.5-miniType=Language Model2026.05 | 3 | 1.1 | 12.4 | 5.7 | 0.2 | 1.1 | 1.6 | 0.5 | 1.3 | 1.8 | 2.9 | |
| TimeMQA-MistralType=TS Language Model2026.05 | 3.5 | 2.8 | 5.4 | 0.9 | 3 | 10.1 | 10.9 | 10.8 | 7 | 3.9 | 5.8 | |
| Mistral-7BType=Language Model2026.05 | 4 | 1.1 | 0.5 | 0 | 37.6 | 46.3 | 2.3 | 40.7 | 24.6 | 36.6 | 19.4 | |
| TimeMQA-QwenType=TS Language Model2026.05 | 5.5 | 0.8 | 2.9 | 0.6 | 0.7 | 1.7 | 0.8 | 2.1 | 0.7 | 0.8 | 1.7 | |
| TimeMQA-LlamaType=TS Language Model2026.05 | 9.6 | 8.6 | 7.9 | 6.1 | 15.6 | 20.1 | 16.8 | 7.6 | 7.9 | 11.9 | 11.2 | |
| Random2026.05 | 25 | 33.3 | 20 | 33.3 | 50 | 50 | 50 | 50 | 50 | 50 | 41.2 | |
| Llama-3.1-8BType=Language Model2026.05 | 59.9 | 30.5 | 73.4 | 2.8 | 46.5 | 48.3 | 45.2 | 48.8 | 26.9 | 31.7 | 41.4 | |
| Qwen2.5-7BType=Language Model2026.05 | 71.5 | 37 | 73.3 | 30.3 | 53.6 | 48.6 | 58.8 | 46.7 | 50.2 | 46.6 | 51.7 | |
| TimeOmni-1-7BType=TS Reasoning Model2026.05 | 75 | 35.6 | 53.1 | 31.9 | 51.7 | 50.7 | 57.6 | 55.8 | 48.3 | 49 | 50.9 | |
| Gemma-2-9BType=Language Model2026.05 | 78.9 | 0 | 58.4 | 30.9 | 47.4 | 36.9 | 28 | 16.4 | 0.1 | 0 | 29.7 | |
| Qwen2.5-7B (w/o CoT)Training=SFT on FinTSR-Bench, Chain of Thought (CoT)=false2026.05 | 79.6 | 41.6 | 75.9 | 36.2 | 50 | 51.1 | 68.7 | 58.2 | 46.9 | 48.1 | 55.6 | |
| Qwen2.5-7B (w/ CoT)Training=SFT on FinTSR-Bench, Chain of Thought (CoT)=true2026.05 | 97.1 | 52.6 | 93.5 | 36 | 50.7 | 49.4 | 59.1 | 46.8 | 49.9 | 51.6 | 58.7 | |
| FinSTaRChain of Thought (CoT)=true2026.05 | 99.6 | 92.2 | 99.5 | 89 | 55.6 | 80 | 75 | 77.3 | 50.1 | 64.3 | 78.3 |