Structured output generation for long-document QA on Loong Finance
84.42Spotlight Locating ASGPT-4o-mini
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-4o-miniModel Size=8B2026.03 | 84.42 | 70 | 80.4 | 67 | 77.38 | 40 | 65.35 | 18 | 78.08 | 51 | |
| Deepseek-R1Model Size=671B2026.03 | 84.27 | 62 | 78.97 | 55 | 75.42 | 34 | 74.4 | 35 | 78.18 | 46 | |
| GPT-4oModel Size=200B2026.03 | 84.1 | 73 | 80.53 | 60 | 81.5 | 50 | 64.3 | 25 | 79.32 | 54 | |
| Qwen2-7B-Instruct (LITECOST)Model Size=7B2026.03 | 83.97 | 62 | 81.55 | 59 | 81 | 43 | 67.98 | 18 | 79.93 | 48 | |
| Qwen2.5-14B-InstructModel Size=14B2026.03 | 83.74 | 57 | 82.12 | 56 | 69.96 | 24 | 66.41 | 10 | 75.6 | 38 | |
| LLaMA-3.2-3B-Instruct (LITECOST)Model Size=3B2026.03 | 81.27 | 53 | 78.08 | 49 | 78.34 | 36 | 64.75 | 16 | 76.95 | 40 | |
| Qwen2-7B-Instruct (Base)Model Size=7B2026.03 | 63.1 | 36 | 67.85 | 37 | 60.83 | 18 | 52.25 | 9 | 62.1 | 26 | |
| LLaMA-3.1-8B-InstructModel Size=8B2026.03 | 55.03 | 20 | 51.6 | 15 | 51.5 | 4 | 44.75 | 2 | 51.32 | 10 | |
| LLaMA-3.2-3B-Instruct (Base)Model Size=3B2026.03 | 49.9 | 16 | 52.1 | 14 | 47.89 | 7 | 46.85 | 6 | 49.37 | 11 |