Structured Data Extraction and Reasoning on Loong
73.95Spotlight Locating Accuracy (AS)GPT-4o
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-4oContext Length=128k, Structured Data (SD)=false2026.03 | 73.95 | 62 | 50.5 | 28 | 44.29 | 9 | 57.05 | 28 | 53.47 | 26 | |
| LITECOST (GPT-4o-mini)Context Length=128k, Structured Data (SD)=true, Base Model=GPT-4o-mini2026.03 | 63.23 | 44 | 53.04 | 38 | 59.63 | 21 | 55.98 | 26 | 58.02 | 29 | |
| LITECOST (GPT-4o)Context Length=128k, Structured Data (SD)=true, Base Model=GPT-4o2026.03 | 62.11 | 33 | 63.27 | 41 | 68.06 | 29 | 53.52 | 22 | 62.51 | 30 | |
| GPT-4o-miniContext Length=128k, Structured Data (SD)=false2026.03 | 59.46 | 49 | 51.9 | 27 | 34.55 | 4 | 64.28 | 39 | 49.25 | 24 | |
| Claude3.5-SonnetContext Length=200k, Structured Data (SD)=false2026.03 | 58.45 | 49 | 54.21 | 25 | 45.77 | 7 | 43.92 | 25 | 48.85 | 23 | |
| LITECOST (Qwen2-72B-Ins)Context Length=128k, Structured Data (SD)=true, Base Model=Qwen2-72B-Ins2026.03 | 57.3 | 31 | 54.8 | 38 | 61.46 | 23 | 46.36 | 16 | 55.7 | 25 | |
| Qwen2-72B-InsContext Length=128k, Structured Data (SD)=false2026.03 | 54.17 | 36 | 42.38 | 20 | 36.71 | 4 | 47.76 | 18 | 43.29 | 15 | |
| LITECOST (Claude3.5-Sonnet)Context Length=200k, Structured Data (SD)=true, Base Model=Claude3.5-Sonnet2026.03 | 47.6 | 34 | 54.64 | 41 | 66.95 | 31 | 50.45 | 23 | 57.32 | 31 |