LLM data comprehension on Real-world data
74.3JSON Validity RateGPT-5.1-codex
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GPT-5.1-codexFamily=OpenAI2026.04 | 74.3 | 71.4 | 2.9 | 35 | |
| GPT-5.1Family=OpenAI2026.04 | 71.4 | 62.9 | 8.6 | 35 | |
| GPT-5-miniFamily=OpenAI2026.04 | 71.4 | 71.4 | 0 | 35 | |
| Gemini 3 ProFamily=Google2026.04 | 68.6 | 68.6 | 0 | 35 | |
| Kimi K2Family=Moonshot2026.04 | 62.9 | 68.6 | 5.7 | 35 | |
| Qwen3 32BFamily=Alibaba2026.04 | 60 | 57.1 | 2.9 | 35 | |
| Overall (10 models)2026.04 | 59.1 | 59.4 | 0.3 | 350 | |
| Llama 3.3 70BFamily=Meta2026.04 | 54.3 | 54.3 | 0 | 35 | |
| Llama 3.1 8BFamily=Meta2026.04 | 45.7 | 48.6 | 2.9 | 35 | |
| GPT-OSS 120BFamily=Open-src2026.04 | 42.9 | 45.7 | 2.9 | 35 | |
| Llama 4 ScoutFamily=Meta2026.04 | 40 | 45.7 | 5.7 | 35 |