Option Query Language (OQL) Generation on OQL Query Quality (evaluation set)
95VRGPT-4.1-Mini
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT-4.1-MiniSize=Large2026.03 | 95 | 72.1 | 60.5 | |
| GPT-4.1Size=Large2026.03 | 93.5 | 77 | 69.8 | |
| LLaMA-3.1-8BSize=Small2026.03 | 92 | 58.2 | 43.2 | |
| Gemini-2.5 FlashSize=Large2026.03 | 87.5 | 74.3 | 60.6 | |
| DeepSeek-V3Size=Large2026.03 | 87 | 82.2 | 66.4 | |
| Qwen3-8BSize=Small2026.03 | 78 | 80.8 | 59.3 | |
| Qwen2.5-Coder-7BSize=Small2026.03 | 76 | 76.3 | 55.3 | |
| Qwen3-4BSize=Small2026.03 | 71.5 | 67.1 | 47.6 | |
| DeepSeek-Coder-6.7BSize=Small2026.03 | 66 | 65.9 | 54.5 |