Code Generation on DS-1000 (Library Proficiency)
68.6Matplotlib ScoreLLaMA3.1-8B (DomRetriever) + GPT-4o (Code Gen)
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| LLaMA3.1-8B (DomRetriever) + GPT-4o (Code Gen)Size=-2026.03 | 68.6 | 64.2 | 50.8 | 49.1 | 50.7 | 56.4 | 53.3 | 58.6 | |
| GPT-3.5-TurboSize=-2026.03 | 65.8 | 32.7 | 30.2 | 36.8 | 39.6 | 40 | 42.2 | 39.4 | |
| GPT-4oSize=-2026.03 | 65.2 | 56.8 | 41.9 | 47.1 | 48.1 | 50.4 | 46.7 | 51 | |
| LLaMA3.1-8B (DomRetriever) + LLaMA3.3-70B (Code Gen)Size=70B2026.03 | 63.5 | 49.3 | 44.6 | 41.4 | 44 | 46.3 | 49.2 | 47.9 | |
| DomAgent (Qwen2.5-7B)Size=7B2026.03 | 62.5 | 37.9 | 32.7 | 33.5 | 30.6 | 29.1 | 36.9 | 39.2 | |
| DomAgent (LaMA3.1-8B)Size=8B2026.03 | 62.5 | 43.1 | 33.1 | 29.2 | 31.5 | 29.3 | 37.1 | 40.5 | |
| LLaMA3.3-70BSize=70B2026.03 | 60.2 | 40 | 36.5 | 37.8 | 40.1 | 42 | 43.5 | 40.3 | |
| MagicoderS-CL (large-scale fine-tuning)Size=7B2026.03 | 55.9 | 40.6 | 40.4 | 28.4 | 28.8 | 35.8 | 37.6 | 37.5 | |
| CodeLLaMA-PythonSize=7B2026.03 | 55.3 | 34.5 | 19.9 | 16.4 | 22.3 | 17.6 | 28.5 | 28 | |
| WizardCoderSize=15B2026.03 | 55.2 | 33.6 | 26.2 | 16.7 | 22.4 | 24.9 | 26.7 | 29.2 | |
| LLaMA3.1-8bSize=8B2026.03 | 55.1 | 36.2 | 26.4 | 18.3 | 24.2 | 22.5 | 30.1 | 30.4 | |
| Magicoder-CLSize=7B2026.03 | 54.6 | 34.8 | 24.7 | 19 | 25 | 22.6 | 28.9 | 29.9 | |
| Qwen2.5-7BSize=7B2026.03 | 54.2 | 34.9 | 25.3 | 16.8 | 23.1 | 21.4 | 29.7 | 29.3 | |
| WizardCoder-CLSize=7B2026.03 | 53.5 | 34.4 | 25.7 | 15.2 | 21 | 24.5 | 28.9 | 28.4 | |
| StarCoderSize=15B2026.03 | 51.7 | 29.7 | 21.4 | 11.4 | 20.2 | 29.5 | 24.5 | 26 |