Multi-turn Interaction-based Problem Solving on MINT-Bench 1.0 (test)
11.76Code Generation ScoreLLAMA PRO - INSTRUCT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LLAMA PRO - INSTRUCTinteraction turns (k)=52024.01 | 11.76 | 29.1 | 9.81 | 14.68 | |
| Mistral-Instruct-v0.1interaction turns (k)=52024.01 | 6.62 | 34.33 | 8.54 | 13.99 | |
| CodeLLaMA-7B-Instructinteraction turns (k)=52024.01 | 2.21 | 17.16 | 7.91 | 8.7 | |
| AgentLM-7Binteraction turns (k)=52024.01 | 1.47 | 9.7 | 8.86 | 7.34 | |
| LLaMA2-7B-Chatinteraction turns (k)=52024.01 | 0 | 0 | 13.61 | 7.34 |