Instruction Satisfaction on CODI
94.6ISRDVR (coldstart)
Evaluation Results
| Method | Links | |
|---|---|---|
| DVR (coldstart)Base Model=Llama3.1 8B2024.10 | 94.6 | |
| DVR (warmstart)Base Model=Llama3.1 8B2024.10 | 94.6 | |
| DVR (warmstart)Base Model=Llama3 8B2024.10 | 94.4 | |
| DVR (coldstart)Base Model=Llama3 8B2024.10 | 94.2 | |
| DVR (warmstart)Base Model=Mistral 7B2024.10 | 93.2 | |
| CRITICBase Model=Llama3 8B2024.10 | 93 | |
| DVR (coldstart)Base Model=Mistral 7B2024.10 | 92 | |
| CRITICBase Model=Llama3.1 8B2024.10 | 91.2 | |
| CRITICBase Model=Mistral 7B2024.10 | 88.6 | |
| ReActBase Model=Llama3.1 8B2024.10 | 81.8 | |
| Reject SampleBase Model=Llama3.1 8B2024.10 | 81.4 | |
| ReActBase Model=Llama3 8B2024.10 | 81 | |
| Reject SampleBase Model=Llama3 8B2024.10 | 80.8 | |
| ReActBase Model=Mistral 7B2024.10 | 80.4 | |
| Reject SampleBase Model=Mistral 7B2024.10 | 79.8 | |
| USCBase Model=Llama3 8B2024.10 | 70.2 | |
| ReflexionBase Model=Llama3 8B2024.10 | 70 | |
| ReflexionBase Model=Llama3.1 8B2024.10 | 69.8 | |
| USCBase Model=Llama3.1 8B2024.10 | 69.6 | |
| ReflexionBase Model=Mistral 7B2024.10 | 69.4 | |
| USCBase Model=Mistral 7B2024.10 | 69.2 | |
| VanillaBase Model=Mistral 7B2024.10 | 68.8 | |
| VanillaBase Model=Llama3 8B2024.10 | 68.8 | |
| VanillaBase Model=Llama3.1 8B2024.10 | 68.6 | |
| BSMBase Model=Llama3.1 8B2024.10 | 68.6 | |
| BSMBase Model=Llama3 8B2024.10 | 68.4 | |
| BSMBase Model=Mistral 7B2024.10 | 68.2 |