Coding on LiveCodeBench generation_lite LeetCode easy (test)
27.4AccuracyZero-Shot Classifier
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Zero-Shot ClassifierModel=Mistral-7B2024.06 | 27.4 | 70.2 | 66.2 | |
| Lora + PromptModel=Mistral-7B2024.06 | 27.4 | 21.4 | 85.1 | |
| Zero-Shot ClassifierModel=Mistral-7B-Instruct2024.06 | 21 | 52.7 | 47.1 | |
| Lora + PromptModel=Mistral-7B-Instruct2024.06 | 21 | 56.1 | 70.2 | |
| Zero-Shot ClassifierModel=LLaMa-2-7B2024.06 | 3.2 | 41 | 56.9 | |
| Lora + PromptModel=LLaMa-2-7B2024.06 | 3.2 | 46.4 | 80 |