Code Generation on MBPP 1-shot
48.46Pass@16TOP
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| TOPModel Size=7B, Shots=1-shot2025.08 | 48.46 | 51.48 | 54 | |
| MTPModel Size=7B, Shots=1-shot2025.08 | 45.66 | 48.03 | 50.1 | |
| NTPModel Size=7B, Shots=1-shot2025.08 | 44.64 | 46.95 | 49.2 | |
| MTPModel Size=1.8B, Shots=1-shot2025.08 | 44.29 | 46.41 | 48.2 | |
| NTPModel Size=1.8B, Shots=1-shot2025.08 | 41.34 | 44.16 | 46.7 | |
| TOPModel Size=1.8B, Shots=1-shot2025.08 | 40.71 | 43.16 | 44.8 | |
| DS-MTPModel Size=7B, Shots=1-shot2025.08 | 39.01 | 42.13 | 44.9 | |
| DS-MTPModel Size=1.8B, Shots=1-shot2025.08 | 27.81 | 30.33 | 31.9 |