Code Generation on MBPP (MBPP Acc %)
59.8MBPP AccuracySVD
Evaluation Results
| Method | Links | |
|---|---|---|
| SVDModel Size (Billion)=6.742025.07 | 59.8 | |
| FWSVDModel Size (Billion)=6.742025.07 | 59.8 | |
| ASVDModel Size (Billion)=6.742025.07 | 59.8 | |
| AFMModel Size (Billion)=6.742025.07 | 59.8 | |
| IMPACTModel Size (Billion)=6.742025.07 | 59.8 | |
| SVDModel Size (Billion)=6.142025.07 | 54.8 | |
| FWSVDModel Size (Billion)=4.772025.07 | 54.8 | |
| ASVDModel Size (Billion)=4.102025.07 | 51.9 | |
| IMPACTModel Size (Billion)=3.582025.07 | 51.6 | |
| IMPACTModel Size (Billion)=2.642025.07 | 49.7 | |
| AFMModel Size (Billion)=3.762025.07 | 49.5 | |
| AFMModel Size (Billion)=2.812025.07 | 49.2 | |
| SVDModel Size (Billion)=3.132025.07 | 46.8 | |
| ASVDModel Size (Billion)=3.432025.07 | 44.7 | |
| FWSVDModel Size (Billion)=2.942025.07 | 44.2 | |
| ASVDModel Size (Billion)=2.772025.07 | 43.9 | |
| AFMModel Size (Billion)=2.042025.07 | 43.1 | |
| IMPACTModel Size (Billion)=1.902025.07 | 42.6 | |
| FWSVDModel Size (Billion)=2.192025.07 | 36.2 | |
| SVDModel Size (Billion)=2.372025.07 | 35.7 | |
| ASVDModel Size (Billion)=2.112025.07 | 32.3 | |
| IMPACTModel Size (Billion)=1.302025.07 | 28 | |
| AFMModel Size (Billion)=1.422025.07 | 26.7 | |
| FWSVDModel Size (Billion)=1.542025.07 | 23.8 | |
| SVDModel Size (Billion)=1.692025.07 | 19.6 | |
| IMPACTModel Size (Billion)=0.842025.07 | 16.4 | |
| ASVDModel Size (Billion)=1.452025.07 | 16.1 | |
| AFMModel Size (Billion)=0.922025.07 | 12.2 | |
| FWSVDModel Size (Billion)=0.962025.07 | 9.5 | |
| SVDModel Size (Billion)=1.092025.07 | 6.1 |